microsoft word 0a_editorial_3_1 international electronic journal of elementary education vol. 3, issue 1, october, 2010. issn:1307-9298 copyright © iejee www.iejee.com v dear iejee reader, with this special issue of international electronic journal of elementary education (iejee) you will get the opportunity to be informed about the state-of-the-art in the field of out of school education. dr. christian w. beck and dr. thomas spiegler are two of the active researchers within the field in their countries. they have a close contact with many other active researchers internationally. their collaborative research network within the field of out of school education challenges the many existing conceptions of education. as the editors of this special issue of iejee, they have succeeded in getting researchers from different countries together. by focusing on social justice, human rights, respect for the individual, parental rights and fundamentalism in the home education movement, the authors inform us about the contemporary approaches and perspectives, but also about the challenges the field of out of school education is being facing in different countries. the content of this special issue is not only about education outside the public or private sector. articles present several principal approaches to education that all of them can be considered as important questions for a societal debate in each society. i express my gratitude to dr. beck and dr. spiegler and all the other contributors to this special issue of iejee. i am also grateful to dr. turan temur, dr. gökhan özsoy and other editorial reviewers for positive attitudes and pursuing extra job for the sake of research based knowledge development and dissemination of the results through iejee. prof. dr. kamil özerk editor in chief microsoft word !iejee_6_2_editorial international electronic journal of elementary education, 2014, 6(2) issn:1307-9298 copyright © iejee www.iejee.com dear iejee reader, it’s a great pleasure for me to present you one of the most comprehensive issues of international electronic journal of elementary education; volume 6, issue 2. in eleven papers, eighteen scholars are addressing and exploring important educational topics like academic self-concept, developing reading skills among children with reading difficulties, theory of mind, self-concept, gender-role orientation, student’s perspective on leadership, children’s enjoyment, interest, and comprehension of graphic novels compared to heavily-illustrated and traditional novels, the role of nonchallenging education and teacher control as factors for marginalization of students in diverse settings, teachers’ thinking styles and their attitutes to teaching, mathematic teacher candidates’ problem solving skills, a comparative study of education of linguistic minority students in los angeles and oslo, the role of drawings in evoking interest for a carrier as math teacher and underrepresentation of male in teacher education. international electronic journal of elementary education (iejee) is an open access journal. papers published in iejee are accessible for everybody. iejee is an international, multi-disciplinary, peer-reviewed journal that is online published three times in a year. one of our issues is an annual special issue with a clear focus on a given current and important topic. the topic of the next special issue of iejee is reading fluency and the special issue editor is dr. timothy rasinski, professor of literacy education at kent state university, usa. iejee is abstracting or indexing in scopus, proquest, doaj, education research complete™, google scholar, edna, index copernicus, norwegian social science data services, newjour, open j-gate, academic journals database. i thank to all of the authors of this issues. i also would like to express my deep gratitude to dr. turan temur, dumlupinar university, turkey; dr. gökhan özsoy, aksaray university, turkey; and hasan tabak, gazi university, turkey; h. gül kuruyer, aksaray university, turkey; mustafa bakir, dumlupinar university, turkey for their tirelessly works for coordination and quality assurance of submission, correspondence with authors, reviewing, revision, and publishing processes. sincerely, kamil özerk, university of oslo editor in chief microsoft word editorial international electronic journal of elementary education vol. 2, issue 3, july, 2010.   iv   dear iejee readers, iejee presents four articles in this number. kristína žoldošová and iveta matejovičová from trnava university, slovakia adress the issues of objectively evaluation of development of skills by elementary school children in science education. they discuss and present sound ideas about development and evaluation of basic science process skills, causal thinking and empirical thinking, child’s right to education is an universal important topic. glenda jackson, monash university, australia and sonia allan from university of adelaide. australia take up a the fundamental elements in examining a child’s right to education. they present an informative paper about several aspects of home education. ‘out of school education’ will be the main theme of the next special issue of iejee. jackson and allan’s paper gives us the scientific, social and legislative frames surrounding the home education in australia. their paper is a valuable contribution to the field, particularly for those researchers who are interested in highligting the home education-movement from a comparative and international perspective. gökhan baş and ömer beyhan from selcuk university, konya, turkey investigated in an experimental studied the impact of multiple intelligences supported project-based teaching method and traditional foreign languageteaching environment on 5th graders’ achievement and their attitude towards the english as a foreign language subject. their experimental research has shown that children taught in english through multiple intelligences supported project-based teaching outperformed the students who were taught in a traditional foreign language-teaching environment with regard to the development of english language skills, attitudes towards teaching and motivation to english teaching. international electronic journal of elementary education vol.2, issue 3, july, 2010 v   eyyup coşkun and murat alkan of mustafa kemal university, hatay, turkey conducted a small-scaled study on the impact of the turkish national curriculum reform of 2005 on teaching-learning processes in the classroom. according the offical turkish national curriculum document the teaching-learning processes must be based on a constructivistic and childcentered educational perspectives. the study revealed that despite the teachers’ lack of loyalty to the official curricular prescriptions, their teaching practices are child-centered and the students considered central elements of the national curriculum as positive. as the reader may see, coşkun and alkan address important issues related to the discrepancy between the formal, practiced and experienced curriculum. the gaps between different curriculum levels are educationally challenging and need to be addressed more broadly and in a way that can give the practioners and the policy makers informative feedback. prof. dr. kamil özerk editor-in-chief international electronic journal of elementary education, 2016, 8(4) issn:1307-9298 copyright © iejee www.iejee.com dear iejee reader, in this number of the international electronic journal of elementary education, you’ll find ten articles. they are addressing several important topics: stem-education, teacherstudent relationship, pisa, professionalism in early childhood education, child-centered reading education, creative writing, math education, the challenges for math teachers, teaching english as a third language and factors related to reading comprehension. the first one is about the effects of an inquiry-based didactic method for preschool science in a real practical classroom setting. this is an impressing study on teaching science in early childhood in belgian context. the article highlights the important aspects of stemeducation based on problem-solving and inquiry based teaching and learning. the second article is about teacher-student relationship and how the students and their teachers reciprocally construe each others personality in teaching-learning context. the article has a comprehensive literature review and an empirical part. on the basis of their findings the two authors discuss how different categories of students’ perspectives and their teachers’ perspectives construct each others respectively ‘ideal teacher profile’ and ‘ideal student profile’. although this study is not the only one, it’s one of the original studies on the topic. the third article is also addressing an international theme: pisa. three researchers from mugla sitki kocman university explore the relationship among new literacies, reading, mathematics and science performance of turkish students in pisa 2012. as most of you may accept, pisa-results started to have much and sometimes too much impact on educational debates in many countries. as i once put it “pisa has tendency to act as a shadow minister of education in many countries”. to what extend this is true for turkish education, is a matter of research, but this article is a good piece of scientific work. the fourth article is from canada. two researchers brilliantly formulated a tittle for their study of early childhood educators’ narratives about ‘professionalism’. the tittle of “babysitter or professional” most likely will show many researchers around the world that they are not alone in their efforts to improving the competencies of those who the parents deliver their child to. more and more children in many countries use to bring longer and longer of their childhood together with early childhood educators. this article adds a critical dimension to the debate on professionalism in the early childhood education/care. the fifth article explores an approach to a child-centered reading intervention. two researchers tried out an interesting practical intervention strategy to help a low achieving reader to improve his reading skills. the article is based on a single-subject case study, and it’s very informative. the article presents ‘hand-on’ type of materials, steps and strategies and illustrates how a teachers can help a struggling 4th grader to improve his skills in reading and writing. the sixth article is about an experimental study on creative writing instruction program based on speaking activities. the researcher investigates the effect of a well-planned creative writing program on fourth-grade primary school students' creative writing http://www.iejee.com/ editorial / özerk vi achievements and writing attitudes. the experimental method based on the pre-test/posttest model was utilized. this can be a good contribution to the field of writing methodology. the seventh article is examining pre-service elementary school teacher beliefs and instructional practices in math class in indonesian context. this a single-subject study, but a quite impressive study. the three researchers’ findings on one hand indicate that the instructional practices do not necessarily reflect the beliefs that are held by a pre-service elementary school teacher. on the other hand, beliefs about the nature of mathematics influence more dominant than the other beliefs against instructional practices. the eight article is a study in which the researcher examined prospective middle school mathematics teachers’ problem-posing skills by investigating their ability to associate linear graphs with daily life situations in a turkish context. the study, in line with several other studies, argues that the teachers’ mathematics content knowledge is a crucial factor influencing the quality of mathematics teaching. the author goes further and explains what the mathematic content knowledge is comprised of. in his empirical study he identifies the challenges the prospective middle school math teachers have due to poor mathematics content knowledge and experience. the author stresses the importance of more research on problem-posing in mathematic teaching. introducing english as a third language teaching in early grades in kazakhstan is the topic of the ninth article. kazakhstani researchers discuss the ideological grounds of this language policy in the educational system of their own country. furthermore they address the role of the teachers in implementation of this policy at operational curriculum level. their study will be a good contribution to the widespread international trend of introducing english teaching at early stages of elementary education. the last article in this number of iejee is about an exploratory study of the relationships between reading comprehension competence, reading attitude and the vividness of mental imagery among 103 turkish 4th graders. the author employed a reading comprehension test, developed by himself, vividness of imagery questionnaire and elementary reading attitude survey, developed by other researchers. the findings of this study strengthen the existing belief in the field that there is a positive relationship between reading comprehension and attitudes to reading /recreational reading, and between reading comprehension and vividness of mental imagery. i want to express my deep gratitude to all the authors in this number of the international electronic journal of elementary education. i also want to thank to the blind-reviewers who did a great job for this young and promising journal. last but not least i want to thank to my closer colleagues and executive editors dr. turan temur, dr. gökhan özsoy and dr. hayriye gül kuruyer, and editorial assistants doctoral candidate hasan tabak, and doctoral candidate mustafa bakir for the demanding job they have accomplished. editor-in-chief kamil özerk, professor of education, university of oslo international electronic journal of elementary education, december 2016, 9(2), 239-242 issn:1307-9298 copyright © iejee www.iejee.com introduction to the special issue on autism spectrum disorders approaches to training, teaching, and treatment kamil özerk a shahla ala’i-rosales b svein eikeseth b a university of oslo, norway a university of north texas, usa b oslo and akershus university college of applied sciences, norway introduction the strength of an educational system rests in its design and implementation. part of what informs the design process is knowledge of student needs and valued goals and outcomes. as systems are designed, research evidence informs the implementation process. we are pleased to present a series of articles in this special issue that will aide special educators and administrators design and implement effective educational systems for children with autism spectrum disorders. so much of the progress achieved in improving the lives of children with autism can be attributed to the work of dr. ivar lovaas. in the lead article of this issue, k. özerk, vea, eikeseth, and m. özerk (2016) capture both the scholarly and personal dimensions of lovaas’ life. dr. lovaas conducted his work at a time that society was moving from warehousing people with disabilities to providing active treatment. dr. lovaas created a pathway for educating for children with autism. the scholarly and clinical descendants of lovaas write many of the articles in this special issue. özerk and colleagues provide a fascinating account of lovaas’ life. he was a complex and brilliant contributor to the field of behavior analysis and to the lives of so many children with autism and their families. one of the alarming features of our times is that are so many children with autism. in our second article, özerk (2016) provides a comprehensive view of prevalence, especially with regard to the demographic and geographic dimensions relevant to educational policy making. to illustrate some of these points in context, sugita (2106) provides an account of educational trends in one specific region of the world, california, usa. this paper reflects a keen understanding of the complexity surrounding prevalence and educational trends. similarly, roll-pettersson, olsson, and alai-rosales (2016) provide an analysis of proximal and distal barriers involved in the implementation of early and intensive behavioral interventions in educational systems serving children with autism. the issues explored in this article support the implementation science notion that successful training, teaching, and treatment are achieved through systemic planning and evidence based practice. http://www.iejee.com/ international electronic journal of elementaryeducationvol.9, issue 2, 239-242, december, 2016 240 given the background of asd rapidly rising across the globe, we turn to a series of articles by authors from several different regions of the world that address issues related to effective implementation and training. t. cihon, j. cihon, and bedient (2106) provide an example of an essential starting point: developing an agreed upon vocabulary to communicate between staff serving children with autism. in this article they share examples of a protocol successfully used in a very large school district in the midwestern united states. also working in an implementation setting, suhrheinrich, dickson, rieuth, lau and stahmer (2016) address variables related to classrooms and fidelity of implementation. specifically, they assess the degree of procedural fidelity of an evidencebased procedure, classroom pivotal response training (cprt), and found that, while successfully implemented, there were variations that suggest that the procedures might be adapted to particular types of classrooms. the theme of adaptation resonates very clearly in the article by j. leaf, j. cihon, r. leaf, mceachin, and taubman (2016). the authors provide educators with a set of “progressive dtt” guidelines for implementing responsive treatment that results in progress, as opposed to rigid, recipe based approaches that are implemented regardless of child responsiveness. behavioral interventions such as dtt are only as good as the fidelity of effective implementation. teikari and eikeseth (2016) provide a useful article that clearly illustrates the superior effects of practice and feedback under performance conditions as compared to lectures, instructional manuals, or no training. studies, such as this one that demonstrate, with social validity, effective means of insuring competent implementation are informative and necessary. j. leaf, r. leaf, mccray, lamkins, taubman, mceachin, and j. cihon (2016) employed a similar training model. the authors developed a tool to assess entire classrooms in terms of fidelity of behavioral interventions for children with autism. they went on to show that classrooms that received direct instruction, practice and feedback indicated improved classroom performance. the specific procedures that are implemented within a behavioral intervention program for children with autism are built upon a foundation of evidence. the next series of articles provides an exploration of the methods, concepts and outcomes of research in several areas relevant to teaching and treatment. miramotez and schwatrz (2016) lead this section with a concise review and a data based example of the use of physical activity during school activities. they present an informative method and measurement of the changes in on task behaviors of four young boys with autism before and after physical activities. this paper is followed by a review and discussion of the recognition and expression of emotion in children with autism (daou, hady and paulson, 2016). daou and colleagues summarize the treatment research and provide directions and recommendations for educators addressing this area of core deficit in children with autism. ingvarsson (2016) provides guidance in another area of core deficit, verbal behavior. this article is a tutorial on a behavior analytic conceptualization of language. ingvarsson provides educators with the basic concepts, terms, and approaches found in the analysis of verbal behavior from a skinnerian point of view. as this is one of the dominant perspectives in behavioral instruction, the tutorial will be a valuable aid for educators. eldevik, kazemi and elsky (2016) present an excellent example of a behavioral teaching program to teach one class of verbal behavior. the authors validate a practical and useful procedure for producing generalized use of past tense verbs. similarily, tekiniftar and olcay-gul (2016) provide a comparison of the effectiveness of two different prompting procedures when teaching academic skills to children with autism. finally, at a most necessary and practical level, toussaint & doa (2016) present evidence for a successful toilet training procedure. additionally, they offer practitioners a method for assessing family preference for any behavioral intervention selected for use by educators. introduction to the special issue / özerk, ala’i-rosales & eikeseth 241 we hope that you find this a useful collection of articles. we have attempted to address issues related to big picture policymaking as well as the everyday actions and goals that comprise education. all are important to improve the lives of people with autism. • • • kamil özerk is a professor of education at the university of oslo, faculty of educational sciences, oslo, norway. he teaches classes on teaching and learning, curriculum development, educational counseling, bilingualism, language revitalization, reading and autism. dr. özerk served as vise-chair for academic issues at the department of education and a member of steering committee of the norwegian language council, the norwegian government's advisory body in matters pertaining to the norwegian language and language planning. dr. özerk holding also a professor ii position at the sami university for applied sciences. his recent books are on bilingual development, reading comprehension strategies and autism and pedagogy. shahla ala’i-rosales is an associate professor in the department of behavior analysis at the university of north texas, where she teaches classes on ethics, autism intervention, parent training, applied research methods, and behavior change techniques. she is also the director of behavior analysis online (bao), a distance learning program for behavior analysts. her research and development activities take place at easter seals north texas autism treatment program, a service, training and research program for children with autism and their families. dr. alai-rosales served on the governing board of the behavior analysis certification board (bacb) and as a bacb subject matter expert on supervision and on ethics. shahla was the first behavior analyst to be awarded an onassis foundation fellowship for her work with families and continues international collaborations with colleagues in greece, italy, iceland, and sweden. svein eikeseth is a professor of psychology at the department of behavioral science, oslo and akershus university college of applied sciences. he has a ph.d. from the university of kansas and has been a postdoctoral researcher at the university of california, los angeles (ucla). dr. eikeseth is the director of several major international research projects, and has made important contributions the study of autism and to the field of behavior analysis. eikeseth has published numerous scientific articles, book chapter and books. he is a consultant for the associanzione pianeta autismo, rome, italia, research and clinical director the uk young autism project, director for banyan center, stockholm, sweden, and a consultant for the institute of child development, gdansk, poland. international electronic journal of elementaryeducationvol.9, issue 2, 239-242, december, 2016 242 www.iejee.com this page is intentionally left blank http://www.iejee.com/ microsoft word iejee_4_3_ryan international electronic journal of elementary education, 2012, 4(3), 563-571. issn:1307-9298 copyright © iejee www.iejee.com elementary supervision and the supervisor: teacher attitudes and inclusive education thomas g. ryan ∗∗∗∗ nipissing university, canada jodi gottfried nipissing university, canada received: june, 2011 / revised: april, 2012 / accepted: june, 2012 abstract: evidence has emerged which suggests that as a supervisor, the importance of knowing oneself, and knowing those that he or she is supervising, is vital to the success of the group. we argue that when conflicting values, attitudes, and beliefs are present amongst the members of the group over an issue (inclusion), or over the behaviours of a member (non-inclusive), the entire group can break down. therefore, to successfully implement a program, such as inclusion, knowing the attitudes of the staff is vital as a program such as this cannot be successful without positive support. keywords: supervision, inclusion, attitudes, self-knowledge introduction glickman, gordon, and ross-gordon (2010) describe the term supervision as a common vision ”that is developed collaboratively and brought into reality together. it forms connections that focus organizational and individual goals, objectives and efforts into an overarching strategy” (p. 56). capacity is built into the system as the supervisor encourages employees to reach their full potential, and helps to develop interpersonal relationships and a productive organizational culture (dessler, munro & cole, 2011). these outcomes are achieved by daily informed supervision. the supervisor, by definition, is someone who assists, guides, directs, and oversees the people that he/she is managing, however there is much more to being a supervisor than simply overseeing the jobs that people are doing (langton, robbins & judge, 2011). in order to be a successful supervisor, it is important that one understands not only their own beliefs towards education and approaches towards individuals and groups, but that they also understand the beliefs and approaches of their supervisees. ∗ thomas g. ryan, nipissing university, faculty of education, 100 college drive, north bay, ontario, p1b 8l7, phone: 1 705 474 3461 x4403 fax : 1 705 474 1947. e-mail : thomasr@nipissingu.ca international electronic journal of elementary education, 2012, vol.4, issue 3, 563-571 564 one issue that has been controversial in most schools for many years is the issue of inclusive education. “inclusion is primarily an overarching philosophy that advocates for the regular classroom as the first placement option for students with exceptionalities” (edmunds & edmunds, 2008, p. 24). arguably, the successfulness of inclusive education relies heavily upon the attitudes and beliefs of the teachers. as a supervisor, it is necessary to have an understanding of the supervisee’s belief systems in order to successfully implement inclusive education programs within a school because without the support of the teachers, these programs are destined to malfunction. know thyself in order to improve and advance our instruction, and hence improve student learning and experiences, we believe that we need to first look to ourselves to determine how our “present thinking, beliefs, and practices in the field of supervision interact with instruction and the assumptions about students… as learners” (glickman, gordon & ross-gordon, 2005, p. 78). we make judgements about our students on a daily basis whether we realize that we are doing it or not. we believe that these judgements can impact the way that we view that student and the level of interaction that we provide for that student, hence influencing the level and quality of learning that that student experiences. the johari window one way to recognize our personal thinking, beliefs, and practices is through the reflection that comes from the use of the johari window. the act of looking in or back is often misunderstood. bolton (2010) suggests, reflection is a state of mind, an ongoing constituent of practice, not a technique, or curriculum element. reflective practice can enable practitioners to learn from experience about themselves, their work, and the way they relate to home and work, significant others and wider society and culture. it gives strategies to bring things out into the open, and frame appropriate and searching questions never asked before. it can provide relatively safe and confidential ways to explore and express experiences otherwise difficult to communicate. (p. 3) the combination of reflection and a tool such as the johari window can be a prominent and constructive approach to understanding ourselves and our experiences. this tool “provides a graphic way to look at what we know and do not know about our behaviour” (glickman, gordon & ross-gordon, 2005, p. 101). this visual tool allows us to reflect on the different levels of self and the attributes that we allow to be known. four categories exist including the public self (open), the blind self, the private self (hidden), and the unknown self. the public (open) self is where both the supervisor and the supervisee are aware of the behaviours. the blind self occurs where the supervisees are aware of what behaviours take elementary supervision and the supervisor / ryan & gottfried 565 place, but the supervisor is unaware of these behaviours. the private (hidden) self is the knowledge that the supervisor has about him/herself but the supervisees do not. finally, the unknown self is the behaviours that both the supervisor and the supervisee are not aware of (glickman, gordon & ross-gordon, 2005). the reasoning behind the use of the johari window is that as supervisors, we cannot know if we are being effective for our team unless we know what we are doing (langton et al., 2011). in an educational setting, this remains true for both principals and for teachers. we must first know ourselves, before we can be effective supervisors and effective educators for our students. the johari window is based on the premise of communication and improving methods of communication through asking questions, or telling information (armstrong, 2006). this exposure allows us to understand where the other person is coming from, what background experiences they have to shape their position, and what beliefs, values, and ideas they hold (armstrong, 2006). this information cannot necessarily be shared without effective communication and provides great opportunities for understanding between a supervisor and a supervisee (langton et al., 2011). although the authors have described the johari window in terms of the supervisor, we believe that this process can begin with the supervisee. to first understand where i stand with my own beliefs, what i am willing to disclose to others, and what i choose to keep hidden, all effect my levels of communication with people, and how i choose to interact with people. we must first understand ourselves before we can begin to understand others. in order to understand myself, honesty and accuracy is vital (bolton, 2010; armstrong, 2006). if i am not being honest with my own beliefs and perceptions, cognitive dissonance can result between what i believe about myself, and what others believe about me. cognitive dissonance cognitive dissonance occurs when one has an image of themselves while others have a different image. “cognitive dissonance may allow individuals to make their implicit conceptions explicit, and examine their implicit conceptions from a new light” (olson, colasanti & trujillo, 2006, p. 282). to do this however requires an inner awareness and for those who lack this presence of mind cognitive dissonance may remain problematic. for instance, when considering the relationship between a supervisor and a supervisee, differing views of a person have the potential to cause confusion within a group. to illustrate this case in point, while on practicum, we had the opportunity to observe many teachers around the school and we were surprised to have a conversation with one colleague who had once been a teacher of ours. we did not have a good experience with this teacher, we found her to be intimidating, controlling, and basically, forbidding. this teacher, however, spent a great deal of time talking to us about the importance of building relationships with students, getting to know your students, and respecting your students. from our experiences with this teacher how we were taught, and what we observed, these were not traits that were being shown, and this teacher was in a state of cognitive dissonance. we found it difficult to then communicate with that teacher honestly, to ask questions, and to expect an honest response. we did not go back to that classroom to observe and instead moved on to observe other teachers who we felt comfortable with. this short experience showed us how important it is to know yourself and how your actions move towards other people. we did not feel comfortable having a conversation with this particular teacher because we did not feel that she was a person who was unable to act upon this cognitive dissonance. it was very obvious that she did not see herself as being intimidating to students. we feel then, that in a group dynamic, it is necessary to be in tune with your personality, your beliefs, and your values and to ensure that these beliefs and international electronic journal of elementary education, 2012, vol.4, issue 3, 563-571 566 values are being acted upon through your actions and words. if cognitive dissonance results and perceptions do not match, then the group dynamic can be affected because honesty and accuracy are not being practiced. we find it difficult to be honest around people who are not being honest themselves, and we believe that this situation also occurs within groups of people. this makes it necessary for both supervisors and supervisees to know themselves so that they can portray an honest image to their peers. attitudes and inclusive education inclusive education is one issue which is consistently being debated; it is also an issue that is heavily reliant upon the positive support of teachers. in order to demonstrate the importance of being aware of personal attitudes and being aware of attitudes of other group members, we will use the issue of inclusive education to show how important attitudes are to the successfulness of these programs. as well, we will then show how the supervisor can help the reluctant teachers to feel more comfortable in these situations. prevalence of teacher attitudes in the research studies which examine the attitudes of teachers towards inclusive education consistently state that the attitudes of teachers have a great effect on the successfulness of these programs. however, the present day literature does not consistently state that teachers are generally in favour of inclusion, or not in favour of inclusion indicating that a divide still exists amongst educators on whether students with special needs should be included within the regular education classroom. positive attitudes. several studies published within the last several years have indicated that teachers generally have a positive attitude towards inclusive education. subban and sharma (2005) concluded, “teachers in victorian schools may generally hold positive attitudes toward the inclusion of students with disabilities into mainstream settings” (p.9). mdikana, ntshangase, and mayekiso (2007) investigated pre-service educators’ attitudes towards inclusive education and revealed that 60% of the 22 students surveyed responded positively towards inclusive education. as stated in monsen and frederickson’s (2004) work, studies conducted by janney, snell, beers and raynes (1995) and stanovich (1999) both discovered that teachers had very positive attitudes towards inclusive education. these attitudinal orientations are key since, inclusion is a philosophy that brings students, families, educators, and community members together to create schools based on acceptance, belonging, and community. inclusionary schools welcome, acknowledge, affirm, and celebrate the value of all learners by educating them together in high-quality, age-appropriate general education classrooms in their neighbourhood schools. (salend, 2005, p. 36) negative attitudes. the need to examine teacher attitudes can be linked directly to classroom cohesion and socio-emotional climate within classrooms (mdikana, ntshangase, and mayekiso, 2007, p. 130). ryan (2009) suggested, the cohesion and climate within the inclusive classroom is partly due to the fact that students begin to notice differential treatment as early as the primary grades (k-3) and at about age 8 most children become aware of differences in others and in the manner the adult acts towards students. . . . children, at 8 can often now see clearly when people are being treated in a different way. hence the actions of a (negative) teacher are not only sensed by young children they understand often that a teacher has assumed an unhelpful (negative) attitude toward certain children with or without elementary supervision and the supervisor / ryan & gottfried 567 exceptionalities. obviously, the impact and the effects can be detrimental to the development of all students in this classroom who sense this treatment. (p. 17) hammond, helen, ingalls, and lawrence (2003) illuminated the attitudes of elementary school teachers toward inclusion and discovered “an overwhelmingly strong pattern of either a negative feeling or uncertainty toward inclusion,” although the majority of the respondents had inclusive education programs operating in their schools (hammond et al., 2003, p.3). another study conducted in the united arab emirates studied the attitudes of general education teachers toward inclusion. this study concluded that “general education teachers in the uae, in general, tend to have negative attitudes towards the inclusion of students with disabilities” (alghazo & gaad, 2004, p. 97). implications of teacher attitudes these studies show a general divide amongst educators concerning this issue of education. we have to then consider what the implications are for the programs, and for the supervision of groups of people with such differing opinions towards a single issue. first, we consider the implications for inclusive education programs considering the general attitudes of educators towards this issue. second, we consider what a supervisor may need to do in order to alter the attitudes of his or her supervisees considering the attitudes of the staff and the implications that can result from these attitudes. effects on student learning advocates for inclusive education have argued that full inclusion of students with special needs aids in the learning of both the exceptional student, as well as the regular student. dixon (2005) stated that “this learning is more genuine when students simply attend school together, rather than when students with disabilities visit regular classrooms” (p. 41). in addition to this, dixon argued that inclusion has the power to teach all involved, students and teachers, how to understand and accept people with disabilities as a part of life. this idea is very positive for advocates and supporters of inclusion. teachers who favour inclusion have the ability to expose and enlighten all students, exceptional or not, to what we can all teach each other. a concern stemming from this idea is that those teachers who do not view inclusion positively will not recognize or embrace these opportunities to learn from our differences. combs and harper (1967) recognized that if a teacher’s attitude towards a child is negative, that the behaviour of others could extend the students exceptionality rather than aiding the child. this idea is very troublesome when recognizing that negative attitudes towards students in the inclusive classroom still exist. effects on teaching as previously stated, the most obvious effect of negative attitudes towards inclusive education is that without teacher support, it is almost impossible to implement a successful inclusive education program. hammond et al. (2003) stated that an unsuccessful inclusive education program “would only strengthen negative attitudes of uncertainty regarding inclusion and its benefits” (p.4). it seems then, that negative attitudes result in an unsuccessful program, and an unsuccessful program results in strengthened negative attitudes (gottfried, 2007), showing a supervisor the importance of combating these attitudes within the group early on. this cyclical pattern could prove to be the downfall of inclusive education if educators’ views continue to hold negative opinions. therefore, it is necessary to first have a solid program in place, and then introduce and train teachers to be international electronic journal of elementary education, 2012, vol.4, issue 3, 563-571 568 successful in this program. with an unsuccessful program, teachers will only become more frustrated with the system and form a negative opinion of the concept as a result. kuyini and desai (2007) sought to discover if educator attitudes towards inclusive education and educators’ knowledge of inclusive practices were related to effective inclusive school practices. the study found that “attitudes towards inclusion . . . and knowledge of inclusive education . . . were predictive of effective teaching in inclusive classrooms” (kuyini & desai, p.109). this conclusion builds on the idea presented in burke and sutherland’s (2004) findings that without positive teacher attitudes towards inclusion, inclusive classrooms would not be successful because teachers would not have the commitment to implement inclusive practices. therefore, in order to present effective teaching in an inclusive classroom, a commitment to and a positive attitude towards inclusion must be present (gottfried, 2007). teachers possessing a positive view towards inclusive education can become contribute to negative effect on teaching and learning. talmor, reiter, and feigin (2005) concluded that teachers’ attitudes towards inclusion were most significantly linked to burnout as compared to the other background variables (p.212). the authors go on to state that those who had a positive perception towards inclusion also had high expectations and realized that they could not meet these high expectations (gottfried, 2007). they, therefore, experienced a higher rate of burnout as compared to those who did not have a positive attitude towards inclusion (talmor et al.). this trend is, perhaps, not a deficit in educator ability but a deficit of training and resources as the authors stated that “teachers seemed to feel that they hardly had any information at all, and once the student was enrolled in their classroom the help they received was minimal” (talmor et al., p.222). this result seems particularly disturbing with educational policies increasingly moving towards inclusive education. with the teachers who favour the practice most favourably leaving the profession, it seems that implementing a successful inclusive education program with committed teachers could be a challenge (gottfried, 2007). the role of the supervisor using this example of inclusive education as an issue in schools, one can easily see how important it is to know your own beliefs, as well as knowing the beliefs of those around you. our attitudes can easily influence others either positively or negatively and as a supervisor it’s necessary to use those influences in order to help the group in reaching a common goal. in this scenario, the common goal is to implement inclusive education programs. in a world where a divide exists amongst educators towards reacting positively or negatively to inclusive education, it is realistic to assume that a supervisor will encounter a divide amongst his or her staff. when realizing the consequences of allowing negativities to prevail, including unsuccessful programming, negative teaching practices, and disadvantages to student learning, it is necessary for a supervisor to take hold of the situation and to build positive attitudes within the school. know your staff the question remaining then is: how is a supervisor to turn around the personal attitudes of a staff in order to benefit from inclusive education programs? after reviewing the ideas behind knowing thyself and the consequences that can arise from differing attitudes, we believe that an obvious place for a supervisor to begin is with his or her own staff. to discuss through open communication the concerns that different staff members may have about inclusive education would allow for ideas to be shared, and, perhaps, for areas from the “private self” of the johari window to be moved to the “public self” so that we can all better understand not only ourselves, but each other. without open, honest, and accurate elementary supervision and the supervisor / ryan & gottfried 569 communication, the benefits of the johari window cannot be reached. we believe that it is the role of the supervisor to create an environment where the staff feel a level of comfort and trust that allows them to experience this open, honest reflection with their peers without fear of ridicule or disrespect. with this open, honest communication, some members may learn elements of their “blind self” that others were aware of that the individual was not. with these realizations, areas of cognitive dissonance may be rectified allowing for an individual to be aware of their own beliefs, behaviours, and actions. after open communication has been established, and the supervisor is aware of the attitudes and beliefs that are held by the staff, we believe that input from the staff as to why they hold the beliefs that they do, and what they need to help with the situation would be invaluable. we believe that sometimes finding a solution can be as simple as asking what needs to be done and acting on that. professional development the literature concerning teacher attitudes and inclusive education showed an obvious reason why concerns toward inclusive education have continued throughout the years. a lack of knowledge and a lack of training were consistently cited in the literature stemming the 1970’s to today. brooks and bransford (1971) felt that “from knowledge comes understanding and from understanding comes acceptance” (p. 259). they went on to conclude that reasons behind negative attitudes of the time stemmed from a lack of knowledge concerning the roles and functions surrounding special education. as well, van reusen, anthony k., shoho, alan r., barker, and kimberly s. (2001) concluded in their study concerning high school teacher attitudes towards inclusion that levels of special education training, knowledge, and experience in working with these students were related to teachers having a positive attitude toward inclusion. subban and sharma (2005) included in their discussion towards understanding educator attitudes towards inclusion that “the most negative views about inclusive education are held by teachers with little or no training in special education” (gottfried, 2008). if we assume that these concerns are general concerns amongst opponents to inclusive education, then this is exactly where the supervisor needs to begin: with knowledge and training. in education, knowledge and training generally grow via professional development which is essentially, “the continuous education of educators” (glickman, gordon & ross-gordon, 2010, p. 276). common characteristics exist as to what should be involved in successful professional development, these include, and are not limited to, the involvement of participants in planning, implementing, and evaluating the programs; developing programs based on school goals; and developing long-range plans (glickman, gordon & ross-gordon). using the example of inclusive education, first the staff would need to be involved in the planning to consider what their needs are; this could come through open communication of teachers and staff explaining what they need in order to be more successful in this program. school-wide goals would need to be established to foster an inclusive environment for all students throughout all classrooms in the school. this could include school wide events, combining classes for different activities and subjects, and making values of inclusion for all students, including race, religion, disability, etc. a daily occurrence. finally, these goals of inclusion would need to be factored into the long-range goals of the school to ensure that these values are continued and developed. considering that negative attitudes towards inclusive education can weaken the program and create strengthened negative attitudes, i feel that it is safe to assume that positive attitudes can strengthen the program and create strengthened positive attitudes. from the standpoint of the supervisor trying to implement professional development for his or her international electronic journal of elementary education, 2012, vol.4, issue 3, 563-571 570 staff, it would then be beneficial to use the positive attitudes of staff members to show those who are negative what benefits can come from inclusion and how inclusion can work in a classroom. the necessity of knowing ones staff is vitally important here as a supervisor would need to know not only the attitudes of the staff, but the needs of individual staff members and the leadership and training styles that best suit each person so that staff members are not left feeling uncomfortable or unwilling to participate when other staff members are highlighted for their teaching performances in inclusive classrooms. conclusion as a supervisor, the importance of knowing oneself, and knowing those that he or she is supervising, is vital to the success of the group. when conflicting values, attitudes, and beliefs are present amongst the members of the group over an issue, or over the behaviours of a member, we believe that the entire group can break down. for a supervisor to successfully implement a program, such as inclusion, knowing the attitudes of the staff is vital as a program such as this cannot be successful without positive support. the only way that these issues can be managed is if individuals first understand their values and share these honestly. with this openness, shared goals, and collaborative decision making are next required to create professional development initiatives. without this open understanding of each others concerns, these programs can become weak and ineffective leading to negative attitudes. these inclusive agendas are fated for failure without the positive support of staff however, it always begins with the individual values before it can become a collective value. . . . references alghazo, e.m., & gaad, e.n. (2004). general education teachers in the united arab emirates and their acceptance of the inclusion of students with disabilities [electronic version]. british journal of special education, 31(2), 94-99. armstrong, t.r. (2006). revisiting the johari window: improving communications through selfdisclosure and feedback [electronic version]. human development, 27(2), 10-14. bolton, g. (2010). reflective practice: writing and professional development. thousand oaks. brooks, b.l., & bransford, l.a. (1971). modification of teachers’ attitudes toward exceptional children [electronic version]. exceptional children, 38(3), 259-260. burke, k., & sutherland, c. (2004). attitudes toward inclusion: knowledge vs. experience [electronic version]. education, 125(2), 163-172. combs, r.h., & harper, j.l. (1967). effects of labels on attitudes of educators toward handicapped children [electronic version]. exceptional children, 33(6), 399-403. dessler, g., munro, c.r., & cole, n.d. (2011). management of human resources (3rd can.ed.). toronto, canada: pearson prentice hall. dixon, s. (2005). inclusion—not segregation or integration is where a student with special needs belongs [electronic version]. the journal of educational thought, 39(1), 33-53. edmunds, a. l., & edmunds, g. a. (2008). special education in canada. mcgraw-hill ryerson: toronto, on. glickman, c.c., gordon. s.p., & ross-gordon, j.m. (2005). the basic guide to supervision and instructional leadership. pearson education. elementary supervision and the supervisor / ryan & gottfried 571 gottfried, j. (2008). inclusive education: teacher attitudes as the key to success. unpublished paper, nipissing university. gottfried, j. (2007). teachers’ attitudes towards inclusive education: a review of the research. unpublished paper, nipissing university. hammond, h., ingalls, t., & lawrence, k. (2003). teachers’ attitudes toward inclusion: survey results from elementary school teachers in three southwestern rural school districts. rural special education quarterly, 22(2). retrieved october 14, 2007, from ebscohost research database. kuyini, a., & desai, i. (2007). principals’ and teachers’ attitudes and knowledge of inclusive education as predictors of effective teaching practices in ghana [electronic version]. journal of research in special educational needs, 1(2), 104-113. langton. n., robbins, s.p., & judge, t.a. (2011). fundamentals of organizational behaviour (4th can. ed.). toronto, canada: pearson prentice hall. mdikana, a., ntshangase, s., & mayekiso., t. (2007). pre-service educators’ attitudes towards inclusive education. [electronic version] international journal of special education, 22, 125-131. monsen, j.j., & frederickson, n. (2004). teachers’ attitudes towards mainstreaming and their pupils’ perceptions of their classroom learning environment. [electronic version] learning environments research, 7(22), 129-142. olson, j.c., colasanti, m., & trujillo, k. (2006). prompting growth for prospective teachers using cognitive dissonance. in novotná, j., moraová, h., krátká, m. & stehlíková, n. (eds.). proceedings 30th conference of the international group for the psychology of mathematics education, vol. 4, pp. 281288. prague: pme. ryan, t.g. (2009). an analysis of pre-service teachers’ perceptions of inclusion. journal of research in special education needs. 9 (3), 180-187. subban, p., & sharma, u. (2005). understanding educator attitudes toward the implementation of inclusive education. disability studies quarterly, 25(2). retrieved october 14, 2007, from ebscohost research database. talmor, r., reiter, s., & feigin, n., (2005). factors relating to regular education teacher burnout in inclusive education. [electronic version] european journal of special needs education, 20(2), 215-229. van reusen, a.k., shoho, a. r., barker., & kimberly s. (2000/2001). high school teacher attitudes toward inclusion. high school journal, 84(2). retrieved october 14, 2007, from ebscohost research database. microsoft word corrected5_ruth_morton_3_1 international electronic journal of elementary education vol. 3, issue 1, october, 2010. issn:1307-9298 copyright © iejee www.iejee.com home education: constructions of choice ruth morton ∗∗∗∗ university of warwick, united kingdom abstract families who choose to home educate generally do so due to dissatisfaction with schoolbased education. common perceptions of home educators oscillate between images of the 'tree-hugging hippy' and the 'religious fanatic'. whilst attempting to go beyond such stereotypical dichotomies, this paper will examine three very different groupings of home educators and their varying constructions of childhood and the social world, demonstrating the spectrum between home education as an expression of human rights and of fundamentalism. the first grouping construct home education as a 'natural' choice, often presented in political opposition to existing social structures. for the second grouping home education is predominantly a 'social' choice relating to the conscious transmission of various forms of capital. finally there are 'last resort' home educators for whom home education is not perceived as a choice. based on qualitative research, this paper will argue that, even where home education is constructed as natural, the social aspects and impacts of home education choices cannot be ignored. keywords: educational choice, lifestyles, alternative education, home education, homeschooling introduction home educators are not an homogeneous group (in fact it is debatable as to whether the term 'group' is appropriate due to their fragmented nature), and there is no one way of 'doing' home education, methods of home education are at the parents' discretion. home education is a growing but under-researched phenomenon in england and wales. the 1996 education act (section 7) states that parents are responsible for ensuring that their children received an 'efficient' and ' suitable' full-time education. there is ∗ correspondence: ruth morton, sociology department, university of warwick, coventry, cv4 7al, uk. e-mail: ruth@mortonmail.com international electronic journal of elementary education vol.3, issue 1, october,2010 46 no clear definition of what is meant by an 'efficient' and 'suitable' educationsomething which has at times led to conflict between home educators and local authorities (baker, 1964; monk, 2004; petrie, 1992). there is also no legal requirement for home educating families to register with their local authority meaning that the number of home educated children at any one time can only be estimated, with widely varying results (m. fortune-wood, 2006; hopwood, o'neill, castro & hodgson, 2007), although there are some indications that home education is a growing phenomenon in england and wales. the parameters and characteristics of the home educating population therefore remain unknown. common perceptions of home educators are of social 'misfits': either 'tree-hugging hippies', religious fanatics or 'hothousing' parents determined that their offspring should achieve academic excellence at an early age. my research and that of others suggests that the reality is much more diverse and complex and that home educators cover a broad spectrum between and beyond the stereotypes often presented. this paper examines three different ways in which home educators construct their choice to home educate, as either 'natural', 'social' or 'last resort', and concludes by arguing that although the choice to home educate is constructed in different ways by different home educators, there are common themes running through those choices which have wider social implications beyond the home education arena. methodology the paper is based upon data collected as part of my phd research between june and december 2007. qualitative data collection methods were employed, mainly through unstructured interviews with 19 families and one local authority official, but also through participant and non-participant observation at various home educators' meetings and a week's participant observation at a large home educators' annual camp. overall, 40 to 45 families participated in an intensive way in my study sharing their lives and experiences of being home educators. the research focuses predominantly upon home educating parents: whichever way families presented their constructions of childhood, choices around home education were almost entirely in the hands of the parents, particularly at the 'crunch point' of deciding to home educate. while my research focussed on parents, on the whole i was uncovering the mothers' story. of my sample, only one father had the main responsibility for home education, two couples initially told me that they shared responsibility equally but later admitted that the mother had a greater responsibility. in one further couple the husband was very involved in educating his children, but in all other accounts of home education in two-parent families the father was peripheral or even completely absent. in order not to exclude the contribution of those fathers who were active in home education this paper home education: constructions of choice / morton 47 often refers to 'parents', however it is to be remembered in the reading that home education appears to be predominantly a project of motherhood. three groups, three forms of choice? during data analysis three groupings of home educators emerged in terms of the way they constructed their choice to home educate. home education is an educational choice option for parents. there has however, been little analysis of home education from a choice perspective apart from that by aurini and davies in canada (aurini & davies, 2005). parents invoked many of the rationales for educational choice used by middle class parents about their choice of private school, such as social milieu, acquisition of wider life skills and the transmission of specific values (kenway, 1990; walford, 1990; west & noden, 2003). the first category identified was those who described education as a 'natural' choice, the second constructed home education as a 'social' choice, while for the third grouping home education was perceived not as a choice at all but as a 'last resort'. there is a degree of overlap and movement between the groups, in particular members of the third group tended to reconstruct their ideas about education over time, gravitating towards either the 'natural' or 'social' groups. the groupings are by no means homogeneous, and not all the home educators i met fit neatly into one category. indeed, what my rough groupings display is the degree of similarity between a wide diversity of home educators. the groups' different perceptions of their choice to home educate tended also to be linked to their approaches to education, childhood and parenting. the rest of this paper will examine the different constructions of the choice to home educate and briefly explore the characteristics of the three different groups. home education as a natural choice 'natural' was a term used by families within this grouping to describe their choices around childhood, education and lifestyle. in using the term 'natural' they were often referring to a way of life that was outside what they saw as false social structures and often evoked images of an idealised pre-industrial lifestyle. for many of these families home education was part of a conscious effort to reject conventional social structures and conformity to what they saw as a tyrannical system. this often involved positioning themselves in opposition to the 'other' of the state and institutional structures: we got involved in home education is as part of a change of lifestyle, addressing some of the imbalances in the way that we were living. ... stressful environments and stressful lifestyles, the western environment is full of stressful lifestyles and stressful diseases. (alan, interview). for many 'natural' families this lifestyle involved a rejection of consumer culture, and a concern for the environment which manifested itself in the adoption of vegetarian or vegan diets and a subsistence lifestyle. international electronic journal of elementary education vol.3, issue 1, october,2010 48 in alan and sarah's case this had led them to live in a yurt (mongolian tent) for the previous two years as part of an alternative community. for other families the lifestyle choice was less extreme in its difference from mainstream culture: instead of a hostile rejection of mainstream life and institutions, an attempt to improve on it. for several 'natural' families a concern for the notion of family was what guided the lifestyle, with the family seen as the core unit of society and the place where children should be nurtured. home education was therefore a lifestyle choice, integral to and often convenient to 'natural' home educators' philosophy of life, rather than being a choice purely related to education (although perceptions of 'education' played an important role). for these families the existence of the formal school system was in itself problematic as it contradicted the lifestyle that they wanted to live. for several families home education was something that they had drifted into. rather than organising for their children to go to school they had started home educating, enjoyed it and so had decided to continue, as for selina who had moved to england from south africa: i wanted to keep them home because we were in a foreign country and it was nice to go to the pier together or to do things or to see things, and then i realised that they are getting bored, ... and i realised that they were obviously raring to go more academically ... so i thought ok, so maybe i should find out about the local schools, ... so i found space for ruth in the one school – 3 miles in one direction and for liezl in the other direction one and a half miles which i just thought logistically won't work, you know with mia [a] baby, ... and these two in one direction and then in the other direction and then again 12 o'clock, with four hours difference, there is no way. ... then i started to think ok i need to stimulate them ... ... and then i started becoming so conscious of if you start school now in the morning at 7 when do you stop school, cos i realised that if we go for a walk you collect different leaves then you go and research what trees it is, ... so i just realised that wow, this is actually very stimulating, ... ... and i thought ok later i might research schools, but i just really got so into it, (interview with selina) earlier in the interview selina had talked about her previous dislike of home education. however, having fallen into home education she now described it as 'natural' and fitting closely into the rhythm of family life, allowing her to enjoy time with her children rather than needing to conform to what she perceived as the limiting structures of school life. in selina's comments there is also the emerging theme of children as individuals in their own right, rather than merely becomings (lee, 2001) or empty vessels to be filled with knowledge. this was also expressed by sarah: children are seen as empty vessels to be filled up, and i don't see that they are that, i think, my understanding is that they are quite full up and that we empty them in a way, they have lots of innate and intuitive things and you actually systematically get rid of that through systems and structures and things. home education: constructions of choice / morton 49 for many of these families the school system and its representation of what was often perceived as an oppressive state and/or oppressive capitalist structures was the problem that they were escaping by choosing home education. home education was therefore a mechanism which allowed parents to protect their children from negative state structures and preserve their innocence. talking to these families often felt as if i had walked into a reading of bowles and gintis' schooling in capitalist america (1976) with references to the long shadow of work over schooling and the production of conformist obedient workers to feed the capitalist industrial project. others spoke in an almost althusserian (1972) way of the ideological moulding done by school to children and the use of social services and the education system as a repressive apparatus to restrict and persecute home educators. accounts and examples of state persecution were rarely recounted firsthand however, and over time i realised that few of the people i spoke to had read or heard of bowles and gintis or althusser, and that what i was hearing was what had become a collective justification and by-line for many 'natural' home educators, fed by some of the popular home education literature and also by key home education activists. this was particularly evident at the home educators' camp i attended, where school as an oppressive capitalist structure ran as a theme throughout. in addition to the adoption of neo-marxist viewpoints on the functions of state structures in general, and schooling in particular, there was among these families, without any sense of contradiction in their minds, a strong sense of individualism. this was often to the exclusion of the notion of society and the impacts of individuals' behaviours upon others. there was a strong belief in the freedom of the individual and a child-led approach to education where structures were rejected as constraining and conformity was perceived negatively. many families aspired to autonomous home education (totally informal and child-led), with parents acting as facilitators for the child's learning interests. this sense of individualism over community meant that 'natural' home educators tended to operate as independent units with relationships with other home educating families being fluid and transient, existing to meet families' needs. there was often also suspicion of other home educating families, particularly those who were not 'natural' home educators. 'natural' home educators were ambivalent about the notion of education, often being unclear as to what they thought education was. they were clearer about what they felt the aims and outcomes of education should be, many talked about children fulfilling their individual potential, and there was also a grudging acknowledgement that their children should be able to survive economically: charles: ultimately i suppose they've got to become economically independent haven't they, that's what education is supposed to be for ... rm: so do you see that as one of the aims of what you're trying to get out of her education is the ability to earn a living? international electronic journal of elementary education vol.3, issue 1, october,2010 50 jill: ultimately yes, but also at the moment i think it's just exploring what she wants to explore, and then see where that takes her. charles: yes, you've got to find your interests haven't you, ... then when you've found things that really interest you then you go full steam ahead on those, and if you're successful enough you'll want to earn a living from them and you'll be in that happy position where where your life's work actually earns you a living. as this indicates, for many families, despite the ideal of an escape from economic and institutional structures, there was an acceptance that economic labour was a necessity for survival and that education should equip children for that reality. at the same time jill and charles' comments also highlight the fact that the idea of education as a process of selfdiscovery and self-interest remained foremost in importance. home education as a social choice in contrast to 'natural' home educators, parents for whom education was a 'social' choice did not see the structures of the formal education system as problematic. their issues were specifically with the social interactions that they associated with schools, between pupils and with teachers, and the values communicated through those interactions. for many of these families private schooling was seen as an alternative to home education, but one which was often financially impracticable. all but one of the interviewed families in this group described themselves as 'christians' and their beliefs and values were central to their choice of home education over mainstream schooling (although there were also christian families in the 'natural' group). however, other families that i met and talked to during my periods of observation who fitted into the 'social' group were not christians. for these families home education was linked to perceptions of parental responsibilities, they saw themselves as having ultimate responsibility for their children's upbringing, both moral and social. parents felt that they should not relinquish the responsibility to the anonymous and morally ambivalent structure of the school system. while there was an element of protection in terms of values and morals for these families, it was a different kind of protection from that of both the 'natural' and 'last resort' groups. few of the parents who were home educating for 'social' reasons expressed serious concerns about bullying in schools (in fact several expressed regret that their children had not been exposed to such dynamics) and the sense of large groups of children as inherently 'unnatural' was not in evidence in the same way as for 'natural' home educators. instead parents expressed greater concerns about the values that other people's children brought into schools and the influence those children might have upon their offspring: unfortunately 'cos of the breakdown of families, especially in what is quite a poorish area, you get a lot more moral ambiguity. that megan and owain would home education: constructions of choice / morton 51 have been exposed to all kind of things i didn't want them to be exposed to. so really by the time i was expecting a child i was wanting to have them christianly educated. (sophie, interview) in many ways these parents echoed the rationales of parents who choose private schooling for their children (see for example frazer, 1993; griffiths, 1991; kenway, 1990; walford, 1990) as a way of exerting control over what their children are exposed to, especially with regard to they kinds of children they mixed with. in addition to these ‘social’ home educators were concerned about the interactions between teachers and students and the fact that a parent could not guarantee moral and value homogeneity between themselves and the teacher: we had considered home education, i think one of the things that made us consider it more seriously was the things that she was bringing home from school were contradicting what we were teaching her at home. religious-wise, behaviour-wise, you know there were various things which even the teacher ... for example one situation that happened was one little boy was pulled up in front of the class and told that he was disgusting, ... so it wasn't just necessarily things that were coming back from the children, the teacher had our child for most of her waking hours and so her behaviours were reflected upon our own child. (janet, interview) janet's comments emphasise the view that came from several parents that they did not want their children 'confused' by receiving mixed messages from school and home. for them the importance of ensuring that their children received the right social messages and internalised the right values were linked to their perceptions of their children as semi-formed individuals and to their responsibility as parents for their children's formation into wellsocialised adults: the children are not in charge of themselves, autonomous education is a thing which i disagree with, the theme which runs through eo [education otherwise] is children are just as good as adults well yes they are, they are but they don't know everything yet, and it is our job to train them and educate them and prepare them to go out there so that they can be voting people, be good neighbours, be good work people, at the moment they are still in training and they don't know their own mind. (tanya, interview) home education was therefore the performance of the parents' duty to give better social conditioning to their children than the school options available to them. this concern with the interactions over the structures of the education system were reflected in the ways these families tended to 'do' home education. most 'social' families followed a formal programme of home education, often following a set curriculum (typically imported from the united states) which reflected their value system. one family were at the unstructured end of semi-formal home education with rough plans for progress drawn up each term by the mother and father which were then adapted or abandoned as appropriate. on the other hand, janet and sophie international electronic journal of elementary education vol.3, issue 1, october,2010 52 had recreated 'school at home', with school uniforms and a timetable. when i visited her at home janet was busily involved in organising a sports day for the group of home educators to which she belonged – another aspect of recreating school structures while maintaining moral and social control over their children's education. despite the fact that many of these families valued structure and replicated school structures at home, they also spoke of valuing the flexibility of life that home education afforded them. timetables tended to be dropped towards the end of terms, or they would re-shuffle their work completion targets in order to take a day off for an educational trip or mother-daughter shopping trip. often these alterations were justified with comments that home education was also about life skills, displaying a broader conception of education than that offered by the traditional school structures. there was also a gentle movement towards more informal educational methods, although not as marked as that noted by thomas (1998), with the introduction over time of extended project work in addition to more formal curriculum-based work. like the 'natural' home educators, 'social' home educators also valued the opportunity to individualise their children's education to individual strengths, weaknesses and interests, although to a lesser extent. home education as no choice? for eight families interviewed (and many others encountered) home education had been a 'last resort', rather than a perceived choice. several families spoke of their children having come close to emotional breakdown prior to being removed from school, including self-harming and suicide attempts. often this was cited as due to bullying and/or linked to a child having special educational needs (sen). one family had come to home education because of their daughter's health needs; for them, home education was a means to provide what they felt was education appropriate to their daughter's current capacity, whilst escaping pressure from the school and the local authority. home education therefore provided an escape route for these families from what had become for them untenable situations. many used the term 'last resort' and talked about the length of time it had taken them to come to the decision to home educate, with numerous attempts to make school 'work'. in several situations the children had struggled all the way through primary school (ages 5-11) in the hope that secondary school would be an improvement, with the decision to remove them being taken in the last year of primary school or the first year of secondary school. home education was initially seen as a period for recovery. often the initial intention had been to withdraw a child from school for a short period of time with a view to reintegrating them into mainstream school having overcome the problems that had led to withdrawal. however, return to mainstream education seemed rare before the child reached 16. one mother home education: constructions of choice / morton 53 had tried returning her daughter to school unsuccessfully, removing her again when she had started to self-harm, another family had recently put one son into a small private school after family health issues had made continuing the home education of all three children impossible, at the time of interview this was successful. hilary and beth's description of the gradually receding idea of a return to school was typical; beth had been taken out of school during the final term of primary school due to bullying and had been educated at home for 3 years: hilary: initially we kept your place, 'cos she did have a place already booked at the secondary school, so initially we kept that and thought perhaps, that if she had the break, then she could start afresh. although a lot of the same children were going to that school. but it was clear that she wasn't going to be ready to go back to school, so ... rm: so you started home education and that was it really? ... hilary: we kept our minds open, in fact we've still got our minds open, but the likelihood of her going back now is pretty slim i think. rather than perceiving the school system in its entirety as problematic, 'last resort' families tended to describe their negative experiences of school in terms of their individual child and the attitudes of the individual school. lydia was disparaging about the way her son's primary school had handled his recurrent bullying: the primary school were no good, the primary school couldn't give a toss really, they sort of did what they had to do but there was none of this. it would start and then it would start again and they would start back at the beginning of the policy even though it was the same kids involved instead of like “we got to there we're gonna start there again and then move on further because you haven't got the hint”. for five of the families interviewed and several other families i encountered, only one child had initially been removed from school, with two families later removing their other child when they also experienced bullying. this demonstrates the way in which, for 'last resort' families, home education is about an individual child's experiences. there were frequent comments about the perceived negative attitudes of schools and individual teachers towards catering for children's individual needs, with help being refused or a lack of interest shown. anna described the frustration she had encountered when she had tried to get help for her daughter sandy, who was suicidal, to stay in school: at the time she had a play therapist and the play therapist and i arranged to go in to see the ewo [education welfare officer] and the headmistress. headmistress didn't turn up, so it was just the three of us there and i'd written an a5, a4, sheet of all the different things that i thought could help sandy. and we got to the bottom and they couldn't do any of them and the last thing on the bottom of the sheet was well i could home educate her then. and the woman turned round to me and said to me “well if you're not interested in your daughter's education then that's an option isn't it” and i thought i've just sat here international electronic journal of elementary education vol.3, issue 1, october,2010 54 for an hour and i obviously am interested in her education otherwise i wouldn't put that at the bottom would i? so i took her out. all the parents in this grouping described their children as different in some way. because of past traumas they had experienced or their learning difficulties, they saw their children as individuals with individual problems and therefore in need of individual solutions. they saw the failure of their children's individual schools and teachers to find a solution as problematic rather than the inherent structure of the education system. this view was reinforced for lydia and anna by their experiences of helpful actions by previous schools. for 'last resort' families home education was just that, a final option when all else had failed and they needed to protect their vulnerable children: i did it because i felt i had no choice. i did it because the system she was in wasn't working, i didn't do it, i would have rather that system had worked to be honest but it didn't. so i guess, yeah – then you put your lioness head on and gather in and try and protect don't you? (anna, interview) anna's metaphor of a lioness expresses powerfully the sense that at a certain point children cannot be left as vulnerable beings to cope on their own. it is also noticeable that in all the 'last resort' families it was the mother who had made the decision to protect her child by taking (or advocating for) the decision to home educate. conclusions as has already been stated, common perceptions place home educators at extreme ends of what is in fact a broad spectrum. this study found that there were three main categories of home educators in terms of the choice to home educate: 1) those who saw home education as a 'natural' choice compared to the false and restrictive structures of the formal education system; 2) those for whom home education was a 'social' choice tied in with their wish to transmit certain moral and social values and behaviours to their children; 3) parents for whom home education had been a last resort, and was perceived as a non-choice. although there were significant differences between these groupings there were also significant similarities. the key similarity is the theme of the individual. all of the families perceived their children as individuals and the parent (usually the mother) as the expert on those individual children. all families had found some fault, be it structural, social or at an individual level, with the school system, in particular the state-maintained education system and their solution had been to choose home education for their individual child. although almost all parents conceded that there might be children and/or families for whom home education would not be appropriate, there was a lack of interest in solutions that went beyond their individual child or children. home education: constructions of choice / morton 55 this individualisation of choice expressed in home education is in many ways an echo of a wider mantra of 'choice' and 'the individual' within society and within social policy. changing conceptions of childhood and children as well as of individual rights are reflected in educational policy with the rise of parental choice in education and also government schemes to individualise education to meet each child's needs. home education is in many ways a logical extension of the educational choice mechanism. it also raises many of the same issues as private schooling: questions of elitism, social engineering and also of the exit of articulate and socially powerful parents from the state education system where their voices and choices might be used to go beyond improving the individual educational circumstances and experiences of their children and to have a wider social impact. the choice of home education also perhaps serves to highlight some of the different ways in which parents are dissatisfied with the school system and also continuing mismatches between schools and wider social cultures. it also highlights the broad spectrum of parents' concerns and constructions beyond a simplistic human rights/fundamentalist dichotomy. in this context researchers need to be thinking more widely about home education and its broader social impact. • • • ruth morton is currently completing her doctoral thesis in sociology at the university of warwick. with experience of teaching in both secondary and further education ruth's research interests are centred around education, parenthood and the family. references althusser, l. (1972). ideology and ideological state apparatus. in b. cosin (ed.), education, structure and society (pp. 242-280). middlesex: penguin. aurini, j., & davies, s. (2005). choice without markets:homeschooling in the context of private education. british journal of sociology of education, 26(4), 461-474. baker, j. (1964). children in chancery. london: hutchinson & co. bowles, s., & gintis, h. (1976). schooling in capitalist america. london: routledge & kegan paul. fortune-wood, m. (2006). the face of home-based education 2: numbers, support, special needs. nottingham: educational heretics press. frazer, e. (1993). talk about class in a girls' public school. in g. walford (ed.), the private schooling of girls: past and present (pp. 127-152). london: woburn press. griffiths, j. (1991). small private schools in south wales. in g. walford (ed.), private schooling: tradition, change and diversity (pp. 84-97). london: paul chapman. hopwood, v., o'neill, l., castro, g., & hodgson, b. (2007). the prevalence of home education in england: a feasibility study no. 827). london: department for education and skills. international electronic journal of elementary education vol.3, issue 1, october,2010 56 kenway, j. (1990). privileged girls, private schools and the culture of "success". in j. kenway, & s. willis (eds.), hearts and minds: self-esteem and the schooling of girls (pp. 131-156). london: falmer press. lee, n. (2001). childhood and society. maidenhead: open university press. monk, d. (2004). problematising home-education: challenging 'parental rights' and 'socialisation'. legal studies, 24(4) petrie, a. (1992). home-education and the local education authority: from conflict to cooperation. unpublished university of liverpool, thomas, a. (1998). educating children at home. london: cassell. walford, g. (1990). privatization and privilege in education. london: routledge. west, a., & noden, p. (2003). parental choice and involvement: private & state schools. in g. walford (ed.), british private schools: research on policy and practice (pp. 177193). london: woburn press. microsoft word 9iejee_6_2_lee_zeppelin international electronic journal of elementary education, 2014, 6(2), 333-346. using drawings to bridge the transition from student to future teacher of mathematics ji-eun lee ∗∗∗∗ oakland university, usa mary zeppelin oakland university, usa received: 24 june 2013 / revised: 18 october 2013 / accepted: 1 january 2014 abstract this study examines a group of prospective teachers’ reflections upon the way they were taught (set 1) and the way they want to teach (set 2) through drawings which respectively describe their past learning experiences as students and their future plans as teachers. the purpose of this study is to identify: (a) the emerging themes that appear in each set of drawing data, (b) the possible factors that influence prospective teachers’ drawings, and (c) the implications for mathematics teacher educators. overall, prospective teachers showed predominantly negative or mixed feelings about their past experiences as mathematics students. in response to their own past negative experiences and struggles, the prospective teachers tended to highlight emotionally supportive classroom environment and versatile instructional teaching strategies in their future plans. this study suggests that this activity of reflecting past experience and planning future teaching assimilates prospective teachers’ identities as math students and math teachers and provides a window into the thinking of others. keywords: pre-service teacher education, teacher attitude, reflective drawing. introduction one day, a woman was about to cook a roast. before putting it in the pot, she cut off a small slice. when asked why she did this, she paused, became a little embarrassed, and said she did it because her mother had always done the same thing when she cooked a roast. her own curiosity aroused, she telephoned her mother to ask why she always cut off a little slice before cooking her roast. the mother’s answer was the same: “because that’s the way my mother did it.” finally, in need of a more helpful answer, she asked her grandmother why she always cut off a little slice before cooking a roast. without hesitating, her grandmother replied, “because that’s the only way it would fit in my pot” (langer, 1989, pp. 43-44). ∗ ji-eun lee, oakland university, 2200 n. squirrel rd. 485f pawley hall rochester, mi usa 48309 phone: 1-248-370-2119. e-mail: lee2345@oakland.edu international electronic journal of elementary education vol.6, issue 2, 333-346,2014 334 we often hear that teachers teach the way they were taught. although teaching is more complicated than cooking, we frequently hear this saying and often observe that teachers teach in a certain way because that is the way their teachers did it. today’s classrooms and students are ever evolving as the content and pedagogical measures in mathematics education continue to advance. the teachers of today are expected to demonstrate effective mathematics teaching that “requires understanding what students know and need to learn and then challenging and supporting them to learn it well” (national council of teachers of mathematics [nctm], 2000, p. 16). successful mathematics teaching cannot be demonstrated just by parroting the way my teacher did. it requires “continuing efforts to learn and improve” (nctm, 2000, p. 19) in a mindful way. the nature of mindfulness in langer’s (1989) work exemplifies the characteristics that impactful teachers demonstrate to improve their teaching: the ability to create new categories, the willingness to welcome new information, the capacity to present more than one perspective, the power to manage context, and the desire to put process before outcome. we believe that teacher educators should provide turning points to help prospective teachers “unpack the way it is” (hinchey, 1998, p. 17). in other words, as prospective teachers are about to embark on their own professional journey, they need to mindfully reflect upon their own learning experiences and use that process as a means to improve their own future teaching practices. with this in mind, we utilize drawing exercises in an attempt to promote this opportunity for reflection. the second author began incorporating drawings into an elementary mathematics methods course in early 2000. the first author began using the technique shortly after and together we have collected data of drawings from our prospective elementary teachers. our archival data consist of several themes including teachers’ depiction of “how i feel about math” and their portrayal of “my math teacher and me.” this study reports the findings from drawings of “my math teacher and me.” we asked prospective teachers to draw their past mathematics learning experiences taught by their mathematics or homeroom teachers along with a descriptive paragraph at the beginning of a semester-long elementary mathematics methods course. at the end of the semester, the same assignment was given but students were instead asked to draw and write about their own mathematics classrooms in five years. the purpose of this activity is for prospective teachers to critically reflect upon the way they were taught and to mindfully plan and transform their own teaching. in this article we will discuss: (a) the emerging themes that respectively appear in the prospective teachers’ reflections upon their past learning experiences and their future plans, (b) the possible factors that influence the prospective teachers’ future plans, and (c) the implications for mathematics teacher educators. related issues in the literature issues on mathematics teacher preparation. previous research studies on mathematics teacher preparation have developed into several themes including previous experiences, knowledge structures, and belief systems (ambrose, clement, philipp, & chauvot, 2004; frykholm, 1999; munby, russell, & martin, 2001). these themes are interrelated and ultimately influence the effectiveness of mathematics teaching methods. it has been purported that prospective teachers in the united states lack sound subject matter knowledge, and that their beliefs about mathematics teaching and learning do not quite match the new vision for teaching and learning (e.g., ball, 1990; ma, 1999). teacher preparation programs offer various opportunities to enhance subject matter knowledge and to influence prospective teachers’ beliefs. even so, these beliefs are often difficult to change in the limited time the programs have. as noted in several previous studies, a major obstacle is the prior experience and using drawings to bridge / lee & zeppelin 335 knowledge that prospective teachers carry into the program (e.g., eisenhart, borko, underhill, brown, jones, & agard, 1993; thompson, 1992). in particular, the beliefs about what to teach and how to teach it have been shaped through years of experience spent observing what their own k-16 teachers did when they were students and it is almost unrealistic to completely unite their prior experience with the new vision of mathematics teaching (e.g., ambrose, 2004). as doerr, lesh (2003) and others have noted, the prospective teachers’ prior experience is obstacle, not because they have observed poor teaching, but because they have not been exposed to how teachers think. one problem for prospective teachers is that their actual teaching practice is commonly limited to a few hours in field experience settings when they take the mathematics methods course. because of this, many researchers encourage teachers to reflect upon their own teaching as a way of changing their beliefs and determining best practice for their students (e.g., cooney, 1999; schön, 1983; simon, 1995). with this in mind, we believe that reflecting upon their own learning experiences is an alternative way to provide prospective teachers with opportunities to think about effective teaching and learning processes. drawings as research method over the last several decades, psychologists and other researchers have utilized drawings to facilitate the rich exploration of children’s and adults’ views on multiple phenomena (mitchell, theron, stuart, smith, & campbell, 2011). in the field of education, drawings are used predominantly to investigate young students’ perceptions, emotions, and attitudes towards various content areas. for example, students’ drawings of scientists and mathematicians were used to examine children’s perceptions of mathematics and science (e.g., finson, 2002; picker & berry, 2000). zambo and zambo (2006) used thought bubble pictures to examine students’ feelings toward mathematics. perceptions of literacy and environmental issues were also examined through students’ drawings (e.g., alerby, 2000; kendrick & mckay, 2004). some previous research studies suggested extending this method to include adults or pre-service and in-service teachers (e.g., finson, 2002). our archival data collection efforts are consistent with these suggestions. we recently discovered a number of published studies that use drawings to investigate prospective teachers’ attitudes and feelings about mathematics (e.g., burton, 2012; rule & harrell, 2006). the majority of research studies on pre-service and in-service teachers’ subject matter knowledge, attitudes, and beliefs primarily use paper and pencil tests, structured interviews, or specific math concept related performance tasks. the alternative approach, utilizing drawings, can help to determine additional factors that influence prospective teachers’ beliefs and knowledge that cannot be solely obtained by a paper and pencil survey or specific math concept oriented performance tasks. this approach may also provide a less stressful research environment. realizing the existence of math anxiety in the general public (burns, 1998) as well as in the teaching profession (trice & ogden, 1986/1987), we believe an alternative approach, like drawing, will provide ways of facilitating teacher candidates’ self-reflection. methodology participants we gathered two sets of drawings and accompanying written descriptions which respectively focused on prospective teachers’ past mathematics learning experiences as students and their future plans as teachers. these samples were taken from 100 prospective teachers who enrolled in one of five sections of an elementary mathematics methods course over two semesters. this elementary mathematics methods course is a 4-credit, required course for all elementary education majors at a international electronic journal of elementary education vol.6, issue 2, 333-346,2014 336 midwestern united states university and is typically taken prior to student teaching. all of the prospective teachers had successfully completed their mathematics content courses prior to this methods course. participants consisted of 84 female and 16 male teacher candidates. throughout the semester, participants engaged in various modes of instruction, including lectures, large and small group discussions on theories and educational trends or issues, and hands-on activities that involved technology tools and manipulatives. in addition, participants were asked to complete several course assignments in their field setting while they interacted with actual students. those assignments included developing and implementing a mathematics lesson and assessment for their field students. data source prospective teachers’ reflections upon their past mathematics experiences and plans for future teaching were identified through the drawings and corresponding descriptions that they completed on two separate occasions during the semester. the first set of drawings and descriptions (set 1) was collected at the second class meeting. participants were asked to draw a picture that portrayed their past math teachers or other memorable mathematics learning experiences on a standard sheet of paper. participants were also asked to include a written paragraph that described their picture and clarified the meaning embedded in their drawing, as suggested in other similar studies using drawings as research methods (mitchell et al., 2011). these were shared in a small group discussion and a few volunteers even presented their drawings to the class. the second set of drawings and descriptions (set 2) was collected on the last day class. this time, participants were asked to draw a picture that portrayed their own elementary mathematics classes in five years. participants’ drawings were presented in various formats including hand-drawings, computer clip arts, and collages. some drawings contained realistic descriptions of classroom settings or people while others used metaphorical objects or words. in order to encourage participants to respond honestly, it was promised that the quality of their artwork and writing would not be assessed and students would earn full credits by simply completing their work. these two sets of drawings were worth approximately 5 percent of the total course assignment points. data analysis participants’ drawings and written descriptions were examined based on aspects of open-ended coding and a double-coding procedure (miles & huberman, 1994; strauss & corbin, 1998). this study was not intended to utilize the pre/post design that asks the same question to compare changes. instead, the sets of drawings and written descriptions were analyzed separately highlighting participants’ views on teaching and learning mathematics when they positioned themselves in different roles (i.e., as a student or as a teacher). we created a text translation of the drawings by listing specific items or settings depicted in each (e.g., “a crying face in the middle surrounded by numbers and signs of operations”). we then noted specific words and phrases in the corresponding written descriptions. the text translations and notes made from the written descriptions were used together as data and categorized into several themes. initially, we reviewed the data independently to identify recurring themes and intentions. we then revised and refined the identified themes together through comparison and discussion and then coded our findings. doing this together allowed us to resolve coding discrepancies immediately. after the completion of coding, frequencies of coded themes were using drawings to bridge / lee & zeppelin 337 identified. in the results section, selected excerpts and examples of drawings were used to illustrate the common themes identified. results analysis of set 1 set 1 portrayed past mathematics learning experiences and was categorized into three main themes that represent positive, mixed, and negative feelings toward mathematics learning experiences. table 1 shows the categories of themes and frequencies followed by additional explanations of the sub-themes identified. table 1. themes in set 1 major themes sub-themes number of entries (%)* positive experiences: entries that describe positive aspects of past teachers or personal experiences in their classrooms • positive portrayal/description of teachers’ and/or the participant’s subject matter knowledge 4 • positive portrayal/description of teachers’ and/or the participant’s affective aspects 8 mixed experiences: entries that describe both positive and negative aspects of past teachers or personal experiences in their classrooms • the participant’s inconsistent perceptions about the teacher 4 • the participant’s liking of mathematics and disliking of the teacher’s teaching practices 3 • the participant’s disliking of mathematics and liking of the teacher’s teaching practices 4 • varying perceptions about mathematics by grade level 7 negative experiences: entries that describe negative aspects of past teachers or personal experiences in their classrooms • participants’ negative emotions towards mathematics 18 • participants’ negative emotions towards teachers 20 • physical distance and emotional disengagement between the teacher and student 22 • static nature/environment of math instruction 25 (* note: the percentages total more than 100% as some entries contain multiple sub-themes) it was noted that, even within one entry of drawing/description, participants frequently referred to several sub-themes; identifying multiple dimensions of their thoughts on the different aspects of teaching and learning that they experienced as students. thus, one entry often contained multiple themes, particularly those that fell into the negative category. the coding process focused on whether the specific theme was present or absent in each participant’s drawing/description. positive experiences. twelve out of 100 entries expressed positive past math experiences with their teachers. while a few words/phrases referred to teachers’ knowledge (e.g., knowledgeable, talented), most presentations and descriptions were about their emotional experiences. for example, some of the most frequently used expressions included: pictures of smiley faces, hearts, a teacher and a student holding hands, and words like fun, enthusiastic, approachable, supportive, helpful, and patient (see figure 1). international electronic journal of elementary education vol.6, issue 2, 333-346,2014 338 figure 1. example of set 1 drawing: positive past math experiences mixed experiences. eighteen entries contained both positive and negative experiences. these drawings and written descriptions illustrated participants’ mixed emotions about their math teachers or the subject matter. for example, one entry was divided into two sections: one part has a sun and smiling faces, while the other part has clouds and lightening. its written description states that the picture portrays two different sides of her teacher: very warm, funny, and helpful and at the same time very strict and moody (see figure 2-a). the other entries in this category were divided into three subcategories: (1) overall, i enjoyed math but did not like the way my teachers taught, (2) overall, my teachers were great, but i did not like math, and (3) i had different experiences grade by grade; generally, positive experiences in the early grades, negative experiences in later grades (see figure 2-b). figure 2-a figure 2-b figure 2. examples of set 1 drawings: mixed experiences negative experiences. the remaining entries highlighted negative past experiences. some negative images included sad faces, sleeping students, clouds, and multiple question marks. they often also included words like confusion, boring, frustrated, and struggle. these results support the idea that adults’ math anxiety is deeply rooted in their early mathematics interactions with teachers (e.g., mcleod, 1992; newstead, 1998). a few common themes in the drawings/descriptions further explained the root of participants’ negative experiences: (1) a big emotional and physical distance between the teacher and the student, and (2) the static/unchanging nature of math instruction. in terms of the emotional and physical distance, not a single drawing in the negative responses portrayed the scene of student and teacher working together. in most of the drawings, teachers were lecturing at the board in front of the classroom or were sitting at their desks or overhead projectors in front. some drawings and written descriptions even went so far as to depict teachers’ backs turned to their students. there was a distinct physical distance between teachers and students (see figure 3-a). twenty-two entries in negative responses explicitly or implicitly expressed the emotional disengagement between teachers and students. several extreme expressions include portraying teachers as monsters, witches, dead people, and even using drawings to bridge / lee & zeppelin 339 a woman holding a gun (see figure 3-b). other drawings and descriptions in this category typically show teachers who love math and students who did not like doing math. for example, many pictures show teachers’ outfits, materials in their desks, or posters around the classroom covered with positive and encouraging words, such as “i love math,” “math is so simple,” “math is fun,” “math is easy,” “math + math = my life,” “math rocks.” however, in the same pictures, the feelings and attitudes on the students’ part were described quite negatively. several examples include: teachers’ names were “mrs. confusing” and “mr. no fun”; teachers say “blah, blahp”; students say, “what am i doing here?” “i don’t get it” “i don’t understand”; students think “recess” or something else; students are sleeping (see figures 3-c, 3-d, 3-e, 3-f). figure 3-a figure 3-b figure 3-c figure 3-d figure 3-e figure 3-f figure 3. examples of set 1 drawings: emotional and physical distance between the teacher and the student regarding the depiction of math class and teachers’ ways of teaching, 25 entries categorized as negative responses addressed the teachers’ unchanging pedagogical practices or physical and emotional environments of math classrooms, saying: “teachers taught math pretty much the same way”; “k-8 math teacher were very similar”; “desks were in rows – never changed throughout the years.” typical images or descriptions included: students sit in rows, teachers showing examples on the board, students doing worksheets independently, students memorizing formulas, teacher giving homework, paper/pencil test, and showing all assignments coming from the textbook (see figure 4-a, 4-b, 4-c). international electronic journal of elementary education vol.6, issue 2, 333-346,2014 340 figure 4-a figure 4-b figure 4-c figure 4. examples of set 1 drawing: static nature of math instruction analysis of set 2 drawings and written descriptions from set 2 portray participants’ math classrooms or themselves as active teachers in five years. this set illustrated the beliefs, pedagogical knowledge, and attitudes of participants, as they are close to becoming professional educators. without doubt, the majority of set 2 entries are predominantly positive, highlighting what kinds of teacher they want to be and what kind of math classroom they want to create. table 2 shows the categories of themes and frequencies followed by additional explanations. each entry can belong to multiple sub-themes according to the presence of specific themes in the drawings/descriptions. table 2. themes in set 2 major themes sub-themes number of entries (%)* uncertainty: entries that describe mixed feelings about their future teaching • mixture of some level of uncertainty and hope for improvement 3 emotionally supportive classroom: entries that describe the teacher-student relationship • teachers’ encouragement of students 57 • close physical distance between the teacher and student 23 • students’ positive attitude towards mathematics and teachers 45 incorporation of various teaching methods: entries that describe various ways of teaching mathematics • use of various representations 74 • constructive pedagogy 55 • connections 25 (* note: the percentages total more than 100% as some entries contain multiple sub-themes) uncertainty: only three entries expressed some level of fear, frustration, confusion, and lack of confidence in their ability to teach mathematics effectively. they do show, however, that success increases as math-teaching experience is gained (see figure 5). using drawings to bridge / lee & zeppelin 341 figure 5. example of set 2 drawing: uncertainty (gradual improvement) emotionally supportive classroom. in 57 entries, participants expressed that they would emotionally support their students’ learning. these drawings and descriptions clearly indicated teachers’ efforts to encourage students as shown in statements and exclamations such as: “you can do it”; “there is no stupid question!”; “go explore, you can do it”; and “my door is always open for questions.” smiling faces on both teachers and students appeared frequently in the drawings, as well. there were 23 entries that described close physical distance between the teacher and student (e.g., a teacher sitting at a round table with students, the teacher and students holding hands). in 45 entries, students’ were shown making positive comments about mathematics and the teachers were identified (e.g., students saying “i am never frustrated in this math class”, “we love our math teachers”, “math is fun”, and “math rocks”.) figure 6-a figure 6-b figure 6-c figure 6-d figure 6. examples of drawings in set 2: future plans incorporation of various teaching methods. seventy-four entries demonstrated participants’ willingness to incorporate a variety of representations in their math instructions. these entries included specific manipulatives, pictorial representations, and written and verbal explanations of the mathematical process. fifty-five entries expressed the participants’ desire to change the general format of instruction to be a more student-centered, cooperative learning environment. examples mentioned include grouped seating arrangements, learning centers, group/partner work, and phrases such as “adaptable for all different types of learners”, “learn from each other”, “many ways to teach”, “exploration” frequently appeared. twenty-five entries showed participants’ plans to make sense of mathematics through various mathematical connections. included are connections within mathematics, literature, technology integration (e.g., virtual manipulatives, computer programs), and real-life applications. the drawings that highlighted the sub-themes in “emotionally supportive classroom” international electronic journal of elementary education vol.6, issue 2, 333-346,2014 342 and “incorporation of various teaching methods” are illustrated together in figure 6 since most drawings in set 2 addressed multiple themes. discussion and implications in an attempt to examine teacher candidates’ reflections upon the way they were taught (set1) and the way they want to teach (set 2) in the near future, we present the common themes and features of prospective teachers’ drawings and written descriptions. prospective teachers’ mathematics knowledge, beliefs, and attitudes will eventually be translated into their future teaching methods. therefore, it is important for prospective teachers’ to be aware of their own perceptions toward teaching and learning mathematics. in this study, we provide prospective teachers with opportunities to reflect upon how they were taught and to envision how they will teach by bridging the gap between teacher candidates’ past mathematics experiences and future plans. findings from the negative images and words in the set 1 drawings/descriptions explained the possible causes of participants’ mathematics anxiety that were investigated in previous research studies (e.g., newstead, 1998; taylor & fraser, 2003; trujillo & hadfield, 1999). also, the descriptions of their past teachers’ methods that highlight the static/unchanging nature of instruction confirm the findings in other studies addressing typical traditional math or science instruction (battista, 1999; schoen, fey, hirsch, & coxford, 1999; lindgren & bleicher, 2005; stodolsky & grossman, 2000). the common features of typical mathematics classes in the previous studies and the current study include: mimicking what the teacher demonstrates; a focus on computational procedures; little relevance to students’ lives; and multiple drills and worksheets from textbooks with almost no exploration, investigation, or explanations. this study added additional data regarding prospective teachers’ past learning experiences and their perceptions of mathematics, which seem to be deeply imbedded and consistent. we believe that the results in set 2 depict participants’ reactions to synthesizing their past learning experiences. in other words, the reflection upon the prospective teachers’ past learning experiences using drawings provided an opportunity to elicit the reasons for the struggles they encountered as students and to respond to potential struggles that their future students might have. we also believe that the set 2 results were influenced by the prospective teachers’ learning experiences in the methods course. the multiple pedagogical measures demonstrated and used in the methods course might influence the prospective teachers’ depiction of their future mathematics classrooms. we believe that the reflection process we initiated using drawings along with written descriptions encouraged participants to develop plans to bridge their past and future mathematical teaching experiences. we believe that their plans will continue to change throughout their professional lives as teachers. these prospective teachers now know how they think and how other prospective teachers think about teaching mathematics. the implications for math teacher educators are paramount: this activity assimilates prospective teachers’ identities as math students and as math teachers and provides a window into the thinking of others. as this study contributes to raising awareness about the gap between past experience and future math teaching, the importance of modeling through constructivist pedagogy and reflective practice is evident. these prospective teachers will not mindlessly be cutting off a small “slice of the roast” unless it needs to be done in order to fit into the pan! a follow-up study is needed to resolve some limitations of this study. we noticed that the importance of the teachers’ subject-matter knowledge was very vaguely represented in the drawings/descriptions. for example, some participants stated that using drawings to bridge / lee & zeppelin 343 they felt much more confident and comfortable after taking this methods course without specifics. of course, to utilize various instructional methods, teachers must hold strong subject-matter and pedagogical knowledge. however, it is not clear whether or not participants implicitly addressed the importance of teacher subject-matter knowledge in the way we interpreted. it is also not evident whether or not this result indicates that participants weighed more affective aspects than cognitive aspects as qualities of good math teachers. this result may be influenced by the fact that this study was conducted within the context of a methods course and the immediacy of the methods experience the participants had just encountered. the key question would be whether or not this tendency persists and to what degree of intensity does it persist once the participants are in the field. thus, it will be a meaningful follow-up study involving prospective teachers as they become novice in-service teachers to determine if the same gains hold for participating in the methods course. • • • ji-eun lee is an associate professor at teacher development and educational studies (tdes), school of education and human services (sehs), oakland university (ou), rochester, m. her research interest is focus on mathematics teaching, teacher education. mary zeppelin is a special instructor at oakland university, is distinguished as an outstanding teacher. she is also recognized for changing the traditional methods of teaching mathematics, as advocated by the national council of teachers of mathematics, in the areas of middle school classroom teaching, university teaching, publishing innovative curricula, and presenting professional development workshops and service. references alerby, e. (2000). a way of visualizing children’s and young people’s thoughts about the environment: a study of drawings. environmental education research, 6(3), 205-222. ambrose, r., clement, l., philipp, r., & chauvot, j. (2004). assessing prospective elementary school teachers’ beliefs about mathematics and mathematics learning: rationale and development of a constructed-response-format beliefs survey. school science and mathematics, 104(2), 56-70. ambrose, r. (2004). initiating change in prospective elementary school teachers’ orientations to mathematics teaching by building on beliefs. journal of mathematics teacher education, 7, 191-119. ball, d. l. (1990). the mathematical understandings that prospective teachers bring to teacher education. elementary school journal, 90, 449-466. battista, m. t. (1999). the mathematical miseducation of america’s youth [electronic version]. phi delta kappan, 80(6), 424-433. burns, m. (1998). math: facing an american phobia. math solutions publications. burton, m. (2012). what is math? exploring the perception of elementary pre-service teachers. iumpst: the journal, vol 5 (teacher attribute). [www.k-12prep.math.ttu.edu] international electronic journal of elementary education vol.6, issue 2, 333-346,2014 344 cooney, t. (1999). conceptualizing teachers’ ways of knowing. educational studies in mathematics, 38, 163-187. doerr, h.m. & lesh, r. (2003) a modeling perspective on teacher development. in r. lesh & h.m. doerr (eds.). beyond constructivism: a models and modeling perspective on mathematics problem solving, learning and teaching (pp.125-139). hillsdale, nj: lawrence erlbaum. eisenhart, m., borko, h., underhill, r., brown, c., jones, d. & agard, p. (1993). conceptual knowledge falls through the cracks: complexities of learning to teach mathematics for understanding. journal for research in mathematics education, 24(1), 8-40. finson, k. d. (2002). drawing a scientist: what we do and do not know after fifty years of drawings. school science and mathematics, 102(7), 335-345. frykholm, j. a. (1999). the impact of reform: challenges for mathematics teacher preparation. journal of mathematics teacher education, 2, 79-105. hinchey, p. h. (1998). finding freedom in the classroom: a practical introduction to critical theory. new york: peter lang. kendrick, m., & mckay, r. (2004). drawings as an alternative way of understanding young children’s constructions of literacy. journal of early childhood literacy, 4(1), 109-128. langer, e. j. (1989). mindfulness. reading, ma: addison-wesley. lingren, j. & bleicher, r. e. (2005). learning the learning cycle: the differential effect on elementary preservice teachers. school science and mathematics, 105(2), 61-72. ma, l. (1999). knowing and teaching elementary mathematics: teachers' understanding of fundamental mathematics in china and the united states. mahwah, nj: erlbaum. mcleod, d. b. (1992). research on affect in mathematics education: a reconceptualization. in d.a. grouws (ed.). handbook of research on mathematics teaching and learning (pp.575596). new york: macmillan. miles, m. b., & huberman, a. m. (1994). qualitative data analysis: an expanded sourcebook. thousand oaks, ca: sage. mitchell, c., theron, l., stuart, j., smith, a., & campbell, z. (2011). drawing as research method. in l. theron et al (eds.), picturing research: drawing as visual methodology (pp. 1936). rotterdam, netherlands: sense publishers. munby, h., russell, t., & martin, a. (2001). teachers’ knowledge and how it develops. in v. richardson (ed.), handbook of research on teaching (4 th ed.) (pp.877-904). washington, dc: american educational research association. national council of teacher of mathematics. (2000). the principles and standards for school mathematics. reston, va: nctm. newstead, k. (1998). aspects of children’s mathematics anxiety. educational studies in mathematics , 36, 53 – 71. picker, s. h. & berry, j. s. (2000). investigating pupils’ images of mathematicians. educational studies in mathematics, 43(1), 65-94. using drawings to bridge / lee & zeppelin 345 rule, a. c. & harrell, m. h. (2006). symbolic drawings reveal changes in preservice teacher mathematics attitudes after a mathematics methods course. school science and mathematics, 106(6), 241-258. schoen, h. l., fey, j. t., hirsch, c. r., & coxford, a. f. (1999). issues and options in the math wars. phi delta kappan, 80(6), 444-453. schön, d. a. (1983). the reflective practitioner: toward a new design for teaching and learning in the professions. san francisco: jossey-bass. simon, m. (1995). reconstructing mathematics pedagogy from a constructivist perspective. journal for research in mathematics education, 26(2), 114-145. stodolsky, s. s. & grossman, p. l. (2002). changing students, changing teaching. teachers college record, 102(1), 125-172. strauss, a., & corbin, j. (1998). basics of qualitative research: techniques and procedures for developing grounded theory (2 nd ed.). thousand oaks, ca: sage publication. taylor, b.a., & fraser, b.j.(2003). the influence of classroom environment on high school students’ mathematics anxiety. (eric document reproduction service no. ed 476644). thompson, a. (1992). teachers’ beliefs and conceptions: a synthesis of the research. in d. grouws (ed.), handbook of research on mathematics teaching and learning (pp.127146). new york: macmillan. trice, a. d., & ogden, e. d. (1986/87). correlates of mathematics anxiety in first-year elementary school teachers, educational research quarterly, 11(3), 3-4. trujillo, k.m., & hadfield, o.d. (1999). tracing the roots of mathematics anxiety through indepth interviews with preservice elementary teachers. college student journal, 33 (2), 219232. zambo, d. & zambo, r. (2006). using thought bubble pictures to assess students’ feelings about mathematics. mathematics teaching in the middle school, 12(1), 14-21. international electronic journal of elementary education vol.6, issue 2, 333-346,2014 346 www.iejee.com this page is intentionally left blank international electronic journal of elementary education, june 2016, 8(4), 675-686. issn:1307-9298 copyright © iejee www.iejee.com an exploratory study of the relationships between reading comprehension competence, reading attitude and the vividness of mental imagery among turkish fourth-grade students* mustafa kocaarslan**a a university of bartin, turkey received: 2 february 2016 / revised: 11 march 2016 / accepted: 18 june 2016 abstract this study aimed to establish the possible relationships between reading comprehension competence, reading attitude and the vividness of mental imagery among turkish fourth-grade students. participants were fourth grade students, selected using convenience sampling from two different public schools (n=103) in bartin, turkey. the research was designed as a correlational study to describe the degree to which two or more quantitative variables are related; this was achieved by using a correlation coefficient. the data gathering tools used in this study were (1): the reading comprehension test (rct), which was developed by the researcher and which includes both factual and inferential questions; (2) the vividness of imagery questionnaire (viq), developed by sheveland (1992); (3) the elementary reading attitude survey (eras), developed by mckenna and kear (1990). multiple linear regressions were used for data analysis. the analysis revealed that reading attitude and vividness of mental imagery were significant predictors for students’ reading comprehension competencies. in addition, the results indicated that approximately 14% of the variance in reading comprehension competence could be explained by reading attitude and vividness of mental imagery [r=.369, r2=.136, f(2-96)=7.578, p<.01]. keywords: reading comprehension competence, reading attitude, vividness of mental imagery, fourth-grade students introduction in recent years, research pertaining to reading comprehension has received much attention and has developed into one of the most popular areas where the act of reading is concerned (aloqaili, 2012). according to a report by the national reading panel, published by the national institute of child health and human development (nichd) (2000), reading comprehension is identified as one of the five most important components in reading instruction. * this paper was originally presented at the international conference on new horizons in education (inte). barcelona, spain, 10-12 june, 2015. ** corresponding author: mustafa kocaarslan, ph.d., university of bartin, faculty of education, primary school teacher training, bartin, turkey. phone: +90378223-5453, email: mustafakocaarslan@gmail.com http://www.iejee.com/ international electronic journal of elementary education vol.8, issue4, 675-686, june 2016 676 most educators would agree that reading comprehension is extremely important (jenkins, 2009). mason (2004) claimed reading comprehension to be essential not only in academic learning, but also in life-long learning. reading comprehension is viewed as the “essence of reading” (durkin, 1993) and the “ultimate goal of reading education” (nation, 2005). research indicates that a host of factors affect the reading comprehension process, e.g., the reader, text, working memory, context and vocabulary (baumann, 2009; snow, 2002; stahl and nagy, 2006). in addition, evidence from several sources and studies report a relationship between reading comprehension and reading attitude (conlon, zimmerbembeck, creed & tucker, 2006; diamond & onwuegbuzie, 2001; martinez, aricak & jewell, 2008; mckenna, kear & ellsworth, 1995; ogle, sen, pahlke, jocelyn, kastberg, roey & williams, 2003), and that mental imagery is an effective variable in the reading comprehension process (bourduin, bourduin & manley, 1993; gambrell & bales, 1986; gambrell & jawitz, 1993; jenkins, 2009; kocaarslan, 2015; oakhill & patel, 1991; rose, parks, androes & mcmahon, 2000). the focus of this research was the relationships between terms that included the following: reading comprehension, reading attitude and vividness of mental imagery. theoretical explanations related to these concepts are outlined below. reading comprehension in the past and present, reading has been defined in many different ways. the term 'reading' has for many years in a narrow sense been used to refer to a set of print-based decoding and thinking skills that are necessary for understanding text (harris & hodges, 1981). snow, burns, and griffin (1998) broadened this concept by denoting reading as a complex developmental challenge that we know to be intertwined with many other developmental accomplishments, e.g., attention, memory, language and motivation. reading is not only a cognitive psycholinguistic activity, but also a social activity (p.15). one of the most important skills learned by young students is the ability to understand written text, generally referred to as reading comprehension. comprehension of the information within a text, or of its author’s meaning, is the ultimate reason for reading (nichd, 2000). research studies on reading comprehension have revealed reading to be a complex cognitive activity that is crucial for adequate functioning and for obtaining information in current society, and a process that requires the integration of memory and meaning construction (alfassi, 2004). according to woolly (2011), reading comprehension is the process of making meaning from text. the goal, therefore, is to gain an overall understanding of what is described in the text, rather than to obtain meaning from isolated words or sentences (p.15). good readers have a purpose for reading and use their experiences and background knowledge to comprehend the text. making connections is key to comprehension. we don’t comprehend unless we make connections and are able to process the words that we read at a cognitive level (tankersley, 2003, p.90). the rand reading study group (2002) notes that comprehension is “the process of simultaneously extracting and constructing meaning through interaction and involvement with written language” (p.11). according to the rand reading study group (2002), reading comprehension consists of four components: (1) the reader; (2) the text; (3) the activity; (4) the situational context (p.1). the first three essential components – reader, text and task – occur within the fourth component of reading comprehension, i.e., the situational context. the reader is the one partaking in comprehending and the text is the reading material (e.g., stories, nonfiction selections, etc.). the specific activity refers to what type of reading comprehension competence, reading attitude, and the vividness of mental imagery / kocaarslan 677 comprehension task, skill, strategy or concept the reader is attempting to perform (reutzel & cooter, 2012, p.259). vividness of mental imagery kosslyn (1980) interprets images as functional, quasi-pictorial representations, the special properties of which can affect cognitive processes and characterizes these quasi-pictorial representations as transitory data structures in quasi-pictorial representations as transitory data structures within an analogue spatial medium (p.91). mental imagery refers to any image created in the mind without the presence of the real object or event, including images related to sight, sound, taste, touch, smell, sensations and feelings (jenkins, 2009, p.3). vivid mental imagery has been defined as the ability to create pictures in the mind’s eye, a form of mental representation with visual properties, as well as a cognitive strategy used to form associations for learning, retrieval and problem-solving tasks (hodes, 1990). research has distinguished vivid mental imagery as a significant contributor to meaningful reader-text interaction. long, winograd and bridge (1989) describe imagery as a “naturally occurring part of the reading experience” (p.364). the “vividness” of mental imagery can determine the degree to which a person will undergo significant experiential, behavioural and physiological changes during cognition (macomber, 2001). 'vividness' is a word used to describe the extent to which a person’s images are life-like. developers of mental imagery measurements use a scale of vividness to determine how much elaboration or clarity a person has while imaging (marks,1990), as well as the degree of strength, definiteness and intensity of images (gordon, 1972). reading attitude eagly and chaiken (1993) state that “attitudes [are] a psychological tendency that [are] expressed by evaluating a particular entity with some degree of favour or disfavour” (p.1). specifically, reading attitude is defined by alexander and filler (1997) as, “a system of feelings related to reading which causes the learner to approach or avoid a reading situation” (p.934). similarly, smith (1990) notes that reading attitude is “a state of mind, accompanied by feelings and emotions that makes reading more or less probable” (p.215). both of these reading attitude definitions assume that the more positive the attitude towards reading, the more likely one will engage in reading activities. positive attitudes have also been found to be associated with higher reading achievement (mckenna, kear & ellsworth, 1995). as individual factors, the relationship between reading comprehension, reading attitude and vividness of mental imagery has been well-documented in the aforementioned research through the application of various correlational analyses, as well as through true experiments. however, there are no available studies examining the correlation between the three variables together. for this reason, the purpose of this study was to explore whether there was a significant correlation between reading comprehension competence, reading attitude and the vividness of mental imagery among fourth-grade students, as well as whether reading attitude and vividness of mental imagery were significant predictors of reading comprehension. the following research questions were examined in the current study.  rq1: is there a significant correlation between fourth-grade students' reading comprehension competence, reading attitude and the vividness of mental imagery?  rq2: what is the predictive level of fourth-grade students' reading attitudes and the vividness of mental imagery for their reading comprehension competence? international electronic journal of elementary education vol.8, issue4, 675-686, june 2016 678 method this study was designed as a correlational study describing the degree to which two or more quantitative variables are related. it achieved this by using a correlation coefficient. in correlational research, the relationships among two or more variables are studied without any attempt to influence them (fraenkel, wallen & hyun, 2012, p.331). subjects the subjects in this study were 103 fourth-grade students, who were selected using convenience sampling from two different public schools in bartın, turkey, during the 20142015 academic year. convenience sampling is a non-random sampling method that includes a group of individuals who (conveniently) are available for study (fraenkel, wallen & hyun, 2012, p.99). the two public schools were similar in terms of physical environment, classroom sizes, student ses and parental education. students were primarily from middle and low-ses families. among the subjects, 50 were girls and 53 were boys and the average age was 10. instruments the data collection instruments used in this study were the reading comprehension test (rct) developed by kocaarslan (2015), the vividness of imagery questionnaire (viq) developed by sheveland (1992) and the elementary reading attitude survey (eras) developed by mckenna and kear (1990). information about these instruments is given below. reading comprehension test (rct) the reading comprehension test (rct) developed by kocaarslan (2015) comprises two texts that include both narrative and expository text, and a total of 20 open-ended questions related to these texts. the rct is composed of open-ended questions aimed at understanding both factual and inferential knowledge related to the texts. the chosen texts were judged by four classroom teachers in accordance with a rubric. the rubric assessed the prior knowledge, word difficulty, sentence structure, punctuation and the grammar to be evaluated, according to students’ grade level. according to assessments regarding the classroom teachers, both texts were judged appropriate for the fourth-grade level. for assessments made by the classroom teachers, kendall’s w coefficient of concordance was computed. kendall (1955) presented this coefficient of concordance, w, to evaluate the extent of agreement among a set of judges, each of whom ranks in its entirety a set of objects. this statistic is well-known and has been widely used in socio-metric literature. the possible values for kendall's w ranges from .00 (no agreement) to +1.00 (complete agreement). in this study, the coefficient demonstrated that there was statistically significant agreement among the judgments made by classroom teachers (w= .70, p<.01). when scoring the questions, two points were given for “complete and accurate answers”, one point for “inadequate answers” and zero points for “incorrect answers”. two independent raters who specialized in reading education scored students’ answers. the cohen kappa coefficient, a statistic that measures inter-rater agreement for qualitative (categorical) items, was computed for each student’s reading comprehension scores. among two independent raters, cohen’s kappa coefficient (k) for students’ reading comprehension scores in the current study ranged from .72 to .80. altman (1991) suggests that the value of k, when ranging from .61 to .80, infers “good” in terms of strength of agreement. furthermore, each reading text was prepared using an appropriate font, style and language for fourth-grade students. reading comprehension competence, reading attitude, and the vividness of mental imagery / kocaarslan 679 vividness of imagery questionnaire (viq) the vividness of imagery questionnaire (viq) developed by sheveland (1992) and adapted into turkish by kocaarslan (2015) was employed for detecting the subject’s clarity of mental imagery. this questionnaire instrument consists of 21 likert items, each with five possible responses ranging from "very clear: vivid" (4-point) to "no image, just know" (0point) and evaluates the vividness of evoked imagery according to seven sensory modalities: visual, auditory, coetaneous, kinaesthetic, gustatory, olfactory and organic (sheveland, 1992; kocaarslan, 2015). in the context of the current study, a cronbach's alpha reliability coefficient for the questionnaire was computed as .91. a higher total score on the questionnaire indicates a higher level of vividness of mental imagery. elementary reading attitude survey (eras) the elementary reading attitude survey (eras) developed by mckenna and kear (1990) and adapted into turkish by kocaarslan (2015) was employed to measure subjects' recreational and academic reading attitudes. the eras is a 20-item survey that is five pages long, with four questions on each page. a four-choice picture-rating scale is based on the 'garfield' cartoon character. there are two 10-item subscales in this survey consisting of recreational and academic reading attitude. the eras provides a quick indication of student attitudes toward reading and can be administered to an entire classroom in roughly 10 minutes. each item presents a brief, simply-worded statement about reading, followed by four pictures of garfield. each character pose is designed to depict a different emotional state, ranging from very positive to very negative. according to mckenna and kear (1990), a pictorial format was selected due to its natural appeal to children and because of its comprehensibility by the very young. to score the survey, count four points for each leftmost (happiest) garfield circled, three points for each slightly smiling garfield, two points for each mildly upset garfield and one point for each very upset (rightmost) garfield. three scores for each student can be obtained: the total for the first 10 items, the total for the second 10 and a composite total. the first half of the survey relates to attitudes toward recreational reading; the second half relates to attitudes toward academic aspects of reading. a total raw score of 50 represented the midway point on the scale, indicating a relatively indifferent attitude toward reading (mckenna & kear, 1990). a cronbach's alpha reliability coefficient for the survey in this study was calculated as .88. procedure three tests were administered to all students: the vividness of imagery questionnaire (viq), the elementary reading attitude survey (eras) and a reading comprehension test (rct)”. viq and eras were administered on the same day, while the rct was applied one week after. students were given 30 minutes to complete the viq and eras. for the rct, students had 40 minutes (20 minutes for reading and 20 minutes for answering questions) to complete the test. directions were read at the beginning of the test. first, students were given the narrative text and were allowed to read the text silently for 20 minutes. at the end of the 20 minutes, the narrative texts were taken from the students and question sheets were distributed. students were given 20 minutes to answer the questions. a few minutes after all students had finished answering the questions related to the narrative text, other questions related to exposition were administered in a similar manner. statistical analysis the data collected during the study were analysed using spss version 21.0. initially, descriptive statistics for variables in the study were determined. the relationships between viq, eras and rct scores in the entire sample were examined using the pearson productmoment correlation coefficient (r). correlation analysis was employed to describe the international electronic journal of elementary education vol.8, issue4, 675-686, june 2016 680 strength and direction of the linear relationship between two variables. the pearson product-moment correlation coefficient (r) can only take on values ranging from -1 to +1. the sign out the front indicated whether there was a positive correlation (as one variable increases, the other decreases) or a negative correlation (as one variable increases, the other decreases). the size of the absolute value (ignoring the sign) provided an indication of the strength of the relationship. a perfect correlation of 1 or -1 indicated that the value of one variable can be determined exactly by knowing the value of the other variable (pallant, 2007, p.126). subsequently, a multiple regression analysis was conducted to determine the prediction level of subjects' reading attitudes and the vividness of mental imagery for their reading comprehension competences. multiple regression does not consist of only one technique but a family of techniques that can be used to explore the relationship between one continuous dependent variable and a number of independent variables or predictors. multiple regression is based on correlation, but allows for a more sophisticated exploration of the interrelationships among a set of variables (pallant, 2007, p.146). in many applications, when researchers present multiple regression results, they also introduce bivariate and partial correlation results (green & salkind, 2005, p.283). results descriptive statistics for variables in the study were determined for the entire sample. these statistics such as mean (m), standard deviation (sd), skewness, and kurtosis are presented in table 1. additionally, normality assumption was checked using skewness and kurtosis values for the rct, viq and eras scores, as summarized in table 1. these values were between -2 and +2; that is, they were in the acceptable range (george & mallery, 2001). therefore, it can be said that all scores were normally distributed. table 1. descriptive statistics for variables in the study variables m sd skewness kurtosis rcta 57.66 15.20 -1.905 1.324 viqb 71.59 11.82 -1.986 .158 erasc 69.67 7.98 -.868 .627 note. rct = reading comprehension test; viq = vividness of mental imagery; eras = elementary reading attitude survey. an= 103. bn= 103. cn=103. in this part of the research findings, initially, the correlation between subjects' rct, eras and viq was calculated using the pearson product-moment correlation coefficient (r) in order to first answer sub-questions of the research. the first sub-question of the research was expressed as, “is there a significant correlation between fourth-grade students' reading comprehension competence, reading attitude and the vividness of mental imagery?” the results of this analysis are displayed in table 2. table 2. bivariate correlations among study variables variables 1. rct 2. eras 3. viq 1. rcta — 2. erasb .315* — 3. viqc .237* .340* — note. rct = reading comprehension test; viq = vividness of mental imagery; eras = elementary reading attitude survey. an= 103. bn= 103. cn=103. correlation is significant at the 0.01 level (p<.01). reading comprehension competence, reading attitude, and the vividness of mental imagery / kocaarslan 681 as table 2 demonstrates, pearson product-moment correlation coefficients (r) between rct and eras, rct and viq, and eras and viq were .31 (p<.01), .23 (p<.01) and .34 (p<.01), respectively. thus, there was a significant positive medium correlation between students' rct and eras [r=.31, p<.01]. in other words, it can be stated that an increase in the total score of the reading attitudes affected students’ reading comprehension competences positively and vice versa. similarly, a significant positive medium correlation existed between students' eras and viq [r=.34, p<.01]. in other words, it can be stated that an increase in the total score of reading attitudes affected students’ vividness of mental imagery positively and vice versa. regarding the relationship between student’s rct and viq, there was a significant positive small correlation [r=.23, p<.01]. in other words, it can be stated that an increase in the total score of the vividness of mental imagery affected students’ reading comprehension competencies positively and vice versa. the second sub-question of the research was expressed as, “what is the predictive level of fourth-grade students' reading attitudes and the vividness of mental imagery for their reading comprehension competence?” multiple linear regression analyses were conducted to answer the second sub-question of the research. the results of this analysis are displayed in table 3. table 3. multiple regression analysis results variables b std. error β t constant 1.277 14.762 — .087 erasa .473 .195 .245 2.430 viqb .328 .161 .205 2.035 note. r=.369, r2=.136, f(2-96)=7.578, p<.05. viq = vividness of mental imagery; eras = elementary reading attitude survey. an= 103. bn= 103. as is shown in table 3, the multiple regression analyses results revealed that reading attitude and vividness of mental imagery were significant predictors for students’ reading comprehension competencies. in addition, the results indicated that approximately 14% of the variance in reading comprehension competence could be explained by reading attitude and vividness of mental imagery [r=.369, r2=.136, f(2-96)=7.578, p<.05]. discussion reading comprehension underlines one of the most important aspects of reading education, given the role it plays in school achievement and daily life. the main aim of the present research was to explore the process of reading comprehension by analysing its relationship with the vividness of mental imagery and reading attitude. in the this study, two questions were expressed regarding the relationship between measures of reading attitude, vividness of mental imagery and reading comprehension among turkish fourth-grade children. in the first sub-question of the research, the correlation analyses showed that there was a significant positive medium relation between students' reading comprehension competencies and reading attitudes. this finding is consistent with previous study results that have indicated a positive correlation between reading comprehension and reading attitude scores (mckenna, kear & ellsworth, 1995; conlon, zimmer-bembeck, creed & tucker, 2006; martinez, aricak & jewell, 2008). the relationship between attitudes toward reading and reading achievement has been clearly determined in multiple studies (conlon, zimmer-bembeck, creed & tucker, 2006; martinez, aricak & jewell, 2008; mckenna, kear & ellsworth, 1995; ogle, sen, pahlke, jocelyn, kastberg, roey & williams, 2003). in each case, students with more positive reading attitudes tended to have higher achievement levels with regard to reading. international electronic journal of elementary education vol.8, issue4, 675-686, june 2016 682 in mckenna, kear and ellsworth’s research (1995), which is considered one of the landmark studies to have investigated the relationship between reading attitudes and other components of reading, findings revealed a positive correlation between students’ reading attitudes and reading achievement. in addition, this research concludes that reading attitudes and achievement scores will have an impact on one another. to extend this work and further examine the relationship between reading attitudes and achievement, martinez, aricak and jewell (2008) compared the scores of 76 fourth-graders as it concerned reading attitudes and reading achievement. martinez, aricak and jewell (2008) obtained results that were compatible to those of mckenna, kear and ellsworth (1995). the findings of the current study are also consistent with a study by ogle et al. (2003), which determined the strongest correlation between fourth-grade attitudes toward recreational reading and later achievement scores. in their study, conlon, zimmer-bembeck, creed and tucker (2006) examined family histories and the cognitive abilities and attitudes of 190 elementary school students. the results of this study indicated a correlation between reading attitude and word recognition, comprehension and spelling skills. diamond and onwuegbuzie (2001) conducted a study to determine the factors associated with reading achievements and attitudes among elementary school-aged students. the results demonstrated a small but statistically significant relationship between reading achievement and attitudes toward reading. the results of these studies were in agreement with the findings of the current study. the first sub-question of the present research also examined the relationship between students' reading comprehension and the vividness of mental imagery. a small yet significantly positive correlation was found between these two variables. this finding is supported by the results of previous studies by bourduin, bourduin & manley (1993), gambrell & bales (1986), gambrell & jawitz (1993), jenkins (2009), kocaarslan (2015), oakhill & patel (1991) and rose, parks, androes & mcmahon (2000), which showed that students who employed mental imagery as a reading strategy performed better on reading assessments than students who did not. researchers such as goetz, sadoski, fatemi and bush (1994) claim that mental imagery plays an important part in the complex task of reading. the present study also evaluated the relationship between students' reading attitudes and the vividness of mental imagery. the current study’s results showed a significantly positive medium correlation between these two variables. several studies point to the relationship between mental imagery and text appreciation (long, winograd & bridge, 1989; macomber, 2001; sadoski & quast, 1990). previous research has demonstrated significant correlations between vivid mental imagery created during reading and reading attitudes (cramer, 1980; irwin, 1979; gunston-parks, 1985; macomber, 2001). irwin (1979) reports that the reading attitudes of high school students were significantly correlated to self-reported vividness of mental imagery. cramer (1980) employed elementary students and showed that a reader's general attitude toward reading was significantly related to comprehension, as well as imagery vividness. macomber (2001) employed middle school participants and indicated that students with more vivid mental images had better attitudes towards reading. sadoski and paivio (2001) claim that when students experience mental imagery, they tend to find the text more interesting, more comprehensible and more memorable. the second sub-question of the research investigated the predictive level of fourth-grade students' reading attitudes and the vividness of mental imagery for their reading comprehension skills. the multiple regression analyses results indicated that reading attitude and vividness of mental imagery were significant predictors of students’ reading comprehension competencies. in addition, the results indicated that approximately 14% of the variance in reading comprehension competence could be explained by reading attitude and vividness of reading comprehension competence, reading attitude, and the vividness of mental imagery / kocaarslan 683 mental imagery [r=.369, r2=.136, f(2-96)=7.578, p<.05]. in the above-mentioned research, it was found that reading attitude and vividness of mental imagery were significant variables for reading comprehension. cramer (1980) found that a reader’s general attitude toward reading was significantly related to a combined score for imagery vividness and comprehension. in his study, kocaarslan (2015) conducted experimental research to investigate the effect of mental imagery instruction as a strategy for reading comprehension. the results demonstrated that fourth-graders who had been trained in mental imagery where the language arts were concerned had better scores on reading comprehension measures, as well as better reading attitudes. as shown, the present research findings are consistent with existing research literature. limitations of the research and directions for future research this study emphasized the significance of the vividness of mental imagery in reading in terms of predicting reading comprehension. this variable has often been neglected in previous research. in this context, teachers should encourage students to create mental images during reading in order to better understand text. they should also endeavour to help students develop positive attitudes towards reading. this study was limited to fourth-grade students only in bartin/turkey. for this reason, it cannot be said that participants represented the entire population. to achieve more comprehensive data, further research might involve students studying at different grades and a number of different variables such as word knowledge, reading motivation and prior knowledge. • • • references alexander, j.e. & filler, r.c. (1976). attitudes and reading. newark, de: international reading association. alfassi, m. (2004). reading to learn: effects of combined strategy instruction on high school students. journal of educational research, 97(4), 171-184. doi:10.3200/joer.97.4.171-185 altman, d.g. (1991). practical statistics for medical research. london: chapman and hall. baumann, j. f. (2009). intensity in vocabulary instruction and effects on reading comprehension. topics in language disorders, 29(4), 312-328. doi: 10.1097/tld.0b013e3181c29e22 conlon, eg. zimmer-gembeck, mj., creed, pa. & tucker, m. (2006). family history, self-perceptions, attitudes and cognitive abilities are associated with early adolescent reading skills. journal of research in reading, 29(1), 11-32. doi:10.1111/j.1467-9817.2006.00290.x cramer, e. h. (1980). mental imagery, reading attitude and comprehension. reading improvement, 17(2), 135-139. retrieved from http://eric.ed.gov/?id=ej225276 diamond, p.j. & onwuegbuzie, a.j. (2001). factors associated with reading achievement and attitude among elementary. school-aged students. research in the schools, 8(1), 1-11. retrieved from http://eric.ed.gov/?id=ej663797 durkin, d. (1993). teaching them to reading (6th ed.) .boston: allyn & bacon. international electronic journal of elementary education vol.8, issue4, 675-686, june 2016 684 eagly, a. h. & chaiken, s. (1993). the psychology of attitudes. fort worth, tx: harcourt brace jovanovich. fraenkel, j., wallen, n. & hyun, h.h. (2012). how to design and evaluate research in education (8th ed.). boston: mcgraw hill. gambrell, l.b. & bales, r.j. (1986). mental imagery and the comprehension-monitoring performance of fourth and fifth-grade poor readers. reading research quarterly, 21, 454-464. doi: 10.2307/747616 george d. & mallery, p. (2003). spss for windows step by step: a simple guide and reference. boston: allyn and bacon. goetz, e. t., sadoski, m., olivarez, jr., a., calero-breckheimer, a., garner, p., fatemi, z. (1992). the structure of emotional response in reading a literary text: quantitative and qualitative analyses. reading research quarterly, 27(4), 360372. doi: 10.2307/747675 gordon, r. (1972). a very private world. in p.w. sheehan (ed), the function of nature of imagery (pp. 64-82). new york: academic press. green, s. b. & salkind, n. j. (2005). using spss for windows and macintosh: analyzing and understanding data. upper saddle river, nj: pearson/prentice hall. harris, t. l., hodges, r. e. & international reading association. (1981). a dictionary of reading and related terms. newark, del: international reading association. hodes, c. l. (1990). the induction use, and effectiveness of mental imagery as an instructional variable. dissertation abstracts international, 5109a, 3017. irwin, j. w. (1979). college readers' mental imagery, reading ability and attitude toward reading. reading improvement, 16(2), 124-129. retrieved from http://eric.ed.gov/?id=ej207922 jenkins, m. h. (2009). the effects of using mental imagery as a comprehension strategy for middle school students reading science expository texts. (doctoral dissertation) http://drum.lib.umd.edu/bitstream/handle/1903/9551/jenkins kendall, m.g. (1955). rank correlation methods. new york: hafner. kocaarslan, m. (2014, may). the turkish adaptation of the elementary reading attitude scale: a validity and reliability study. paper presented at the 13rd national primary teaching symposium. kütahya, turkey. kocaarslan, m. (2015). the impact of teaching mental imagery on reading comprehension skills of fourth graders. (doctoral dissertation, gazi university, ankara, turkey). retrieved from https://tez.yok.gov.tr/ulusaltezmerkezi/ kosslyn, s. m. (1980). image and mind. cambridge, ma: harvard university press. levin, j. r. (1981). on functions of picture in prose. in f. pirozzolo & m. wittrock (eds.), neuropsychological and cognitive processes in reading (pp. 203-288). new york: academic press. long, s. a., winograd, p. n. & bridge, c. a. (1989). the effects of reader and text characteristics on imagery reported during and after reading. research quarterly, 24(3), 353-372. doi: 10.2307/747774 macomber, c. (2001). the use of mental imagery and its relation to reading attitudes among fourth grade students. (doctoral dissertation). available from proouest dissertations and theses database. (umi no. 3004915) marks, d. f. (1990). on the relationship between imagery, body, and mind. in p. j. hampson, d. f. marks & j. e. richardson (eds.), imagery: current developments, (pp. 1-38). new york: routledge. martínez, rs., aricak, ot. & jewell, j. (2008). influence of reading attitude on reading achievement: a test of the temporal-interaction model. psychology in the schools, 45(10), 1010-1023. doi:10.1002/pits.20348 reading comprehension competence, reading attitude, and the vividness of mental imagery / kocaarslan 685 mason, l. h. (2004). explicit self-regulated strategy development versus reciprocal questioning: effects on expository reading comprehension among struggling readers. journal of educational psychology, 96(2), 283-296. mckenna, m. c. & kear, d. j. (1990). measuring attitude toward reading: a new tool for teachers. the reading teacher. 43(9), 626-629. retrieved from http://www.jstor.org/stable/20200500 mckenna, m., kear, d. & ellsworth, r. a. (1995). children's attitudes toward reading: a national survey. reading research quarterly, 30, 934-956. doi: 10.2307/748205 nation, k. (2005). children's reading comprehension difficulties. in m. j. snowing and c. hulme (eds.) the science of reading: a handbook (pp. 248-266). oxford: blackwell. national institute of child health and human development (2000). report of the national reading panel. teaching children to read: an evidence-based assessment of the scientific research literature on reading and its implications for reading instruction: reports of the subgroups (nih publication no. 00-4754). washington, dc: u.s. government printing office. oakhill, j. & patel, s. (1991). can imagery training help children who have comprehension problems? journal of research in reading, 14(2), 106-115. doi: 10.1111/j.1467-9817.1991.tb00012.x ogle, l. t., sen, a., pahlke, e., jocelyn, l., kastberg, d., roey, s. & williams, t. (2003).international comparisons in fourth-grade reading literacy: findings from the progress in international reading literacy study of 2001. washington, d.c.: national center for education statistics. retrieved from http://nces.ed.gov/pubs2003/2003073.pdf pallant, j. (2007). spss survival manual: a step by step guide to data analysis using spss for windows. maidenhead: open university press. pressley, g.m. (1976). mental imagery helps eight-year-olds remember what they read. journal of educational psychology, 68, 355-359. retrieved from http://files.eric.ed.gov/fulltext /ed123595.pdf quarterly, 21, 454-464. gambrell, l.b. & jawitz, p.b. (1993). mental imagery, text illustrations, and children's story comprehension and recall. reading research quarterly, 28, 264-276. doi: 10.2307/747998 rand reading study group. (2002). reading for understanding: toward a research and development program in reading comprehension. santa monica, ca: office of education research and improvement. retrieved from http://www.jstor.org/stable/10.7249/mr1465oeri reutzel, d. r. & cooter, r. b. (2012). teaching children to read: the teacher makes the difference. boston: pearson. rose, d. s., parks, m., androes, k. & mcmahon, s. d. (2000). imagery-based learning: improving elementary students' reading comprehension with drama techniques. journal of educational research, 94(1), 55-64. doi: 10.1080/00220670009598742. sadoski, m. (1983). an exploratory study of the relationships between reported imagery and the comprehension and recall of a story. reading research quarterly, 19(1), 110-123. doi: 10.2307/747341 sadoski, m. & paivio, a. (2001). imagery and text: a dual coding theory of reading and writing. mahwah, nj: lawrence erlbaum associates, inc sheveland, d. e. (1992, december). development of sheveland vividness of imagery questionnaire in grades 3-6. paper presented at the 42nd annual meeting of the national reading conference, san antonio, usa. retrieved from http://files.eric.ed.gov/fulltext/ed352634.pdf smith, m. c. (1990). a longitudinal investigation of reading attitude development from childhood to adulthood. journal of educational research, 83, 215–219. retrieved from http://www.jstor.org/stable/27540386 snow, c. e. (2002). reading for understanding: toward an r&d program in reading comprehension. santa monica, ca: rand corporation. international electronic journal of elementary education vol.8, issue4, 675-686, june 2016 686 snow, c.e., burns, s. & griffin, p. (eds.). (1998). preventing reading difficulties in young children. washington, dc: national academy press. stahl, s. a. & nagy, w. e. (2006). teaching word meanings. mahwah, nj: lawrence erlbaum. tankersley, k. (2003). the threads of reading: strategies for literacy development. alexandria, va: association for supervision and curriculum development. woolley, g. (2011). reading comprehension: assisting children with learning difficulties. new york: springer. international electronic journal of elementary education, 2016, 8(3), 351-362 how does supervision support inclusive teacherhood? sanna alila valteri centre for learning and consulting, finland kaarina määttä university of lapland, finland satu uusiautti university of lapland, finland received: 28 september 2015 / revised: 19 november 2015 / accepted: 19 november 2015 abstract supervision is a multidimensional concept and phenomenon. in this study, the advantages of supervision and its development in inclusive teacherhood was studied. inclusive teacherhood means a teacher’s professional development and the school culture’s change toward participatory school for all students. the study analyzed the views of supervisors with a teaching background on how supervision can be a way to support inclusive teacherhood and its development. this was a qualitative research. the data were obtained using the focus-group interview method focused on supervisors with a teaching background. the interviews were conducted in five places in finland. the analysis involved a combination of phenomenography, particularly the application called the variation theory, and the classic analysis that is typical of the focus-group research method. according to the findings, supervision provides individual and communal support to inclusive teacherhood. individual support was given in four ways: empowering and promoting new teacherhood, clarifying teachers’ professional growth and roles, helping teachers to evaluate their work, and supporting teachers in challenges at work. communal support was manifested as strengthening collaboration, promoting a change in the work culture of a school, and developing a communal work approach. at its best, supervision can enable teachers’ professional, communal, and personal development in an inclusive learning environment, but more time, resources, and opportunities for supervision should be arranged for teachers. keywords: supervision, inclusive teacherhood, supervisor with a teaching background, focus group interviews. introduction supervision in the field of teaching means guidance offered for teachers to support their professional growth. at the moment, supervision has an established position as a wellknown and widely-applied method in health care and nursing internationally (e.g., brunero & stein-parbury, 2008; milne & james, 2002), but in education and teaching, it  sanna alila, tilustie 4 d 1, 90650 oulu, finland. phone: +358407497262, e-mail: sanna.alila@tervavayla.fi international electronic journal of elementary education vol.8, issue 3, 351-362, 2016 352 has been less used. for example in finland, teachers’ supervision has been limited, for example, due to lack of funding and scheduling (alila, 2014). as the teacher’s profession becomes more and more versatile, the support provided by supervision can turn out to be quite necessary. especially, the enhancement of inclusion requires the adaptation of new kinds of thinking, attitudes, and methods from teachers (ainscow, 2005). in finland, supervision has been used for supporting professional growth and research for several decades (alila, 2014). although supervision in the field of teaching has been available, it has been established and more used, for example, in nursing. the main reasons for the lesser use in the field of teaching have been lack of resources allocated to it and that it has not an official status in the collective agreement on terms of employment. however, teachers’ participation in supervision is currently voluntary, and therefore, teachers, who participate in it, seem to engage in the supervision process well. research on supervision is still relatively scarce and more information about its benefits and realization is needed in the field. this study focused on supervisors’ perceptions on how supervision supports inclusive teacherhood. they all had a teaching background. their experiences on how supervision supports the development of inclusive teachers were considered valuable and could provide important practical information. the multidimensional nature of supervision due to its varied theoretical background and practices, supervision cannot be distinctively defined (milne, 2007). the lack of a solid theory of supervision has actually led to the diversified practices of supervising education (cf., falender, burnes, & ellis, 2012). the concept of supervision resembles the guidance offered for novices teachers in their professional growth (hobson, ashby, malderez, & tomlinson, 2009; sundli, 2007). other close concepts are, among others, coaching (mclean & hudson, 2012) referring to a supervision relationship where the coach attempts to support the client’s work-related choices. in addition, process consultation and consulting (lambrechts, grieten, bouwen, & corthouts, 2009), and sparring resemble mentoring (alila, 2014). the most central methods of supervision are reflection and dialogue (löfmark, morberg, öhlund, & ilicki, 2009). solution-orientation is at the core, when supervision focuses on recognizing a supervisee’s strengths. the supervisee’s reflection and progress are supported by leaning of experiences of success and goal-setting (mccurdy, 2006; stark, frels, & garza, 2011). supervision can also include methods such as acting and drama (edwards, 2010), and other artistic techniques (denver & shiflett, 2011). in an efficient supervision, the supervisor pays attention to interaction and relationships between supervisees, and sticks to task-centered structure (ladany, mori & mehr, 2013). this way supervision can cover even more themes and offer more options to support change processes (hanna, 2011). a functional group supervision necessitates that the supervisor and supervisees are committed to communal learning (henderson, 2009). inclusion inclusion means basic values of students’ equal participation that direct education (hulgin & drake, 2011). equal opportunities within a heterogeneous group mean that all students get support in their physical, cognitive, emotional, and moral development within a safe, healthy, and intellectually developing environment (lakkala, uusiautti, & määttä, 2014). the increase in students’ participation is most of all resulting from teachers’ educational choices instead of legislation (shevlin, 2010). how does supervision support inclusive teacherhood? / alila, määttä & uusiautti 353 inclusive teaching requires a flexible curriculum noticing various learners, accessible school buildings and premises, segregated teaching and evaluation, and teaching staff who has proper education about inclusive practices (lakkala & määttä, 2011; symeonidou & phtiaka, 2009). teaching practices suitable for students who need special support have proven to be suitable for other students too (spaulding & flanagan, 2012). in all, the development of inclusion in education renews teaching and related values, beliefs, and attitudes (singal, 2008). students’ need for support is fulfilled by adjusting the learning environment accordingly. teachers cannot mold all factors in the learning environment, but they can influence attitudes, attention to segregated methods, and their awareness of students’ reactions (abbott, 2011). according to research, the best support to inclusive teacherhood is education about learning strategies, support of a multi-professional team, and assisting staff in classrooms (alila, 2014). the universal design for instruction (udi) is an approach to teach all students, and it includes the predictive planning and usage of inclusive teaching strategies (mcguire, scott, & shaw, 2006; samuels, 2007). the principles of universal design for learning (udl) provide means to do curriculum planning so that it meets every student’s various needs (meo, 2008). teaching arrangements in classrooms employing the udl model include collaboration, technological equipment, and segregated teaching (evans, williams, king, & metcalf, 2010). learning communities are developed through professional collaboration, reflection, and empowering methods. thus, students, parents, and communality form the core of school development (shepherd & hasazi, 2009). notwithstanding, the development of inclusive teaching means that special education teacher and classroom teacher education has to renew, too. collaboration between student teachers of general and special education should be supported already during their education (laarhoven, munk, lynch, bosma, & rouse, 2007) and prepare them to realize the broadness of a teacher’s work (florian, 2009). supervision in teachers’ work supervision in teaching also leans on collaboration, reflection, and dialogue (pattison, 2010) as it focuses on a more profound understanding of a teacher’s multidimensional role and on a stronger trust in one’s teacherhood (paliokosta & blandford, 2010). in teachers’ supervision, reflection means learning about the practical work. one can develop one’s professional skills and practical work through reflecting one’s experiences with other professionals in supervision (carroll, 2010; clouder & sellars, 2004). supervision provides a secured place to reflect one’s performance in one’s own work (hawkins & shohet, 2012). according to supervision studies, teacher have been able to change their beliefs and raise their awareness when they have been given the opportunity to reflect on issues related to teaching and learning (jordan, schwartz, & mcghie-richmond, 2009). this is especially important to the development of inclusive teacherhood, because the approach requires profound understanding about the nature of inclusion and teachers’ ability to reflect on their personal teaching styles, practices, and teacherhood. supervision is based on experiential learning (milne & james, 2002), constructivism (ibrahim, 2013), team learning (gillespie, 2012), and the principle of life-long learning. interaction with colleagues in the learning work community and the opportunity to see colleagues’ professional development strengthen teachers’ meaningful life-long learning (alila, 2014). international electronic journal of elementary education vol.8, issue 3, 351-362, 2016 354 while supervision is almost a self-evident part of many professionals’ work, such as psychologists and psychotherapists, it is strange that supervision has not become an established part of teachers’ work. and still teachers’ work has fundamentally changed during the shift toward inclusion (potmesilova, potmesil, & roubalova, 2013). supervision is an efficient support for the development of teachers’ expertise. it supports, predicts, guides, and renews teachers within the increasing demands of their work (goodman, brady, duffy, scott, & pollard, 2008; luke, ellis, & bernard, 2011). supervision is needed to help to confront challenges in teachers’ work, which have increased due to inclusion. method the purpose of this study is to describe how supervision supports inclusive teacherhood according to supervisors with a teaching background. the following research question was set for this study: according to supervisors with a teaching background, how does supervision support inclusive teacherhood and its development? the qualitative study approach was employed in this research. to answer the research question, the focus group interview method was chosen as the data collection method. the methodological approach represents fenomenography as it studied human beings’ different ways of experiencing, conceptualize, and understand the surrounding world and its phenomena (marton, 1988). in addition, the research is connected with the variation theory of fenomenography because the other purpose was to analyze the dimension of variation within the phenomenon under investigation and to compare various viewpoints about it (dahlin, 2007). focus group interviews were carried out in the spring of 2010. eleven supervisors with a teaching background and from five places in finland were recruited as the research participants. the criteria of selection were that they had been supervising at least for one year and that they were members of the finnish supervisors’ association. this association accepts only persons, who have completed a long-term supervising education, as its members. the research participants were women aged from 35 to 65 years. a focus group interview is a method for discussing a topic in a group, and the discussion is led by the interviewer (morgan, 2008). in this study, the focus group received nine questions to discuss. the purpose of the questions were to analyze together how mentoring supports inclusive teacherhood (e.g., how could supervision benefit teachers who want to employ inclusive teacherhood?; how could supervision support the development of inclusive teacherhood?) focus group interviews have increased interest in many fields of research as they provide a functional way of sharing understanding about themes that have been less studied previously (hesse-biber & leavy, 2011; stewart, shamdasani, & rook, 2007). when analyzing the data gathered by focus group interviews, themes that emerge in all or several groups and that are mentioned by many persons within one group are the most important ones (hesse-biber & leavy, 2011). in this study, the classic analyzing principles of focus group interviews were followed systematically (krueger & casey, 2009). the core of classic focus group interview analysis happens by cutting, categorization, and organization through comparisons and juxtapositions one interview question at a time (krueger & casey, 2009). each answer to a question is read and compared in order to find if the same theme or issue has been mentioned earlier. similar issues are thus combined together and together they form a category. the analyzing process was long and laborious, and during it, the categories were re-organized and even re-created several times. eventually, all answers found their places in categories that are here introduced as the themes within the main results. indeed, another purpose of the analysis was to find the how does supervision support inclusive teacherhood? / alila, määttä & uusiautti 355 so-called main thoughts in the data. it refers to a mutual understanding about a theme among the majority of research participants (vaughn, schumm & sinagub, 1996). in this study, the main results are organized into themes that represent the number and width of, or the mutual understanding among the supervisors about, these categories in the data. in the results section, themes in these two main results categories are introduced by including quotes from the focus group interviews. the codes consist of numbers and letters. the number after the letter k refers to the ordinal number of the quote in the data. the number placed after comma stands for the page the quote can be found in the data transcript. results the findings are organized according to two main results categories that are supervision as personal support and supervision as communal support. supervision as personal support supervision was mentioned to support inclusive teacherhood in six ways. the first was empowering support for teacherhood. supervision was described to enhance the coping of teachers who work in inclusive learning environments. teachers learned to make choices that helped them control their work loads. thus, the varied ways of benefitting from supervision provided an opportunity of empowering in a teacher’s work. this was described, for example, as follows: “...i think that as an approach it [inclusion] is really hard, so you need supervision to stay in control with the approach..” (k276,24) ”so i guess the purpose is… that also teachers are doing good and are able to enjoy their work.” (k274,24) the second theme was the support to the new kind of teacherhood. supervision provided teachers with opportunities to discover how their teacherhood matched the inclusive teaching approach. this helps them to develop their inclusive teacherhood. when it comes to a teacher’s profession, the change does not only cover their professional identity but their own personality too. supervision can pay attention to this part of the change considerably. “...you could say that it helps to find that other kind of teacherhood too… the so-called other side of yourself that you need when collaborating” (k666,63) “i wonder if it is the increase of self-knowledge; if you see the various sides of yourself, the features in which you are a little weaker or worse, and those that are your areas of expertise and strengths and everything in between.” (k742,70) thirdly, supervision supports teachers’ professional growth, as it helps recognizing the development. supervision can initiate, maintain, and help the development of professional skills and learning at work that inclusion demands. in addition, supervision can help teachers notice their needs for development when they work in an inclusive learning environment. “…the more the teacher works and the supervisor guides him or her…so the idea of inclusive teaching becomes fulfilled as there are always new groups that include [students] who act differently, think differently, and … the teacher always has to practice with every new group and increase his or her inclusive awareness.” (k750,71) “...you would help… sort of those people also in the work community to enter the limit of their zones of proximal development, like we try to help the students in the special education classrooms… in the same way, supervision helps you to find it.” (k770,75) international electronic journal of elementary education vol.8, issue 3, 351-362, 2016 356 supervision can help teachers to perceive their work role. this means that teachers can clarify and understand how their basic task as teachers changes when developing toward inclusive teacherhood. supervision provides an opportunity to perceive teacherhood as shared expertise instead of traditional view of teachers acting alone and by themselves. interviews showed how supervisors discussed the importance of pinpointing the core of teachers’ work and that development toward inclusive teacherhood is a process: “probably also that you know what is sufficient…” (k277,24) “to see your own incompleteness.” (k253,23) fifthly, supervision provides an opportunity to evaluate one’s work. teachers are able to view their work from a distance in supervision. the development toward inclusive teacherhood is supported by guiding the reflection and evaluation of one’s work. supervision teaches a more professional approach to teachers’ discussion and interaction skills, and little by little, this reflective perspective becomes one’s internalized part of developing in a teacher’s work. “so that you can have a little look at it with an outsider” (k228,21) “you can sort of compare various viewpoints” (k189,18) finally, supervision was also perceived to help with coping with various challenges and difficulties related to inclusion. one cannot particularly practice a teacher’s work in front of a class but supervision provides an opportunity to it. supervision helps coping with challenges at work, and the supervisor’s expertise and reliability are extremely crucial in this sense, as highlighted by the interviewees. “supervision can help you to internalize the inclusive approach so that first you are just with regular, nice-behaving schoolchildren and do just that teacher’s basic work, and then these challenging children and the encounters with them can first seem to be far away from your own comfort zone. but when you get more experienced things change, and your ability to work expands, and your comfort zone gets sort of bigger. so you can control your work more.” (k709,68) “and that the provoking situation happens in a supervision situation, it is a safer situation compared to [if you are alone] and then you can… as a teacher, avoid that confrontation inside your classroom and with that student or a group of students, when you catch the situation through supervision” (k753,71) supervision as communal support supervision was perceived to support inclusive teacherhood in a communal sense in five ways. first, supervision was mentioned to support versatile and multiprofessional cooperation. students and teachers in teacher training or continuing education do not learn about how to solve conflicts or get help in their work communally. therefore, supervision was considered a functional channel to practice collaboration and enhance one’s collaboration skills. “if you think that… you school is an inclusive school, you cannot just plan it by yourself, but you have to collaborate with others.” (k822,80) “and then also the collaboration relationships… other teachers and the work community, and parents” (k225,21) secondly, supervision supports the change of the whole school culture toward inclusion. supervision helps developing new practices by adjusting them to the prevailing school culture. in addition, supervision can serve as a tool to lead the change in a controlled manner. how does supervision support inclusive teacherhood? / alila, määttä & uusiautti 357 “exactly, when you aim toward it… it means that the school is in a state of change, and that supervision is about supporting the change and that you cope with the change… cope within the turmoil of change, go through with it, and understand that changes are needed if you are to reach that [inclusion]” (k665,63) “that is really important… to know what suit your own work community, that is really crucial… supervision has a great role in it, i suppose.” (k685,65) thirdly, supervision was mentioned to support teacher individuality within the work community. supervision lets the inclusive work community spring up through noticing its members’ individual capabilities. the importance of reflection as the method of supervision becomes emphasized in this form of support. supervision supports teachers’ individuality in the work community and helps to pay attention to each teacher’s uniqueness. “it points out it if you become heard in your supervision group in the first place, it is really important, and they are easily forgotten in the busy school life… so that you can discuss even a few thoughts and get the feeling that you are heard” (k688,66) “also in that sense, if you think that you could dare to be more genuine, be yourself” (k689,66) fourthly, supervision supports the development of communal work approach. supervision can support the development of inclusive teacherhood by supporting it as a communal process. communal work approach requires new kind of attitude to teacherhood, but the adaptation of the attitude can be fostered by supervision. the interviewees discussed how supervision can promote adults’ collaboration and communality: “and indeed, supervision is functional, when people who work in the same workplace, bring the well-being among those people forward in every way.” (k724,69) “and probably when this work community starts to develop, the one with the highest resistance has to develop to some direction although he or she did not participate in it at all.” (k736,70) finally, supervision is a support to the shared goal of inclusion at school. it takes the whole school community toward inclusive teacherhood and helps finding a shared understanding about inclusion. because, eventually, inclusion can only happen if the whole school community shares the idea about the objectives and meaning of inclusion. “and it will become a shared goal, because this is how you make the rules of the game, the mutual goal where you are heading…” (k206,19) “...one of the requirements of inclusion are that you can have the shared focus” (k499,46) conclusions as the results showed, supervision supports inclusive teacherhood individually and communally. in this study, the focus group interviews surfaced the number of ways supervision can and should support the development of inclusive teacherhood. however, it seems that the theoretical diversity of supervision hinders the support of inclusive teacherhood to some extent. according to this study, the core elements of supervision are goal-orientation, confidentiality, and the supervisor’s professionalism. at its best, supervision can form a learning environment of inclusive teacherhood, in which the stability, regularity, and continuity of supervision are the key. although supervision that aims at supporting inclusive teacherhood is quite manifold, the main points are to pay attention to the practical challenges introduced by student diversity and to reflect on the main issues with a wide perspective. the varied methods international electronic journal of elementary education vol.8, issue 3, 351-362, 2016 358 used in supervision can inspire inclusive practices in teaching, as they encourage teachers to use a wide range of teaching methods and to understand the importance of flexibility in teaching arrangement in practice. the process-like nature and sufficiently long period of supervision supports supervisee’s professional growth. it provides enough time to test and evaluate various solutions and one’s inclusive teacherhood. supervision as the means to support teachers’ professional development and learning helps teacher to discover their strengths and enhances the renewal of school culture. the finnish supervision strongly relies on ethical principles according to which supervision must be highly confidential (alila, 2014). this kind of confidentiality provides a secure and goal-oriented space for teachers to develop professionally toward inclusive teacherhood. the reliability of this study can be evaluated from many points of view. the head researcher of this study is a supervisor with teaching background herself, and therefore, her position is not totally objective. obviously, she has a positive stand and experience about supervision and its significance to the development of inclusive teacherhood. this might have influenced on the way the results are interpreted. however, the personal viewpoint to and experience of the research theme also means immediate experience and profound understanding about the phenomenon. to improve the reliability of analysis, the findings and conclusions were discussed with the research group. in addition, the purpose was to bring out the supervisors’ voices when reporting about the findings. this was also to improve reliability as the excerpts from data illustrate how the themes were brought out and discussed by the research participants. as the number of the research participants was small, the study could even be defined as a mini focus group research (krueger & casey, 2009). however, the size of the group is also justifiable because these people present a marginal group based on their expertise and supervision experience in finland. the interviewees also showed a positive attitude to supervision, and its possibilities to support the development of inclusive teacherhood were, thus, seen very high. their versatile experiences provided a profound discussion about the support supervision can provide to the adaptation of inclusion in schools. the way the supervisees had experienced supervision was not studied in this research, but it would make a good add and presents a need for further research. discussion the findings of this study are in line with other researchers’ findings. for example, hobson, ashby, malderez, and tomlinson (2009; see also sundli, 2007) have confirmed that supervision can promote the renewal of teaching practices. supervision is a method of supporting professional development toward participatory school for all students (potmesilova, potmesil, & roubalova, 2013). in all, supervision is a way to view one’s professional development through a confidential relationship. “confidentiality in coaching and supervision is key”, have noticed also connor and pokora (2012, p. 3). when the supervisor has a special training for this task, it is possible to offer quality supervision. supervision can become a learning environment in which teachers can learn and expand their skills and knowledge toward inclusive teacherhood and in which their values and attitudes to inclusion mold with their professional growth. supervision can also help teachers to cope with the challenges of their work and to develop, learn, and dedicate to it, which was also shown by hawkins and shohet’s (2012) study. fundamentally, supervision as a manifold method helps supervisees to acknowledge their own expertise (see also shachar et al., 2012). the development of inclusive teacherhood necessitates also the recognition of one’s strengths and weaknesses, needs for development. teachers are also expected to react to others’ expectations. supervision how does supervision support inclusive teacherhood? / alila, määttä & uusiautti 359 can support their professional growth so that teachers can feel capable to renew not only themselves as teachers but also their work practices as inclusion necessitates with and in the multiprofessional work community. inclusive education calls for a change in school practices (bourke, 2009), and, therefore, teachers must feel supported to develop toward inclusive teacherhood (ainscow, 2008). although supervision also requires financial investment, its profits can be seen in the future. indeed, for example zepada (2011) points out that the development of inclusive school challenges the school administrators to recognize, understand, and promote features that enhance the success of all students at school. the beneficial outcomes of supervision include better work satisfaction and attendance, renewed and more reasonable teaching arrangements, and overall improvement in work quality. as more time and opportunities for supervision are arranged for teachers, it can become a natural method in the field of education, as well. however, also supervision has to develop in order to meet the needs of changing work (see also connor & pokora, 2012; de janasz & sullivan, 2004). in school work, these changes include, among others, new curricula, the development of learning strategies and equipment, and the diversifying student population with various skills, attitudes, values, and cultures. from this perspective, the framework and multi-method nature of supervision appear a downright advantage and opportunity. • • • references abbott, c. (2011). aiming for inclusion: removing barriers and building bridges. in j. dillon & m. maguire (eds.), becoming a teacher. issues in secondary education (pp. 236-248). maidenhead: open university press. ainscow, m. (2008). teaching for diversity: the next big challenge. in f. connelly, m. he, & j. phillion, j. (eds.), the sage handbook of curriculum and instruction (pp. 240-259). los angeles, ca: sage. doi: 10.4135/9781412976572.n12 ainscow, m. (2005). developing inclusive education systems: what are the levers for change? journal of educational change, 6(2), 109–124. doi: 10.1007/s10833-005-1298-4. alila, s. (2014). ”työnohjaus auttaa löytämään omia vahvuuksia ja.... toimintakulttuurin luomisessa” työnohjaus inklusiivisen opettajuuden tukena [“supervision helps you to find our strengths and… to create the work culture.” supervision as the support for inclusive teacherhood]. (phd diss., university of lapland, rovaniemi, finland.) bourke, p. (2009). professional development and teacher aides in inclusive education contexts: where to from here? international journal of inclusive education, 13(8), 817-827. doi: 10.1080/13603110802128588 brunero, s., & stein-parbury, j. (2008). the effectiveness of clinical supervision in nursing: an evidenced based literature review. australian journal of advanced nursing, 25(3), 86-94. carroll, m. (2010). supervision: critical reflection for transformational learning (part 2). the clinical supervisor, 29(1), 1-19. doi: 10.1080/07325220903344015 clouder, l., & sellars, j. (2004). reflective practice and clinical supervision: an interprofessional perspective. journal of advanced nursing, 46(3), 262-269. doi: 10.1111/j.13652648.2004.02986.x international electronic journal of elementary education vol.8, issue 3, 351-362, 2016 360 connor, m., & pokora, j. (2012). coaching and mentoring at work. developing effective practice. (2nd ed.) maidenhead: open university press. dahlin, b. (2007). enriching the theoretical horizons of phenomenography, variation theory and learning studies. scandinavian journal of educational research, 51(4), 327-346. doi: 10.1080/00313830701485437 de janasz,s. d., & sullivan, s. e. (2004). multiple mentoring in academe: developing the professorial network. journal of vocational behavior, 64, 263–283. doi: 10.1016/j.jvb.2002.07.001 denver, s., & shiflett, c. (2011). art-based supervision techniques. the clinical supervisor, 30(2), 257-276. edwards, d. (2010). play and metaphor in clinical supervision: keeping creativity alive. the arts in psychotherapy, 37(3), 248-254. doi:10.1016/j.aip.2010.04.011 evans, c., williams, j., king, l., & metcalf, d. (2010). modeling, guided instruction, and application of udl in a rural special education teacher preparation program. rural special education quarterly, 29(4), 41-48. falender, c., burnes, t., & ellis, m. (2012). multicultural clinical supervision and benchmarks: empirical support informing practise and supervisor training. the counseling psychologist, 41(1), 8-27. doi: 10.1177/0011000012438417 florian, l. (2009). preparing teachers to work in ‘school for all’. teaching and teacher education, 25(4), 533-534. doi:10.1016/j.tate.2009.02.003 gillespie, j. (2012). teaching note: enhancing social work education through team-based learning. journal of social work education, 48(2), 377-387. doi:10.5175/jswe.2012.200900014 goodman, i., brady, m., duffy, m., scott, j., & pollard, n. (2008). the effects of “bug-in-ear” supervision on special education teachers’ delivery of learn units. focus on autism and other developmental disabilities, 23(4), 207-216. doi: 10.1177/1088357608324713 hanna, f. (2011). freedom: toward an integration of the counseling profession. counselor education & supervision, 50(6), 362-385. doi: 10.1002/j.1556-6978.2011.tb01921.x hawkins, p., & shohet, r. (2012). supervision in the helping professions. (4th ed.) maidenhead: open university press. henderson, p. (2009). supervision training: issues and approaches. volume 2. london: karnac books. hesse-biber, s., & leavy, p. (2011). the practise of qualitative research. (2nd ed.) thousand oaks, ca: sage. hobson, a., ashby, p., malderez, a., & tomlinson, p. (2009). mentoring beginning teachers: what we know and what we don’t. teaching and teacher education, 25(1), 207-216. doi:10.1016/j.tate.2008.09.001 hulgin, k. & drake, b. (2011). inclusive education and the no child left behind act: resisting entrenchment. international journal of inclusive education, 15(4), 389-404. doi: 10.1080/13603110903030105 ibrahim, a. (2013). approaches to supervision of student teachers in one uae teacher education program. teaching and teacher education, 34, 38-45. doi: 10.1016/j.tate.2013.04.002 jordan, a., schwartz, e., & mcghie-richmond, d. (2009). preparing teachers for inclusive classrooms. teaching and teacher education, 25(4), 535-542. doi:10.1016/j.tate.2009.02.010 krueger, r., & casey, m. (2009). focus groups. a practical guide for applied research. (4th ed.) thousand oaks, ca: sage. laarhoven, t., munk, d., lynch, k., bosma, j., & rouse, j. (2007). a model for preparing special and general education preservice teachers for inclusive education. journal of teacher education, 58(5), 440-455. doi: 10.1177/0022487107306803 how does supervision support inclusive teacherhood? / alila, määttä & uusiautti 361 ladany, n., mori, y., & mehr, k. (2013). effective and ineffective supervision. the counseling psychologist, 41(1), 28-47. doi: 10.1177/0011000012442648 lakkala, s. & määttä, k. (2011). towards a theoretical model of inclusive teaching strategies an action research in an inclusive elementary class. global journal of human social science, 11(8), 31-40. lakkala, s., uusiautti, s., & määttä, k. (2014). how to make the neigbourhood school a school for all? finnish teachers´ perceptions of educational reform aiming toward inclusion. journal of research in special educational needs, ifirst. doi: 10.1111/1471-3802.12055 lambrechts, f., grieten, s., bouwen, r., & corthouts, f. (2009). process consultation revisited. taking a relational practice perspective. journal of applied behavioral science, 45(19), 39-58. doi: 10.1177/0021886308326563 luke, m., ellis, m., & bernard, j. (2011). school counselor supervisors’ perceptions of the discrimination model of supervision. counselor education & supervision, 50(5), 328-343. doi: 10.1002/j.1556-6978.2011.tb01919.x löfmark, a., morberg, å., öhlund, l., & ilicki, j. (2009). supervising mentors´ lived esperience on supervision in teach, nursing, and social care education. a participation-oriented phenomenological study. higher education, 57(1), 107-123. doi:10.1007/s10734-008-9135-3 marton, f. (1988). phenomenography. a research approach to investigating different understandings of reality. in r. sherman & r. webb (eds.), qualitative research in education. focus and methods (pp. 141-161). london: falmer-press. mccurdy, k. (2006). adlerian supervision: a new perspective with a solution focus. journal of individual psychology, 62(2), 141-153. mcguire, j., scott, s., & shaw, s. (2006). universal design and its applications in educational environments. remedial and special education, 27(3), 166-175. doi: 10.1177/07419325060270030501 mclean, p., & hudson, f. (2012). completely revised handbook of coaching: a developmental approach. (2nd ed.) somerset: wiley. meo, g. (2008). curriculum planning for all learners: applying universal design for learning (udl) to a high school reading comprehension program. preventing school failure, 52(2), 21-30. doi: 10.3200/psfl.52.2.21-30 milne, d. (2007). an empirical definition of clinical supervision. british journal of clinical psychology, 46(4), 437-447. doi: 10.1348/014466507x197415 milne, d., & james, i. (2002). the observed impact of training on competence in clinical supervision. british journal of clinical psychology, 41(1), 55-72. doi: 10.1348/014466502163796 morgan, d. (2008). the focus groups. in l. given (ed.), the sage encyclopedia of qualitative research methods (pp. 353-355). thousand oaks, ca: sage. paliokosta, p., & blandford, s. (2010). inclusion in school: a policy, ideology or lived experience? similar findings in diverse school cultures. support for learning, 25(4), 179-186. doi: 10.1111/j.1467-9604.2010.01464.x pattison, s. (2010). reaching out: a proactive process to include young people with learning disabilities in counselling in secondary schools in the uk. british journal of guidance & counselling, 38(3), 301-311. doi:10.1080/03069885.2010.491849 potmesilova, p., potmesil, m., & roubalova, m. f. (2013). supervision as a prevention and support teachers in inclusive education. electronic journal for inclusive education, 2(11), art 3, 1-12. samuels, c. (2007). ‘universal design’ concept pushed for education. education week, 27(10), 1-12. shachar, r., nasim, r., leshem, t., rosenberg, j., schmidt, a., & schmuely, v. (2012). power hierarchy, multiple truth and innovations in narrative supervision. journal of systematic therapies, 31(4), 34-48. doi: 10.1521/jsyt.2012.31.4.34 international electronic journal of elementary education vol.8, issue 3, 351-362, 2016 362 shepherd, k., & hasazi, s. (2009). leadership for social justice and inclusion. in i. florian (ed.), the sage handbook of special education (pp. 476-487). london: sage. shevlin, m. (2010). count me in! ideas for actively engaging students in inclusive classrooms. london: jessica kingsley publishers. singal, n. (2008). working towards inclusion: reflections from the classroom. teaching and teacher education, 24(6), 1516-1529. doi:10.1016/j.tate.2008.01.008 spaulding, i., & flanagan, j. (2012). dis2ect. a framework for effective inclusive science instruction. teaching exceptional children, 44(6), 6-14. stark, m., frels, r., & garza, y. (2011). the use of sand-tray in solution-focused supervision. the clinical supervisor, 30(2), 277-290. stewart, d., shamdasani, p., & rook, d. (2007). focus groups. theory and practice. thousand oaks, ca: sage. sundli, l. (2007). mentoring a new mantra for education? teaching and teacher education, 23(2), 201-214. doi:10.1016/j.tate.2006.04.016 symeonidou, s., & phtiaka, h. (2009). using teachers’ prior knowledge, attitudes and beliefs to develop in-service teacher education courses for inclusion. teaching and teacher education, 25(4), 543-550. doi:10.1016/j.tate.2009.02.001 vaughn, s., schuman, j. s., & sinagub, j. (1996). focus group interviews in education and psychology. thousand oaks, ca: sage. zepeda, s. (2011). instructional supervision, coherence and job-embedded learning. in t. townsend & j. macbeath (eds.), international handbook of leadership for learning (pp. 741-756). dordrecht: springer netherlands. doi:10.1007/978-94-007-1350-5_41 students’ performance calibration in a basketball dibbling task in elementary physical education international electronic journal of elementary education, 2013, 5(2), 157-170. issn:1307-9298 copyright © iejee www.iejee.com vocabulary and syntactic knowledge factors in 5th grade students’ reading comprehension kouider mokhtari  the university of texas at tyler, usa dale s. niederhauser iowa state university, usa received: 2 september 2012 / revised: 5 november 2012 / accepted: 11 november 2012 abstract in this study, we examined 5th grade students’ levels of vocabulary knowledge and syntactic awareness relative to their reading comprehension performance. the aim was to explore the contributions of vocabulary and syntactic awareness as potential sources of reading comprehension difficulty for these readers. overall, we found that both vocabulary knowledge and syntactic awareness contributed in significant yet unique ways to students’ reading comprehension performance. students who showed weaknesses in vocabulary and syntax also performed poorly on measures of reading comprehension. additionally, we found that syntactic awareness explained a small amount of additional variance in reading comprehension beyond what was explained by vocabulary. the implications of these findings are discussed in light of research and practice addressing the relationships among syntax, vocabulary, and reading comprehension for more and less skilled readers. keywords: vocabulary development, syntactic awareness, reading comprehension, adolescent readers introduction a fairly consistent finding in reading research over the past seven decades is that vocabulary knowledge contributes significantly to students’ reading comprehension performance (e.g., baumann, kame‘enui, & ash, 2003; davis, 1942, 1944; ruddell, 1994; thorndike, 1973; whipple, 1925). this research has shown that vocabulary knowledge and reading comprehension are highly correlated, making vocabulary a leading predictor of reading comprehension among children and young adults. the unique contribution of vocabulary knowledge to reading comprehension has been consistently established in a variety of factor analytic, correlational, and experimental studies as one of the best predictors of reading comprehension among school-aged children at the  kouider mokhtari, school of education, 3900 university blvd., the university of texas at tyler, tyler tx 75799, usa, phone: 903.566.7177. e-mail: kmokhtari@uttyler.edu http://www.iejee.com/ international electronic journal of elementary education vol.5, issue 2, 157-170, 2013 158 elementary, middle school, high school, and college levels. for instance, in a factor analytic study aimed at determining the basic skills underlying reading comprehension among a large group of college freshmen, davis (1944) found that word knowledge had a factor loading of .80. other studies (e.g., bozo, 1951; clark, 1972; davis, 1968) resulted in vocabulary factor loadings ranging from .41 to .93. these findings are consistent with other research studies showing that vocabulary knowledge strongly predicts reading comprehension among children in the early years of reading development (e.g., de jong & van der leij, 2003; thorndike, 1973; torgesen, wagner, rashotte, burgess, & hecht, 1997). for example, in one study, thorndike (1973) reported correlations ranging from .66 to .75 between vocabulary knowledge and reading comprehension. in a longitudinal study commissioned by the nichd early child care research network (2005), the authors reported a correlation of .56 between students’ grade 1 picture vocabulary and grade 3 reading comprehension performance. various other studies have reported correlations between vocabulary knowledge and reading comprehension of around .50 (e.g., dixon, lefevre & twilley, 1988; hunt, 1953; stanovich & cunningham, 1992). as can be seen in the above studies, the research literature is clear: once children have developed basic decoding skills, the number of words they can read and understand begins to positively influence their ability to comprehend what they read (e.g., beck, perfetti, & mckeown, 1982; chall, jacobs, & baldwin, 1990). these findings have been confirmed in recent large-scale studies. results from the 2009 and 2011 national assessment of educational progress (naep) indicate that students who scored higher on naep vocabulary questions also scored higher in reading comprehension (national center for education statistics [nces], 2012). the contribution of vocabulary knowledge to reading comprehension has also been established in studies with students who are non-native speakers of english. for instance, proctor, carlo, august, and snow (2005) tested a second-language reading comprehension model incorporating decoding and oral language measures on a sample of 135 spanishenglish speaking 4th graders and reported a correlation of .73 between students’ vocabulary knowledge and reading comprehension outcomes. in another study, gelderen et al., (2004) administered tests of english vocabulary knowledge and reading comprehension to 397 dutch students in grades 8 through 10 and found a correlation of .63. these studies provide evidence that vocabulary is critically important in first language as well as in second language reading. however, findings from syntheses of reading research and practice (e.g., cain, 2010; cain & oakhill, 2007; national institute of child health and human development, 2000; snow, 2002; snow, burns, & griffin, 1998) indicate that while learning to read and understand word meanings is necessary, it is not consistently sufficient to ensure that readers effectively comprehend written texts. although text comprehension is the ultimate goal of the act of reading, assessment reports often indicate that a significant number of children who have been identified as having successfully learned to read in grades one through three often face difficulties reading texts with understanding in grade four and beyond (e.g., gregg, donahue, & dion, 2007; nces, 2012). the gap between students’ understanding of word meanings and poor reading comprehension is well documented. during the past several years, teams of prominent researchers (e.g., cain, 2010; cain & oakhill, 2007; clarke, snowling, truelove, hulme, 2010; nation, cocksey, taylor & bishop, 2010) have examined and written extensively about the correlates, causes, and consequences of children’s reading comprehension difficulties. a particularly important finding is that poor comprehenders can read words and sentences at 5th grade students’ reading comprehension / mokhtari & niederhauser 159 age-appropriate levels fluently, but have difficulties comprehending texts, even when they are matched on vocabulary knowledge and word reading, and compared to peer controls. in one longitudinal study of early reading and language skills in children with poor reading comprehension, nation et al. (2010) reported that while these children (ages 5 through 8 years) showed normal reading accuracy and fluency at all ages, they exhibited “mild impairments in expressive and receptive language, listening comprehension, and grammatical understanding” (p. 1031). consistent with these findings, clarke et al. (2010) found that weaknesses in understanding and using spoken language play a causal role in the reading comprehension difficulties faced by poor comprehenders. they reported finding that a language-based intervention program designed to enhance the oral language skills of 8-year-old children with poor comprehension skills resulted in significant improvements in both reading comprehension and expressive vocabulary. these results seem to indicate that effective reading comprehension requires both a fundamental knowledge of the meaning of words and an understanding of the internal structures of words and sentences. in addition, a relatively small but growing body of research indicates that in addition to vocabulary knowledge, shortcomings in children’s knowledge or awareness of certain aspects of language development (e.g., syntax) are considered significant sources of reading and writing difficulty and are presumed to account for a significant portion of variance in children’s reading comprehension performance. recent advances in language and literacy research suggest that while vocabulary knowledge deficits are at the core of reading problems among upper grade students, deficits in syntactic awareness skills, here defined as children’s understanding of the syntactic structure of sentences and their ability to reflect on and manipulate that structure, are likely to play a larger role in explaining reading comprehension variance among developing and striving readers than originally assumed (e.g., cain, 2010, 2007; demont & gombert, 1996; scott, 2004; tunmer & bowey, 1984). however, it is important to note that research findings exploring the relationships between children’s syntactic abilities and reading comprehension performance have been mixed. for instance, in one study, bowey and patel (1988) assessed the syntactic awareness of 6-yearolds using a sentence correction task. they reported that performance on this measure did not result in any significant variance in either reading comprehension or word decoding after individual differences in vocabulary had been accounted for. in another study, layton, robinson, and lawson (1998) examined the effects of syntactic awareness training on 4thgrade students’ reading comprehension and accuracy and oral reading ability. their findings indicate that while students made improvements in syntactic awareness skills, no evidence was found for a systematic effect of improved syntactic awareness on reading ability. in a related study, willows and ryan (1986) assessed sixto eight-year-old children’s syntactic awareness using an oral cloze task and found that this measure was significantly associated with reading comprehension after controlling for vocabulary and iq. however, it is difficult to interpret the nature of this association because semantic knowledge may have influenced children’s performance on the cloze task, which requires decisions based on the semantic as well as the syntactic structure of words. in some longitudinal studies, researchers found only weak associations between syntactic awareness and reading comprehension. for instance, demont and gombert (1996) documented children’s progress from preschool to second grade using measures of phonological awareness, syntactic awareness, word reading rate and accuracy, and sentence comprehension, after taking into account verbal and general ability. they reported more impressive results for the relationship between phonological skills and decoding than international electronic journal of elementary education vol.5, issue 2, 157-170, 2013 160 between syntactic awareness skills and comprehension. in a similar longitudinal study, blackmore and pratt (1997) found only weak relations between preschool grammatical awareness and later reading comprehension. oakhill and cain (2007) maintain there might be developmental differences in the influence of syntactic awareness and reading comprehension. in a longitudinal study, oakhill, cain, and bryant (2003) found that while syntactic ability did not predict reading comprehension ability (or word reading accuracy) among seven and eight year-olds after controlling for vocabulary and iq, it explained significant variance in reading comprehension, but not word accuracy in the same sample of children a year later. the above findings indicate that the role of syntactic knowledge or awareness and skills in text understanding among children and adolescent readers may be more important than previously assumed. in fact, several published studies reported that syntactic awareness predicts later comprehension of longer written texts (e.g., muter, hulme, snowling, & stevenson, 2004). syntactic awareness has been shown to aid reading comprehension in direct ways such as when attempting to comprehend words within individual sentences and integrating the meaning of multiple sentences in larger units of texts, as well as in indirect ways such as when making grammaticality judgments or correcting grammatical errors in sentences (e.g., tunmer & bowey, 1984). research has shown that children’s ability to detect and correct syntactic errors is directly related to their ability to comprehend textual materials (e.g., bentin, deutsch, & liberman, 1990; demont & gombert, 1996). poor readers have been shown to differ on a number of syntactic processing tasks (e.g., sentence correction, grammaticality judgments) and these differences, according to some researchers (e.g., leikin, 2002; tunmer & bowey, 1984; tunmer & hoover, 1992), suggest the existence of a syntactic processing deficit among some struggling readers. syntactic awareness has been shown to help readers interact successfully with text, in that it gives them guidance in how to parse text into meaningful “chunks” or units. kintsch (1998) argues that grammar or syntax is an essential feature of language and goes as far as suggesting that syntactic knowledge can be even more important than topic or domain knowledge. in other words, “if a text is wellwritten, syntactic cues can tell the reader what is important in the text, even in the absence of specific domain knowledge” (kintsch, 1998, p. 134), thus allowing the reader to construct an adequate interpretation of text. in summary, our review of research indicates that the nature of relationship between vocabulary knowledge, syntactic awareness, and reading comprehension is to some extent ambiguous. there is strong support for the fact that vocabulary knowledge predicts later comprehension performance and that that relationship may be reciprocal. on the other hand, the role of syntactic awareness skills is less clear-cut, although available research seems to suggest that vocabulary knowledge is more highly predictive of reading comprehension than is syntactic awareness. the present study explores these relationships among upper elementary grade students. aims of the study in this study, we examined 5th-grade students’ levels of vocabulary knowledge and syntactic awareness in relation to their reading comprehension performance. the aim was to explore the contributions of vocabulary knowledge and syntactic awareness as potential sources of reading comprehension difficulty for these readers. specifically, we were interested in finding out whether students’ levels of vocabulary knowledge and syntactic awareness were significantly associated with their ability to read with comprehension. in light of findings from prior research addressing vocabulary and syntactic awareness as potential sources of comprehension difficulty, we predicted that lower levels of vocabulary knowledge would 5th grade students’ reading comprehension / mokhtari & niederhauser 161 contribute significantly to explaining variance in low reading comprehension performance among our study participants. additionally, we predicted that syntactic awareness would explain additional variance in students’ reading comprehension above and beyond that explained by vocabulary knowledge. specific research questions included the following: 1. what are the relative contributions of vocabulary knowledge and syntactic awareness in explaining variance in 5th grade students’ reading comprehension performance? 2. does syntactic awareness explain additional variance in reading comprehension performance beyond what is explained by vocabulary knowledge? method participants the participants consisted of 32 fifth-grade students (17 males; 15 females) enrolled in two classrooms in a small neighborhood school in the south central united states. of the 32 students (mean age = 11.32; sd = .642), twenty (63%) were caucasian, two (6%) were african american, three (9%) were hispanic american, and seven (22%) american indian. school records indicated that seven of the 32 students received daily supplemental assistance in reading, three qualified for and received special education services, and one was enrolled in the school’s gifted and talented program. none of the students received assistance in the areas of speech and language development. we selected students in upper-elementary grades (i.e. 5th grade) for two main reasons. first, during this period of development, vocabulary knowledge increases exponentially along with growth in other language skills such as morphology and syntax (e.g., nagy & herman, 1987; nippold, 1998, 2004). as students progress through the grades, they become increasingly capable of analyzing the structure of derived words and compounds, and inferring the meaning of words and sentences from word parts. second, as students transition from earlyto upper-elementary grades, they become progressively more immersed in reading, writing, and thinking about language because linguistically complex words and sentences are sufficiently common in children’s texts to make it likely that the processing of these language skills plays an increasingly important role in reading (carlisle, 2004; scott, 2004). data collection we administered two sets of tests to measure students’ reading vocabulary, reading comprehension, and syntactic awareness. a brief description of each of these tests follows. the gates-macginitie reading tests (macginitie, macginitie, maria, dreyer, & hughes, 2000). the gates-macginitie reading tests are standardized norm-referenced reading tests designed to provide a general assessment of reading achievement. the tests consist of two subtests: reading comprehension and vocabulary. the comprehension subtest measures students’ ability to read and understand passages of published works including narrative, expository, literacy non-fiction, and poetry text selections. the content of the selections is esoteric, which reduces the likelihood that a student would have knowledge of the various subjects addressed. most of the questions (42 out of 48) are phrased differently from the text and require inferential processing. within the gates-macginitie test, vocabulary is considered a separate construct, which is designed to measure students’ reading vocabulary skills. this subtest consists of 45 questions that assess a range of grade-appropriate words by requiring students to identify a synonym for a word provided in a sentence or short phrase. some words assessed have double international electronic journal of elementary education vol.5, issue 2, 157-170, 2013 162 meanings, and require students to identify the intended meaning from context. questions consist of either short sentences or sentence stems with the vocabulary word underlined. some of the questions are based on understanding words or phrases such as nearly, seldom, or straight away. the number and type of questions cover an array of word meanings to make the scores reliable for individual students as well as groups of students. the gates-macginitie reading tests take a total of 55 minutes to complete with 35 minutes devoted to reading comprehension and 20 minutes for vocabulary. the two subtests have excellent psychometric properties as indicated by reliability coefficients ranging from .90-.95 for reading vocabulary and .88-.94 for reading comprehension. we used normal curve equivalent (nce) scores as a unit of analysis for both the comprehension and vocabulary subtests. nce scores are normalized standard scores that range in value from 1 to 99, with a mean of 50 and a standard deviation of 21.06. test of language developmentintermediate [told-i: 3] (hammill & newcomer, 1988). the told-i:3 test battery consists of six individually-administered norm-referenced subtests that are designed to assess the language skills (spoken language, semantics, syntax, listening, and speaking) of children ages 8 through 12. collectively, the tests are designed to measure a child's ability to combine sentences, understand word meanings, produce appropriate sentence structures, make generalizations, and locate incorrect grammatical structures. overall test performance is indicated by the spoken language quotient, which reflects a composite of the child's listening, speaking, vocabulary, and grammar skills. for purposes of this study, we used students’ performance on three of the six subtests in the test battery (sentence combining, word ordering, and grammatical competence) to assess syntactic awareness. the result of the three subtests yielded a syntax quotient (synq), which we used to determine participants’ syntactic knowledge or awareness. we used normalized standard scores, which are based on a distribution with a mean of 100 and a standard deviation of 15. reviews indicate that the told-i: 3 is considered to be a valid and reliable test of language ability assessment. coefficients for all subtests exceed .84, and all composites are .90 or greater. data analyses we used multiple regression analysis to determine whether we could make a prediction about 5th grade students’ reading comprehension based on information obtained from the two predictor variables, namely vocabulary knowledge and syntactic awareness. for purposes of this study, we conducted a power analysis using the g*power3 software (faul, erdfelder, lang, & buchner, 2007) to determine how many participants would be needed to achieve 85% power for detecting a statistically significant regression model. the results indicated that with two predictors in our study, and an alpha (α) level for testing the regression model set at .05, we needed 30 participants to achieve power of .80. the number of participants needed to achieve higher levels of power for detecting a statistically significant regression model increased from 33 participants for 85% power to 44 participants for 95% power. our sample size of 32 was therefore sufficient to conduct our analyses assuming an 80-85% power and an effect size of .15. 5th grade students’ reading comprehension / mokhtari & niederhauser 163 results table 1 provides a summary of the results obtained using a format for presenting information proposed by tabachnick and fidell (2013). initial examination of our reading comprehension data for a sample of thirty-two 5th-grade students indicates that there were positive correlations among all three pairs of variables, namely reading comprehension, vocabulary knowledge, and syntactic awareness. it is important to note, however, that the correlation between the predictor variables of vocabulary and syntax (r = .733) shows an acceptable level of multicollinearity, indicating that these two variables are less likely to contribute too much superfluous or redundant information. table 1. results of standard multiple regression to predict reading comprehension from vocabulary and syntactic awareness variables readcomp vocabulary syntax b β sr2 unique vocabulary .768*** .362** .415 .283 syntax .786*** .733*** 627*** .481 .327 intercept = -27.56 means 50.78 53.50 94.06 sd 16.05 18.41 12.32 r2 = .676; r2adj = .676; r = .835*** n=32; ** p < .01; *** p < .001. with respect to the overall multiple regression to predict 5th-grade students’ reading comprehension based on information obtained from their vocabulary knowledge and syntactic awareness levels, we found that when both vocabulary and syntax were used as predictors, about 68% of the variance in reading comprehension could be predicted. the overall regression was statistically significant, f(2, 29) = 33.38, p <.001. vocabulary knowledge was predictive of reading comprehension when syntactic awareness was statistically controlled, t(29)=2.77, p=.010. the positive slope of vocabulary knowledge as a predictor of reading comprehension was about a .36 unit increase in reading comprehension for each unit increase in vocabulary, controlling for syntactic awareness. the squared semi-partial correlation that estimated how much variance in reading comprehension was uniquely predicted from vocabulary knowledge was .283. in other words, about 28% of the variance in students’ reading comprehension was uniquely predictable from their vocabulary knowledge when controlling for their knowledge of syntax. the data provided in table 1 indicates that syntactic awareness was also predictive of reading comprehension when vocabulary knowledge was statistically controlled, t(29)=3.20, p=.003. the slope to predict reading comprehension from syntactic awareness was .627. in other words, there was slightly more than a half point increase in reading comprehension for each 1-point increase in syntactic awareness. the semi-partial correlation for syntactic awareness (sr2=.327) indicates that students’ levels of syntactic awareness uniquely predicted about 33% of the variance in their reading comprehension when we controlled for their vocabulary knowledge. overall, the above findings indicate that the original zero-order correlation between vocabulary knowledge and reading comprehension (r=.768) was partly but not entirely accounted for by syntactic awareness. when syntactic awareness was statistically controlled, vocabulary knowledge still uniquely predicted 28% of the variance in reading comprehension. similarly, the correlation between syntactic awareness and reading comprehension (r=.786) was partly but not entirely accounted for by vocabulary knowledge. when vocabulary international electronic journal of elementary education vol.5, issue 2, 157-170, 2013 164 knowledge was statistically controlled, syntactic awareness uniquely predicted 33% of the variance in reading comprehension. discussion there are at least two possible interpretations for these findings. first, the regression analysis indicates that 68% of the variance in reading comprehension was predictable from vocabulary knowledge and syntactic awareness in combination (i.e., r2=.676). second, the results show that 28% and 33% of the variance in reading comprehension was uniquely predicted from vocabulary knowledge and syntactic awareness, respectively; thus the remaining variance could be predicted equally well by reading vocabulary or syntactic awareness since these two predictors could be confounded or redundant to some extent. to the extent that vocabulary knowledge and syntactic awareness are positively correlated with each other, they appear to compete in explaining some of the variance in reading comprehension. it is evident in our findings that both vocabulary knowledge and syntactic awareness contribute uniquely useful predictive information about reading comprehension. the finding that syntactic awareness explained additional variance in reading comprehension challenges the widespread assumption in the literature that vocabulary knowledge serves as the leading predictor of reading comprehension. it is indeed a challenge to various correlational and experimental research studies, in which syntactic awareness was either not associated with reading comprehension performance (e.g., layton et al., 1998) or found to play a lesser role than vocabulary knowledge (e.g., demont & gombert, 1996, blackmore & pratt, 1997). on the other hand, the finding appears to be consistent with some longitudinal research studies reporting that syntactic awareness is an important predictor of children’s reading comprehension ability (e.g., muter et al., 2004; oakhill et al., 2003). overall, these results suggest that syntactic awareness plays at least an equally important role in students’ reading comprehension than previously assumed, and that our understanding of students’ reading comprehension performance can be enhanced by examining their levels of syntactic awareness as well. indeed, it makes sense to argue for a compensatory processing conceptualization of the contribution of these two variables to reading comprehension. such a conceptualization recognizes the fact that both knowledge sources act in a collaborative, synergistic manner rather than an additive one. implications for research and practice the findings of this study have important implications for literacy research and practice. these findings indicate that reading requires fundamental knowledge of the meaning of words and fundamental knowledge of the internal structure of sentences. it is important to note, however, that reading also requires fundamental knowledge of other variables (e.g., reader, text, and context variables) that have not been accounted for in this study. with respect to broader research implications, it is important to recognize that while the research insights gained over the past seven decades have helped advance our understanding of how students’ reading vocabulary and syntactic awareness contribute to their understanding of what they read, future research has yet to account for the impact of reading vocabulary and syntactic awareness against the backdrop of other variables that are likely to have an impact on reading comprehension. first, the study of metalinguistic variables such as phonology, morphology, and syntax are critically important areas of research that are well beyond the realm of reading and writing. consequently, literacy researchers should consider teaming up with experts in the domains of applied linguistics and other speech sciences so as to explore the roles of metalinguistic variables such as syntax in reading comprehension. 5th grade students’ reading comprehension / mokhtari & niederhauser 165 second, interdisciplinary teams of literacy and language researchers should consider using a range of methods and procedures to account for variables that contribute to reading comprehension, thus enabling the impact of new issues or questions to be revealed. it is conceivable that inconsistent findings across studies may be due to the specific methods used when analyzing the data pertaining to the roles of vocabulary knowledge and syntactic awareness in reading comprehension. shiotsu and weir (2007) reported on three studies in which they found support for the relative superiority of syntactic knowledge over vocabulary knowledge in predicting performance on a text reading comprehension test. they made the case for using structural equation modeling (sem), rather than conventional regression techniques, as a more robust data analysis method, which, they argue, is more capable of accounting for the differential reliabilities of scores obtained on the measures employed. they maintain that in some published studies sem techniques, which are capable of partialing out measurement errors, might have produced coefficients less in favor of vocabulary. in our study, we were not able to use sem due to the limited number of participants and variables; however, we believe that sem would be a useful analytic technique in future research. third, researchers could design studies aimed at determining the impact of reading vocabulary and syntactic awareness instruction on students’ reading comprehension performance. establishing meaningful, causal links among these variables requires the design and implementation of empirical investigations using randomized experimental or quasiexperimental designs, which are beyond the scope of this study. the idea is that if struggling readers, given intensive instruction in vocabulary knowledge and/or syntactic awareness, make significant gains in reading comprehension performance, then the causal effect of these variables would be empirically established. the design of studies that could test the effectiveness of vocabulary and syntactic awareness instruction would be very helpful in resolving inconsistencies across studies examining the roles of vocabulary knowledge and syntactic awareness on students’ reading comprehension performance. finally, the findings of this study have important implications for curriculum development and classroom instruction. the findings point to the development of curriculum materials aimed at improving students’ vocabulary and syntactic awareness competencies, then measuring the consequences of such curricula on students’ reading achievement, as has been done quite successfully with other literacy components such as phonemic awareness. developing vocabulary and syntactic awareness curricula and conducting experimental intervention studies aimed at improving children’s vocabulary and syntactic awareness would be important steps toward establishing critical causal links between vocabulary knowledge, syntactic awareness, and reading performance. intensive instruction in vocabulary and syntax is largely untested and researchers need to evaluate the effectiveness of such instruction on students’ reading performance. in an effort to significantly advance students’ reading vocabulary and syntactic knowledge, we recommend that teachers consider integrating the teaching of vocabulary and syntax across the language arts, mathematics, science, and social studies disciplines. this approach has been used quite successfully by a group of researchers and educators affiliated with the strategic education research partnership (serp) at harvard university (snow & white, 2008). these researchers developed an evidence-based academic vocabulary program, word generation®, which has been implemented successfully in various schools in the united states and around the world. the word generation program focuses on the teaching of academic vocabulary for middle grade students across the language arts, science, mathematics, and social studies classrooms. in light of the findings of this study, which highlight the critical roles of both vocabulary and metalinguistic skills, we encourage teachers representing the international electronic journal of elementary education vol.5, issue 2, 157-170, 2013 166 language arts, social studies, science, and mathematics disciplines to devote some instructional time to the teaching of both vocabulary and syntax and to do so across their respective disciplines. . . . kouider mokhtari is the anderson-vukelja-wright endowed professor of education in the school of education at the university of texas at tyler. his research interests focus on the acquisition of language and literacy by first and second language readers. dale s. niederhauser received his ph.d. in foundations of education with an emphasis in instructional technology from the university of utah and is currently an associate professor in the school of education at iowa state university. his research interests include learning from hypertext, illstructured problem solving, and technology-using teacher development. references baumann, j. f., kame’enui, e. j., & ash, g. (2003). research on vocabulary instruction: voltaire redux. in j. flood, d. lapp, j. r. squire &, j. jensen (eds.), handbook of research on teaching the english language arts (pp. 752-785). mahwah, nj: lawrence erlbaum. beck, i. l., perfetti, c.a., & mckeown, m.g. (1982). effects of long-term vocabulary instruction on lexical access and reading comprehension. journal of educational psychology, 74, 506-521. bentin, s., deutsch, a., & liberman, ly. (1990). syntactic competence and reading ability in children. journal of experimental child psychology, 48, 147-172. blackmore, a. m. & pratt, c. (1997). grammatical awareness and reading in grade 1 children. merrillpalmer quarterly, 43, 567-590. bowey, j. a., & patel, r. k. (1988). metalinguistic ability and early reading achievement. applied psycholinguistics, 9, 367–383. bozo, w. a. (1951). a factorial study of reasoning and closure factors. psychometrika, 16, 361-386. cain, k. (2007). syntactic awareness and reading ability: is there any evidence for a special relationship? applied psycholinguistics, 28, 679-694. cain, k. (2010). reading development and difficulties. west sussex, united kingdom: british psychological society & blackwell, ltd. cain, k., & oakhill, j. (2007). reading comprehension difficulties: correlates, causes, and consequences. in k. cain & j. oakhill (eds.), children’s comprehension problems in oral and written language: a cognitive perspective (pp. 41-75). new york: guilford press. carlisle, j. f. (2004). morphological processes that influence learning to read. in a. stone, e. silliman, b. ehren, & k. apel (eds.), handbook of language and literacy: development and disorders (pp. 318–339). new york, ny: guilford press. chall, j. s., jacobs, v. a., & baldwin, l. e. (1990). the reading crisis: why poor children fall behind. cambridge, ma: harvard university press. clark, n. l. (1972). hierarchical structure of comprehension skills. hawthorn, victoria, australia: a.c.e.r. clarke, p.j., snowling, m.j., truelove, e., & hulme, c. (2010). ameliorating children’s reading comprehension difficulties: a randomised controlled trial. psychological science, 21(8), 1106–1116. 5th grade students’ reading comprehension / mokhtari & niederhauser 167 davis, f. b. (1942). two new measures of reading ability. journal of educational psychology, 33, 365-372. davis, f. b. (1944). fundamental factors of comprehension in reading. psychometrika, 9, 185-197. davis, f. b. (1968). research in comprehension in reading. reading research quarterly, 3, 499-545. de jong, p.f., & der leij, a.v. (2003). development changes in the manifestation of a phonological deficit in dyslexic children learning to read a regular orthography. journal of educational psychology, 95, 22-40. demont, e., & gombert, j.e. (1996). phonological awareness as a predictor of decoding skills and syntactic awareness as a predictor of comprehension skills. british journal of educational psychology, 66, 315-332. dixon, p., lefevre, j., & twilley, l. c. (1988). word knowledge and working memory as predictors of reading skill. journal of educational psychology, 80, 465-472. faul, p., erdfelder, e., lang, a-g., & buchner, a. (2007). g*power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. behavior research methods, 39 (2), 175191. gelderen, a. v., schoonen r., glooper, k. d., hulstijin, j., simis, a., snellings, p., & stevenson, m. (2004). linguistic knowledge, processing speed, and metacognitive knowledge in first-and secondlanguage reading comprehension: a componential analysis. journal of educational psychology, 1, 1930. gregg w., donahue p., & dion, g. (2007). the nation’s report card: 12th grade reading and mathematics 2005 (nces 2007-468). washington dc: us department of education, national center for education statistics. hammill, d., & newcomer, p. l. (1988). test of language development-intermediate (told-i:3). san antonio, tx: the psychological corporation. hunt, j. t. (1953). the relation among vocabulary, structural analysis, and reading. journal of educational psychology, 44, 193-202. kintsch, w. (1998). comprehension: a paradigm for cognition. new york: cambridge university press. layton, a., robinson, j., & lawson, m. (1998). the relationship between syntactic awareness and reading performance. journal of research in reading, 21, 5-23. leikin, m. (2002). processing syntactic functions of words in normal and dyslexic readers. journal of psycholinguistic research, 31(2), 145-163. macginitie, w.h., macginitie, r. k., maria, k., dreyer, l. g., & hughes, k. e. (2000). gates-macginitie reading tests (4th ed.). boston, ma: riverside. muter, v., hulme, c., snowling, m., & stevenson, j. (2004). phonemes, rimes, vocabulary, and grammatical skills as foundations or early reading development: evidence from a longitudinal study. developmental psychology, 40, 6650681. nagy, w.e., & herman, p.a. (1987). breath and depth of vocabulary knowledge: implications for acquisition and instruction. in m.g., mckeown & m.e. curtis (eds.). the nature of vocabulary acquisition (pp. 19-35). hillsdale, nj: erlbaum. nation, k., cocksey, j., taylor, j., & bishop, d. (2010). a longitudinal investigation of early reading and language skills in children with poor reading comprehension. journal of child psychology and psychiatry, 51, 1031–1039. national center for education statistics. (2012). the nation’s report card: vocabulary results from the 2009 and 2011 naep reading assessments (nces 2013–452). institute of education sciences, u.s. department of education, washington, d.c. international electronic journal of elementary education vol.5, issue 2, 157-170, 2013 168 national institute of child health and human development. (2000). the report of the national reading panel. teaching children to read: an evidence-based assessment of the scientific literature on reading and its implications for reading instruction. washington, dc: u.s. government printing office. nichd early child care research network. (2005). early child care and children's development in the primary grades: results from the nichd study of early child care. american educational research journal, 43, 537–570. nippold, m.a. (1998). later language development: the school-age age and adolescent years. austin, tx: pro-ed. nippold, m.a. (2004). research on later language development: international perspectives. in r. berman (ed.), language development across childhood and adolescence (pp. 1-9). philadelphia, pa: john benjamins publishing company. oakhill, j., & cain, k. (2007). introduction to comprehension development. in j. oakhill and k. cain, children’s comprehension problems in oral and written language. (pp. 1-40). new york: guilford press. oakhill, j., cain, k., & bryant, p. e. (2003). the dissociation of word reading and text comprehension: evidence from component skills. language and cognitive processes, 18, 443-468. proctor, c., carlo, m., august, d., & snow, c. (2005) native spanish-speaking children in english: toward a model of comprehension. journal of educational psychology, 9, 246-156. ruddell, m. r. (1994). vocabulary knowledge and comprehension: a comprehension process view of complex literacy relationships. in r. b. ruddell, m. r. ruddell, & h. singer (eds.), theoretical models and processes of reading (pp. 414–447). newark, de: international reading association. scott, c. (2004). syntactic contributions to literacy learning. in c.a., stone, e.r. silliman, b.j. ehren, & k. apel (eds.). handbook of language and literacy: development and disorders (pp. 340-362). new york, ny: gilford press. shiotsu, t., & weir, c.j. (2007). the relative significance of syntactic knowledge and vocabulary breadth in the prediction of reading comprehension test performance. language testing, 24, 99-128. snow, c. (2002). reading for understanding: toward an r & d program in reading comprehension. arlington, va: rand education. snow, c. e., & white, c. (2008). word generation. boston, mass: strategic education research partnership. snow, c.e., burns, m.s., & griffin, p. (1998). preventing reading difficulties in young children. washington, dc: national academy press. stanovich, k.e., & cunningham, a.e. (1992). studying the consequences of literacy within a literate society: the cognitive correlates of print exposure. memory & cognition, 20, 51-68. tabachnick, b. g., & fidell, l. s. (2013). using multivariate statistics (6th edition). boston: allyn and bacon. thorndike, r.l., (1973). reading as reasoning: a study of mistakes in paragraph reading. journal of educational psychology, 8, 323-332. torgesen, j.k., wagner, r.k., rashotte, c.a., burgess, s., & hecht, s. (1997). contributions of phonological awareness and rapid automatic naming ability to the growth of word reading skills in second to fifth grade children. scientific studies of reading, 1, 161-185. tunmer, w. e., & hoover, w. (1992). cognitive and linguistic factors in learning to read. in p. b. gough, l. c. ehri & r. treiman (eds.), reading acquisition (pp. 175-214). hillsdale, nj: erlbaum. tunmer, w.e., & bowey, j.a. (1984). metalinguistic awareness and reading acquisition. in w.e. tunmer, c. pratt, & m.l. herriman (eds.), metalinguistic awareness in children (pp. 144-168). new york: springer-verlag. 5th grade students’ reading comprehension / mokhtari & niederhauser 169 whipple, g. (ed.). (1925). the twenty-fourth yearbook of the national society for the study of education: report of the national committee on reading. bloomington, il: public school publishing. willows, d. m., & ryan, e. b. (1986). the development of grammatical sensitivity and its relationship to early reading achievement. reading research quarterly, 21, 253–266. international electronic journal of elementary education vol.5, issue 2, 157-170, 2013 170 www.iejee.com this page is intentionally left blank http://www.iejee.com/ international electronic journal of elementary education, june 2016, 8(4), 589-600. issn:1307-9298 copyright © iejee www.iejee.com babysitter or professional? perceptions of professionalism narrated by ontario early childhood educators debra harwood a stephanie tukonic b a brock university, canada b community services of niagara region, canada received: 12 march 2016 / revised: 30 may 2016 / accepted: 7 june 2016 abstract self-constructed perceptions of early childhood educator professionalism were explored in a smallscale mixed-method survey using convenience sampling. fifty-four educators in ontario, canada participated in the study and shared their views related to professionalism. a mixed-methodology was employed using an electronic questionnaire composed of closedand open-ended questions on ideas related to professionalism. the results of the study demonstrated that all the educators held a strong self-perception of professionalism regardless of their level of education, reporting high levels of job satisfaction, competence, recognition as a professional from others, and selfrecognition as a professional. participants’ notions of professionalism focused on the qualities of an individual considered a professional (e.g., good listener, patient, and understanding) whereas discussions of criticality, authority, or the historical, gendered, cultural, racial, and social practices of early childhood education were noticeably absent. keywords: early childhood educator, professionalism, mixed methods study. introduction who is the modern-day early childhood educator? multiple labels have been utilized to describe early childhood educators (eces), ranging from specialists to child-minders or babysitters. thus, it is not surprising that a consistent and well-identified professional identity within this field has yet to materialize. historically, early childhood education and care services (ecec) for children aged birth to age 6 years in many western nations have been housed within a health and social care sector and under the directive of providing ‘care’ (kamerman, 2006; murphy, 2015). early in the 19th century a clear distinction was made between ‘kindergartens’ providing education and ‘day nurseries’ providing care (kamerman, 2006, p. 3). even today “divisions between childcare facilities and those that had an educational mandate [are] not always clear” (prochner, 2000, p. 20) with many ‘day nurseries’ offering educative components as a complement to providing care. here, the authors use the term early childhood education and care to reflect a growing trend within canada to bridge the philosophical divide between kindergartens and childcare.  corresponding author: debra harwood, associate professor, brock university, 1812 sir issac brock way, st. catharines, on canada l2s 3a1 dharwood@brocku.ca (001) 905-688-5550 http://www.iejee.com/ international electronic journal of elementary education vol.8, issue 4, 589-600, june 2016 590 the concept of professionalism within ecec remains a somewhat contested idea; imbued with historical and contextual meanings as well as reconceptualization discourses that aim to challenge notions of professionalism as static and linear (pacini-ketchabaw, nxumalo, kocher, elliot, & sanchez, 2015). thus, this study sought to gain a greater understanding of professionalism from educators themselves. how do educators of young children negotiate this seemingly increasingly complex and shifting system of early education and care? given the canadian context and historical significance of the evolution of care and education within the country, how does the contemporary educator define professionalism? the canadian context the canadian context of ecec is somewhat complex, with 10 provinces and three territories each charged with the authority to legislate in the area of childcare (educational authority is also the responsibility of provinces and territories). thus, the federal government of the country plays no direct legislative role in regulating childcare with the exception of providing social services for aboriginal peoples and the armed forces. indirectly, the federal government can impact the childcare system of individual provinces through constitutional spending powers, that is, the government of canada can divert federal monies to provinces to support specific childcare policies or direct tax credits to families. in 2006, the federal government of canada initiated the universal childcare taxable benefit, a direct payment to parents of $100 per month per child (government of canada, 2016). some critics have argued that the lack of a federal policy and direction on a national childcare system, as well as the grossly inadequate $100 benefit, contributes to the fragmented childcare system that exists within the country (amoroso, 2010). each of the provinces and territories’ childcare systems is unique with only the province of quebec offering universal access to childcare. the legislative and administrative oversight within each province and territory is still largely fragmented with some provinces maintaining oversight for kindergarten with ministry or departments of education and childcare housed within social services departments (e.g., manitoba’s department of family services and community affairs). however, shifts are occurring and “eight jurisdictions within canada now have a lead department responsible for early childhood services (prince edward island, new brunswick, ontario, saskatchewan, north west territories, nunavut, nova scotia, and newfoundland” (akbari & mccuaig, 2014, p. 6). in 2010, ontario (the province where the study was conducted) began implementing fullday early learning kindergarten program (elkp) model within the formal education system through a shift in jurisdiction from the ministry of children and youth services to the ministry of education. currently the jurisdiction for early learning services resides within the ministry of education, and services for children aged 3 years 8 months to 5 are provided within the kindergarten schooling system (the provision of regulated child care services for children aged 0-3.8 is delivered by the private sector). the success of the new elkp model relies on the collaboration and professionalism of both a primary school teacher and early childhood educator within the same classroom (ontario ministry of education, 2010). to date, little research is available that examines if and how these two separate ideological systems of education and care can successfully merge within this ontario elkp model. moreover, the divisive nature of educator roles, educational qualifications, compensation, and licensure of early childhood educators and teachers presents both obstacles and challenges to the model. within the ontario context, early childhood educators’ roles can be complex as some educators act as frontline practitioners and/or directors within childcare, assuming administrative responsibilities and leadership duties in addition to working directly with children. additionally, early childhood educators require a 2-year tertiary college diploma and kindergarten teachers require a 4 babysitter or professional? / harwood & tukonic 591 year university-level degree. moreover within canada, an early childhood educator’s annual average salary is less than half that of an elementary or kindergarten teacher’s annual salary (statistics canada, 2011), and two separate entities are responsible for governance and licensure of these professions. thus, understanding who early childhood educators think they are appears as a logical first step in navigating the new terrain inherent with the ministry of education’s integration of services for young learners. historical evolution of two distinct sectors the 200-year history of programs of care and education for young children within canada contributes to both the diversity and complexity of the services provided for youngsters and their families (prochner & howe, 2000). the discipline of early childhood education (birth to age 6) has largely evolved from a health and social care sector orientation within many westernized nations, including canada (kamerman, 2006; murphy, 2015). historically within north america, the evolution of kindergarten found a home primarily with the education sector (prochner, 2000). initially started as private organizations in canada, by 1887 ontario kindergartens were incorporated into the public school system while early childhood education continued to be delivered within a market-based system (varga, 2000). the market-based system relies on a fee for services model and programs are made available when financial viability can be clearly demonstrated. beach and ferns (2015) argue that a market-based system can negatively impact every aspect of the early childhood educational system, including regulations, programming, educator wages, access, and quality. borrowing a phrase coined by sawyer (1999) in reference to the australian context, perhaps the distinct historical paths within ontario also contributes to the ‘difference and separateness’ of the fields of care and education (clare & jennifer, 2003; sawyer, 2000). the distinct training and qualification processes provide two additional prospects for further exploration into how these practices may perpetuate differences and separateness. (for a complete discussion on the relationship between the field of early childhood education and public education see moss, 2013b). in canada’s evolution of ecec (i.e., kindergarten and childcare), women have played a dominant role (prochner, 2000). up until the 1960s day nurseries (the precursor to childcare centers) were most often operated by private charitable organizations. women or matrons with “experience and ability in managing a household (including the children of the household), a reputation for honesty, skill in dealing with people, and tolerance for the hard work involved” (varga, 2000, p. 78) were employed to head and operate day nursery organizations. the first laboratory preschools in canada were compensatory education programs for ‘disadvantaged’ children and families. these laboratory preschools were also largely the result of dedicated women (mayfield, 2001). for example, in the maritimes compensatory programs were initiated and sponsored by the national council of jewish women (ryan, 1972). resultant from the foresight of dr. mary j. wright, the university of western ontario laboratory preschool was also started and described as “the rolls-royce of university-affiliated schools in canada” (howe, 1994, p. 17). additionally, the development of kindergartens in canada was fuelled by the efforts of adaline agusta marean, the head teacher of toronto’s first public-school kindergarten (established in 1883), which also served as a model for other kindergarten classrooms in southern ontario (mayfield, 2001). historically, education and training within ecec has followed two distinct routes: (1) teacher education and university degree-level credentials for individuals working within the education sector; and (2) college diplomas/certificates and training institutes for those intending to work in the childcare field (varga, 2000). unlike the long-standing tradition of teacher education, formal training requirements for early childhood educators did not appear until well into the 20th century (varga, 2000). early childhood educators within international electronic journal of elementary education vol.8, issue 4, 589-600, june 2016 592 nursery schools and childcare were individuals who received minimal training and little compensation. educators were considered appropriate for the role if they demonstrated ‘motherly’ type characteristics. until the 1930s, nurses were also employed to administer care to the youngest groups of children (varga, 2000); most often this ‘care’ role was fulfilled by women with minimal formal health care training (mayfield, 2001). the idea of specialized training for early childhood educators firmly took root in the 1920’s and was influenced by ‘ages and stages’ developmental theories (gesell & ilg, 1949), and research focused on early experiences and personality development (ainsworth, 1978; bowlby, 1979, 2004; gesell & ilg, 1949; sroufe, 1979; weber, 1984; wright, 2000). contemporary educators the position of the early childhood educator is complex and requires a diversity of skills in “knowing, being, experiencing, and acting” (goodfellow, 2001, p. 17; miller, dalli, & urban, 2012). the role requires an educator to be a keen observer of children’s needs as well as a teacher of skills and knowledge in literacy and numeracy (a common purpose shared with the public elementary school system), as well as social and self-care management (varga, 2000). yet, despite the domain-specific knowledge, pedagogical, and theoretical understanding required of an early childhood educator, regulations governing educational levels and certification processes have been slow to materialize. and given the historical roots that conceptualized early childhood education as a caring vocation, perceptions of professionalism of the workforce have been fraught with tension (moyles, 2001). teaching of young children is largely dominated by women in westernized countries (unesco, 2006) and some researchers question whether the feminization of teaching “result in a reduction in the professional status of teaching” (drudy, 2008, p. 309). the level of care and ‘passion’ educators themselves describe as central to their role (moyles, 2001) has traditionally acted ‘to disempower early childhood practitioners from claiming professional status’ (dalli, 2002). the societal marginalization of the predominantly female workforce has negatively impacted wages, typically low in comparison to their counterparts in the elementary school system, as well as limited educational and career opportunities (doherty, lero, goelman, lagrange, & tougas, 2000). in general, society tends to de-value the role of early childhood educators (noddings, 2001). educators, themselves, have been reluctant to actively participate in the construction of a professional identity and critique of the taken-for-granted ‘good’ ece (langford, 2006). as langford explains, early childhood educators are enculturated into a field that appears to perpetuate marginalization through their training and educational experiences. she contends that what appears to be lacking are opportunities for the “practice of criticality”, that is, opportunities for educators to explore the role of identity, subjectivity, and social positioning in relation to the discourses “within which they are constituted and controlled” (langford, 2006, p. 115). however, the last decade has seen unprecedented international interest in ideas and concepts of professionalism in ecec (harwood, klopper, osanyin, & vanderlee, 2012; harwood & tukonic, 2015; miller & cable, 2011). a growing number of national frameworks and regulatory practices have emerged (miller, dalli, & urban, 2012), as well as materialization of new degree programs (miller & cable, 2011), and spaces for educators to engage in discourse about their practice and sense of professionalism have also flourished. the international project, ‘a day in the life of an early years practitioner’ is just one example of this keen interest in professionalism. the day in the life study involved researchers from six different countries (australia, england, finland, germany, new zealand and sweden) exploring situated concepts of the meaning of professionalism (dalli, 2011). a ‘critical ecology’ of early childhood educator professionalism appears to be emerging. as dalli (2010) explains:a critical ecology as involving alertness to the challenges in its settings and to the babysitter or professional? / harwood & tukonic 593 strengths that can be brought to bear to make the present better... the term ‘critical ecology of the profession’ carries with it the idea that members of a profession would be critical of themselves as a community. (p. 70) integral to this orientation of professionalism is a view of educators as change agents, reflective and self-critical, both advocates and leaders (dalli, 2007; miller et al., 2012). it is not clear how educators would manage to construct a sense of professionalism within a profession that is “fluid, contentious and constantly under reconstruction” (dalli & urban, 2010, p. 151). moreover, given the state of flux within ontario when this study was conducted (in the second year of the 5-year implementation of the elkp model), how an early childhood educator would achieve this level of criticality is unclear. the study (method) the purpose of the study was to explore self-constructed perceptions of early childhood educator (ece) professionalism. a mixed-method research study using an electronic questionnaire was carried out to explore 54 early childhood educators’ ideas related to professionalism. these educators were recruited using a convenience sampling method. potential participants were contacted via email and a letter of invitation and consent materials were distributed on behalf of the researchers by a regional professional development agency. informed by previous research on ece professionalism (dalli, 2008; harwood & tukonic, 2015; martin, meyer, jones, nelson, & ling, 2010; rodgers & raiderroth, 2006), the main purpose of the study was explored, that is how does early childhood educators define professionalism? quantitative and qualitative data sets were collected concurrently using an electronic survey composed of closedand open-ended questions. quantitative data collection relied on likert scale responses to eleven closed-ended questions used to describe the professionalism. for example, participants were asked to rate their (a) level of commitment to their role as an ece, (b) levels of enjoyment of their profession, and (c) how qualified they felt in a mentorship role for others entering the field. qualitative data collection relied on written responses to two open-ended questions related to professional qualities and acting/behaving professionally. in addition to the mixed methods design, a transformative data model (creswell & plano clark, 2007) was adopted to collect and analyze quantitative and qualitative data sets separately. specific to this model, after the initial analysis of each data set, one of the data sets is transformed into the other type. this is accomplished by quantifying the qualitative results or qualifying the quantitative results in which case the “transformation allows that data to be mixed during the analysis stage and facilitates the comparison, interrelation, and further analysis of the two data sets” (p. 65). for the purpose of this study, the qualitative data set was transformed into quantitative data and used to describe the impact of education and experience on perceptions of professionalism. quantitative and qualitative analysis the quantitative data statistical treatment included descriptive analysis and limited inferential statistical examination. descriptive statistics were calculated for the frequency of each response, mean and standard deviation. in addition, there were several nonparametric tests of analysis conducted to evaluate the perceptions of professionalism of early childhood educators and answer the determined research questions using spss 19.0 statistics program. all inferential tests were conducted with a confidence interval of 95%. a spearman rho correlation was used to determine if relationships existed between the perceptions of professionalism amongst early childhood educators. the independent variables in the study were identified as (a) previous roles as an early childhood educator, (b) level of education attained, and (c) number of years of experience. the previous roles international electronic journal of elementary education vol.8, issue 4, 589-600, june 2016 594 as an early childhood educator were defined in three levels – ece, ece and director/manager, and director/manager. the level of education attained was defined in two levels – two years and more than two years of tertiary education. the number of years of experience was defined in three levels – less than 10 years; between 10 and 20 years; and more than 20 years of experience. a mann -whitney u was used to determine if a difference existed between the level of education attained and the perceptions of professionalism of the early childhood educators. a kruskal-wallis test of one-way variance analysis was used to determine if a difference existed in the years of experience and previous roles as an early childhood educator, and the perceptions of professionalism of early childhood educators. finally, a chi square test was used to determine if a relationship existed between the three independent variables identified amongst the early childhood educator sample. qualitative analysis techniques were used in relation to the two open-ended questions asked of participants. the responses to the two open-ended questions were collapsed into one data set and analyzed using nvivo 10 to generate theme-based frequencies (creswell, 2013). the most frequently identified qualities and behaviours of a professional were identified as: (1) knowledge (both content and pedagogical knowledge), (2) child development knowledge, (3) development as a professional, (4) being attentive and demonstrating a care ethic, and (5) the ability to communicate and establish relationships. transformation of data the transformation of the qualitative data into a quantitative data set began with transforming the top five frequently occurring codes as new dependent variables. the variables were coded dichotomously for each participant’s response as being either present or not present. an additional variable was created to report the total sum of the codes used in the responses with a minimum of zero to a maximum of five in total. the quantified transformed qualitative data were then analyzed using several nonparametric tests of analysis to evaluate the perceptions of professionalism of early childhood educators using spss 19.0 statistics program. all tests were conducted with a confidence interval of 95%. a mann whitney u test was used to investigate if a difference existed in the type of words used to describe professionalism in ece between the two levels of education attainment (e.g., 2 years of education and more than 2 years of education). also, a kruskal-wallis test of one-way variance analysis was used to investigate if a difference existed in the type of words used to describe professionalism as a factor of the educators’ levels of years of experience and previous roles. finally, the results of the initial quantitative data analysis, and of the quantified transformed qualitative data analysis, were related and compared to inform the appropriate responses to the research questions. findings the largely female group of participants tended to be proficient or experts in their field (katz, 1972) with many also reporting a previous role as a director/manager of a centre (i.e., in an ontario context the director maintains responsibility for administrative and leadership duties and may or may not also have educator responsibilities working directly with children) (table 1). overwhelmingly, the majority of the respondents held a 2 yeardiploma qualification; adhering to the minimal qualification expectation of the province. babysitter or professional? / harwood & tukonic 595 table 1. participant demographic frequencies and percentages demographic frequency valid percent (%) gender (n=54) male 1 1.9 female 53 98.1 years of experience (n=53) less than 10 years 11 20.7 ten to 20 years 18 34.0 more than 20 years 24 45.3 education level (n=54) college diploma (2 years) 39 72.2 university of applied degree (4 years) 5 9.3 university level course (3 years) 6 11.1 graduate level degree (6 years) 4 7.4 previous role (n=54) ece 39 39 director/manager 72.2 72.2 other 37 37 a mann-whitney mann u test was used to compare variance in the perceptions of professionalism between early childhood educators with two years of education and early childhood educators with more than two years of education. there was no variance in the educators’ perceptions of professionalism between both levels of education. all educators reported a strong sense of professionalism regardless of their level of education, reporting high levels of job satisfaction, competence, recognition as a professional from others, and self-recognition as a professional. the kruskal-wallis test results were statistically significant for the number of years of experience and the reported feelings of being qualified to help others learn about the role of an ece (i.e., participants were asked to rank their comfort level in mentoring others) (h(2)=6.49; p= 0.039). additionally, the variance between the number of years of experience and the perception of being viewed by others as a professional was also statistically significant (h(2)=9.17; p= 0.010) with educators who had more than 20 years’ experience seeing themselves as more qualified to act as a mentor and more highly viewed by others as a professional relative to educators with less than 10 years of experience. furthermore, the perception of being viewed by others as a professional displayed a moderate degree of correlation (r= 0.653, p>0.05) with the knowledge of a single child (and the ability to use that knowledge to plan curriculum for a group of children) only for educators with less than 10 years’ experience. perhaps, this parallel is indicative of the shift to emergent and inquirybased learning frameworks that have also developed within the last 10 years. for example, ontario, british columbia, prince edward island, and new brunswick all have frameworks that place the child and the child’s interest at the center of learning and teaching. the kruskal-wallis test was statistically significant for the type of role held in the early childhood setting and the enjoyment of work (h(7)= 14.38; p= 0.045). specifically, those educators who also had administrative duties and responsibilities attached to their roles international electronic journal of elementary education vol.8, issue 4, 589-600, june 2016 596 reported less overall satisfaction with their job. perhaps, as administrative duties were often added onto one’s role and existing duties, as well as the increasing emphasis and trend toward greater accountability, those performing roles as administrators need differentiated types of professional supports and experiences to maximize job satisfaction. qualities and behaving professionally as described in the methodology section, the transformation of the qualitative data into a quantitative data set allowed for the six new variables (i.e., the five themes and the total number of codes specified by each participant) to be analyzed and used to determine if there was a statistically significant relationship between the descriptors used by the participants and (a) type of role, (b) years of experience, and (c) level of education. in terms of the roles undertaken by the participants (ece, director/manager, combined role) and level of education, no statistical significance was found. however, the years of experience and use of the descriptor ‘knowledge’ in describing professionalism were related in a statistically significant way (table 2). educators with 10 to 20 years of experiences used the term knowledge less often to describe professionalism compared to educators with fewer than 10 years of experience. more experienced educators tended to use terms associated with communication or relationships to describe professionalism in early childhood. table 2. one-way variance analysis-years of experience and perception of ‘knowledge’ codes chi-square value df p-value knowledge 6.599 2 0.037 development of child 0.014 2 0.993 development of professional 1.311 2 0.519 caring and attention 3.871 2 0.144 communication 2.115 2 0.347 total codes 9.728 2 0.008 implications while somewhat limited by the small sample size and relatively homogenous group of participants, this study holds several implications for the professionalism literature and the training of educators. the findings can help to fuel the dialogue and focus on the question, who is the early childhood educator? though once characterized as the “emotive work” (guy & newman, 2004, p. 289) of women, the participants of this study demonstrated a clear understanding of their role as encompassing more than the provision of care. while ‘care’ figures prominently in these educators’ conceptualization of professionalism, other constructs such as foundational knowledge, advocacy, and mentorship were also reported. moreover, knowledge (content and pedagogical knowledge), child development understanding, and growth as a professional were all cited more frequently than ‘care’. perhaps, the participants’ emphasis of the importance of knowledge as a core construct in being a professional aligns with gilligan’s (1982) notion of an ‘ethic of care’ in which actions are seen “as ethical in expressing and maintaining caring relationships rather than in being morally correct in an objective sense” (taggart, 2011, p. 86), where knowledge is key to maintaining these relationships. the following comments from two participating educators help to underscore the significance of the reciprocal and interdependent nature of caring and learning: “i would recognize professionalism by someone who exhibits a true knowledge of development and also a genuine understanding of the many different family dynamics we see today”. babysitter or professional? / harwood & tukonic 597 “professionals are engaged in what the children are doing, listening to parents and children, and mentoring other peers”. concepts of professionalism and critical discourse of what it means to be an educator is noticeably absent from educator training and professional development programs within ontario. as langford’s (2006) ontario-based research also highlighted “the ‘good’ early childhood educator constructed in a college training program is, by and large, taken for granted” (p. 115). moreover commonly used textbooks (e.g., crowther, 2005; essa, 2011; gestwicki & betrand, 2012; gordon & browne, 2007) and training programs tend to focus the professionalism discussion on the personal and qualities needed as an early childhood educator (e.g., care, passion, attentiveness to children), the ethics or standards of practice (i.e., the regulatory aspect of professionalism), and the mechanics of the ece role (e.g., pay, credentialing, burnout). noticeably absent from the discussion is a focus on how educators are co-opted into a pre-existing marginalized ideal of this ‘good’ educator (langford, 2006; moyles, 2001; noddings, 1984; osgood, 2006). our participants also tended to focus on the qualities of an individual they defined as a professional (e.g., a good listener, patience, and being understanding). lacking from the participants responses to the open-ended questions was any discussion of criticality, authority, or the historical, gendered, cultural, racial, and social practices of early childhood education. yet, these types of conversations are essential in constructing a professional identity, enabling a mode of discourse to develop where educators begin “to understand the discursive professional identity within which they are [currently] constituted and controlled” (langford, 2006, p. 115). early childhood education texts and training programs (and research) need to create spaces for educators to discuss, construct, and deconstruct ideals of professionalism and taken-for-granted assumptions about ecec. it is essential that educators’ abilities to be critical and self-reflective be fostered. educators must be empowered to see how they themselves are enculturated into ecec systems, deconstructive type skills that should be persistently fostered within training programs. multiple perspectives of ece professionalism are needed. pacini-ketchabaw et al. (2015) encourage processes that afford the opening up of possibilities to challenge one’s assumptions, to think differently, whereby “educators construct themselves as critical pedagogues” (p. 145) constantly questioning and complexifying the becoming of a professional. conclusion this small-scale study is only a beginning step in understanding how ontario early childhood educators conceptualize professionalism. in the changing landscape of ecec it is important for all educators to engage in processes that foster constructing, deconstructing, and reconstructing a professional identity (dalli & urban, 2010; sachs, 2000; woodrow, 2008; woodrow & christine, 2007). the participants of this study demonstrated a high sense of professionalism and felt others viewed them as such. moreover, the educators discussed their high levels of commitment, role satisfaction, and shared that they did perceive themselves as capable of acting as mentors for others and to act as resources for parents. notably, the participating educators were experts in their field with 45% of the respondents having 20 or more years of experience and it is unclear if they were employed within the integrated full-day kindergarten model that was unfolding within ontario at the time of data collection. perhaps, novice educators and those embedded within the model would have diverse and distinct views and critical insights about professionalism; highlighting potential avenues for future research exploration. although the responses from the participants of this study help fuel the discussion of contemporary professionalism and challenge the historical notions of an ece, noticeably absent from the participants’ responses was any discussion of how international electronic journal of elementary education vol.8, issue 4, 589-600, june 2016 598 professionalism is instituted and controlled. this suggests that future research, foundational textbooks, and early childhood education training programs should create spaces for a critical ecology of ece professionalism to emerge within the ontario context specifically and beyond. • • • references ainsworth, m. d. s. (1978). patterns of attachment: a psychological study of the strange situation. hillsdale, nj: lawrence erlbaum associates. akbari, e., & mccuaig, k. (2014) early childhood education report 2014. toronto: on: institute for studies in education. amoroso, j. (2010). from women to children: reframing child care in canada. queen’s policy review, 1(1), 30–46. beach, j., & ferns. c. (2015). from child care market to child care system. retrieved from https://www.policyalternatives.ca/sites/default/files/uploads/publications/national%20offic e/2015/09/os120_summer2015_child_care_market_to_child_care_system.pdf bowlby, j. (1979). on knowing what you are not supposed to know and feeling what you are not supposed to feel. the canadian journal of psychiatry / la revue canadienne de psychiatrie, 24(5), 403–408. bowlby, j. (2004). the role of childhood experience in cognitive disturbance. in a. freeman, m. j. mahoney, p. devito, & d. martin (eds.), cognition and psychotherapy (2nd ed.). (pp. 101–122). new york, ny: springer publishing. creswell, j. w. (2013). research design: qualitative, quantitative, and mixed methods approaches (4th ed.). los angeles, ca: sage. creswell, j. w., & plano clark, v. l. (2007). designing and conducting mixed methods research. thousand oaks, ca: sage publications. crowther, i. (2005). introduction to early childhood education-a canadian perspective. toronto, on: nelson education ltd. dalli, c. (2002). being an early childhood teacher: images of professional practice and professional identity during the experience of starting childcare. new zealand journal of educational studies, 37(1), 73-85. dalli, c. (2007). towards a critical ecology of the profession: systematic approaches to policies, practices, and understandings of professionalism and professionalisation in early childhood. paper presented at the european early childhood education research association annual conference, prague. dalli, c. (2008). pedagogy, knowledge and collaboration: towards a ground-up perspective on professionalism. european early childhood education research journal, 16(2), 171-185. dalli, c. (2010). towards the re-emergence of a critical ecology of the early childhood profession in new zealand. contemporary issues in early childhood, 11(1), 61-74. dalli, c. (2011). a curriculum of open possibilities: a new zealand kindergarten teacher's view of professional practice. early years: journal of international research & development, 31(3), 229243. babysitter or professional? / harwood & tukonic 599 dalli, c., & urban, m. (2010). professionalism in early childhood education and care: international perspectives. london: routledge. doherty, g., lero, d.s., goelman, h., lagrange, a., & tougas, j. (2000). you bet i care: a canada wide study on wages, working conditions, and practices in child care centres (w. centre for families, and well-being ed.). guelph, on: centre for families, work, and well-being. drudy, s. (2008). gender balance/gender bias: the teaching profession and the impact of feminisation. gender & education, 20(4), 309–323. friendly, m., halfon, s., beach, j., & forer, b. (2013). early childhood education and care in canada 2012 (9th ed.). toronto, on: childcare research and resource unit. essa, e.l. (2011). introduction to early childhood education (6th ed.). belmont, ca: cengage learning. gesell, a., & ilg, f.l. (1949). child development: an introduction to the study of human growth. new york: harper & row. gestwicki, c., & betrand, j. (2012). essentials of early childhood education (4th ed.). toronto, on: nelson education ltd. gilligan, c. (1982). in a different voice: psychological theory and women's development. cambridge, ma: harvard university press. goodfellow, j. (2001). wise practice: the need to move beyond best practice in early childhood education. australian journal of early childhood research, 26(3), 1-6. gordon, a.m., & browne, k.w. (2007). beginning essentials in early childhood education. clifton park, ny: thomson delmar learning. government of canada (2016, february 15). canada revenue agency-universal child care benefits [online]. retrieved from http://www.cra-arc.gc.ca/uccb/ guy, m.e., & newman, m.a. (2004). women's jobs, men's jobs: sex segregation and emotional labor. public administration review, 64(3), 289-298. harwood, d., klopper, a., osanyin, a., & vanderlee , m. l. (2012): ‘it’s more than care’: early childhood educators’ concepts of professionalism. early years: an international journal of research and development, 33(1), 4-17. harwood, d., & tukonic, s. (2015). the glass ceiling effect: mediating influences on early years’ educators sense of professionalism. canadian children, 40(1), 36-54. howe, n. (1994). reflections on early childhood education training: a tribute to mary j. wright. canadian children, 19(2), 16-19. kamerman, s.b. (2006). a global history of early childhood education and care. paper commissioned for the efa global monitoring report 2007, strong foundations: early childhood care and education: united nations educational, scientific and cultural organization. katz, l.g. (1972). developmental stages of preschool teachers. elementary school journal, 73(1), 5054. langford, r. (2006). discourses of the good early childhood educator in professional training: reproducing marginality or working toward social change. international journal of educational policy, research, & practice: reconceptualizing childhood studies, 7, 115-125. martin, s., meyer, j., jones, r.c., nelson, l., & ling, t. (2010). perceptions of professionalism among individuals in the childcare field. child & youth care forum, 39(5), 341-349. mayfield, m. (2001). early childhood education and care in canada: contexts, dimensions, and issues. toronto, on: prentice hall. mccain, margaret, m., fraser j., & mccuaig, k. (2011). early years study 3: making decisions, taking action. toronto, on: fraser mustard institute. international electronic journal of elementary education vol.8, issue 4, 589-600, june 2016 600 miller, l., & cable, c. (2011). the changing face of professionalism in the early years in l. miller & c. cable (eds.), professionalism, leadership and management in the early years (pp. 1-10). london, uk: sage publications. miller, l., dalli, c., & urban, m. (2012). early childhood grows up: towards a critical ecology of the profession (1st ed.). new york: springer. moyles, j. (2001). passion, paradox and professionalism in early years education. early years: journal of international research & development, 21(2), 81-95. murphy, r. (2015). early childhood education in ireland: change and challenge. international electronic journal of elementary education, 8(2), 287-300. noddings, n. (1984). caring: a feminine approach to ethics and moral education. berkeley, ca: university of california press. noddings, n. (2001). the care tradition: beyond "add women and stir". theory into practice, 40(1), 29. osgood, j. (2006). deconstructing professionalism in early childhood education: resisting the regulatory gaze. contemporary issues in early childhood, 7, 5–14. doi:10.2304/ciec.2006.7.1.5 pacini-ketchabaw, v., nxumalo, f., kocher, l., elliot, e., & sanchez, a., (2015). journeys: reconceptualizing early childhood practices through pedagogical narration. toronto, on: university of toronto press. prochner, l. (2000). a history of early education and childcare in canada, 1820-1966. in l. prochner & n. howe (eds.), early childhood care and education in canada (pp. 11-65). vancouver, bc: ubc press. rodgers, c.r., & raider-roth, m.b. (2006). presence in teaching. teachers and teaching: theory and practice, 12(3), 265-287. ryan, t.j. (1972). poverty and the child: a canadian study. toronto, on: mcgraw-hill ryerson. sachs, j. (2000). the activist professional. journal of educational change, 1(1), 77–95. sroufe, a. l. (1979). the coherence of individual development: early care, attachment, and subsequent developmental issues. american psychologist, 34(4), 834–841. statistics canada. (2011). wage and salary statistics. retrieved february 16, 2016 from http://www.servicecanada.gc.ca/eng/qc/job_futures/statistics/4214.shtml#stats taggart, g. (2011). don't we care?: the ethics and emotional labour of early years professionalism. early years, 31(1), 85-95. unesco. (2006). teachers and educational quality: monitoring global needs for 2015. paris, fr. retrieved from www.uis.unesco.org/library/documents/teachers06-en.pdf varga, d. (2000). a history of early-childhood-teacher education. vancouver, bc: ubc press. weber, e. (1984). ideas influencing early childhood education: a theoretical analysis. new york: teachers college press. woodrow, c. (2008). discourses of professional identity in early childhood: movements in australia. european early childhood education research journal, 16(2), 269–280. woodrow, c. (2007). w(h)ither the early childhood teacher: tensions for early childhood professional identity between the policy landscape and the politics of teacher regulation. contemporary issues in early childhood, 8(3), 233–243. wright, m. (2000). toronto’s institute of child study and the teachings of w.e. blatz. in l. prochner & n. howe (eds.), in early childhood care and education in canada (pp. 96–114). vancouver, bc: ubc press. 161 copyright © www.iejee.com issn: 1307-9298 © 2022 published by kura education & publishing. this is an open access article under the cc by ncnd license. (https://creativecommons.org/ licenses/by/4.0/) international electronic journal of elementary education december 2022, volume 15, issue 2, 161-170 effects of balance training using action songs on postural control and muscle strength in preschool children dimitris chatzopoulosa,*, panagiotis varsamisb, panagiotis papadopoulosc, athanasios giannakosd, georgios lykesase, sophia d. papadopoulouf abstract introduction postural control and muscle strength are important factors for the performance of everyday activities and reduce the possibility of fall-related injuries. the purpose of this study was to investigate the efficacy of a balance training program using action songs on balance and strength of preschool children. a total of 51 preschool children were randomly divided into an intervention group (25 children) and a control group (26 children). the intervention group received a balance training program with action songs (six weeks, two times a week, total 12 sessions), while the control group followed the normal curriculum. prior to and after the intervention, both groups were assessed in dynamic balance (walking on balance beams), static balance (single-leg stance on forceplate and flamingo test) and strength (long jump and handgrip). the results showed that the intervention group performed significantly better in the dynamic balance and flamingo test. there were no significant differences in the centre of pressure (cop) displacement in the medio-lateral direction (cop/ml), the anterio-posterior (cop/ap), and the strength variables. the reliability of the single-leg test on the forceplate was moderate. moreover, there were no correlations between balance and muscle strength variables. it can be concluded that the balance training program with action songs constitutes an effective activity for developing preschool children’s balance, but not strength. perhaps balance and muscle strength are independent of each other and may have to be trained with complementary activities. moreover, the results of the study and the behaviour of the children during the one-leg stance test on the forceplate gave rise to questions regarding the appropriateness of this test for preschool children. balance is fundamental in order to safely perform basic daily movements and sport activities. moreover, the risk of falls-related injuries is particularly high in young children, because their postural control system is not fully developed (altınkök, 2015). recent studies show that balance control develops not only as children grow but also from interacting keywords: postural control, muscular strength, physical education, preschool children, action songs received : 6 october 2022 revised : 26 december 2022 accepted : 29 december 2022 doi : 10.26822/iejee.2023.287 a,* corresponding author: dimitris chatzopoulos, school of physical education and sport science, aristotle university of thessaloniki, greece. e-mail: chatzop@phed.auth.gr orcid: https://orcid.org/0000-0001-5069-2613 b panagiotis varsamis, department of educational & social policy, university of macedonia, thessaloiniki, greece. e-mail: varsamis@uom.edu.gr orcid: https://orcid.org/0000-0001-9657-2684 c panagiotis papadopoulos, school of physical education and sport science, aristotle university of thessaloniki, greece. e-mail: ppan@phed.auth.gr d athanasios giannakos, school of physical education and sport science, aristotle university of thessaloniki, greece. e-mail: ayiannak@phed.auth.gr e georgios lykesas, school of physical education and sport science, aristotle university of thessaloniki, greece. e-mail: glykesas@phed.auth.gr orcid: https://orcid.org/0000-0002-1964-9075 f sophia d. papadopoulou, school of physical education and sport science, aristotle university of thessaloniki, greece. e-mail: sophpapa@phed.auth.gr orcid: https://orcid.org/0000-0003-1526-6429 162 december 2022, volume 15, issue 2, 161-170 with their environment (chatzopoulos, 2019). if a child does not receive the appropriate stimuli for balance improvement, this may result in limited efficiency in performing fundamental movement skills (lykesas et al., 2020) and daily activities (murphy et al., 2003). balance control is important for sport performance and past studies investigated the effectiveness of balance training programmes on postural control in healthy trained children, adolescents and adults (gebel et al., 2018; wälchli et al., 2018). the common feature of these programs is the inclusion of many repetitions of the same task (e.g. stand still for 20s, 3 times), many sets (3-5), rest periods (e.g. 30-120 s) between repetitions or sets, and competitive games (abuín-porras et al., 2020; gebel et al., 2020; granacher et al., 2010; wälchli et al., 2018). due to their intense training orientation these programs may be effective for athletes or older adults, but they are not attractive to young children. for instance, granacher et al. (2011) reported that their balance programme was not efficient in improving the balance of elementary school children, and attributed the inefficiency of their programme to the lack of children’s interest in participating in highly structured exercises. their training programme included a 10 min warm-up, 45 min balance activities (4 sets of 20 sec of each exercise with 2 min rest between the exercises) and 5 min cooldown (total duration 60 min). apparently, it is difficult for children to remain concentrated for so long (one hour) in a training program that involves many repetitions, sets and rests. highly structured programs may be suitable for adults and athletes; however, they do not meet the needs of young children. the average concentration span of pre-schoolers is less than 15 min and depends on their interest in the task and the challenges (mahone & schneider, 2012). balance exercises require high levels of concentration and preschool children are overloaded when the duration of assigned structured motor programs is over 30 min (hastie et al., 2018). moreover, kannass et al. (2006), reported that the average duration of young children’s participation in the same motor activity is approximately 3 min. therefore, a balance programme where the duration of each station lasts more than 3 min is difficult to maintain the concentration of young children in high levels. action songs combine music and movement and encourage children to move by following the directions in a song or by making movements to dramatize the lyrics. moreover, action songs offer children the opportunity for personal expression and increase their motivation for participation in physical activities (chatzihidiroglou et al., 2018; sumantri et al., 2021). in the pretend play environment of action songs children can make mistakes and try out new challenging activities without the anxiety for performance outcomes. therefore, an appropriate selection of action songs with balance requirements could be an attractive approach for improving young children’s postural control. the first aim of the present study was to investigate the effects of a balance programme using action songs on children’s postural control. the execution of balance activities requires muscle strength, and recent studies reported that balance training (bt) has the potential to improve strength parameters in adults (participants age up to 40 years) (gruber et al. 2007; zech et al. 2010) and older adults (60 years and older) (shim et al. 2018). regarding youth athletes (10–18-year-old soccer players), hammami et al. (2016) reported medium correlation between balance (stork test) and strength (standing long jump) in 16–18-year-old athletes and low correlation in 10-11-year-old athletes. moreover, gebel et al. (2020), reported that the correlation between balance and muscle strength in youth athletes is low and it increases with maturity. however, these studies provide only a correlational link between balance level and strength rather than a causal effect that balance training improves strength. for example, the correlation between balance and strength in athletes could be attributed to their training content. regarding young elementary non-athlete children (first grades of elementary school), the findings of a few studies investigating the impact of bt on the muscle strength are controversial. wälchli et al. (2018) and schedler et al. (2020) reported strength improvements, whereas, granacher et al. (2011) reported no effects of bt on strength. to our knowledge, no study has investigated the effects of bt on strength performance in preschool children. it is well established that intersegmental coordination is not fully developed in preschool children (4-5 years old) and that they activate more muscle groups for postural adjustments compared to elementary school children (assaiante et al., 2005). specifically, preschool children increase the head-trunk stiffness in order to maintain balance control, which suggests an “en bloc” operation of the head-trunk unit. moreover, preschool children show higher activation levels of hip and knee muscles for balance control compared to elementary school children (assaiante et al., 2000). therefore, the hypothesis of the present study was that balance training programmes may be effective in improving muscle strength in preschool children because the level of their muscle activation is higher compared to that of elementary school children and adolescents (assaiante et al., 2005). previous research examining the effects of balance programs on children has used expensive equipment (e.g. air cushions, bosu) and can hardly be implemented in physical education preschool settings (wälchli et al., 163 effects of balance training / chatzopoulos, varsamis, papadopoulos, giannakos, lykesas & papadopoulou 2018). moreover, typical balance training programs for older children and adults include many repetitions of the same exercises and are not appealing to preschool children. on the other hand, preschool children are involved in action songs with great enthusiasm. if a child is having fun and enjoying participation in an activity (e.g. action songs), it is more likely to repeat the balance activities and thereby increase balance control. therefore, the aim of the present study was to investigate the effects of a bt program using action songs on postural control and muscle strength in preschool children. methods participants according to g*power the minimum required sample size is 34 participants (effect size f = .25, alpha = .05, and power = .80) (faul et al., 2007). the sample of the current study comprised 51 preschool healthy children (24 boys, 27 girls). the children were recruited from 2 volunteering preschool centre and were randomly assigned to the intervention (25 children, 12 boys and 13 girls), and the control group (26 children, 12 boys and 14 girls). the eligibility criteria were a) the children had no physical condition affecting their participation in physical education and b) they had no diagnosed disorder of cognition (assessed by teacher questionnaire). the characteristics of the children are presented in table 1. informed consent was obtained from the guardians of the children, and they could withdraw from the study at any time. the study was conducted in accordance with the ethical guidelines of the local university and all procedures followed the latest version of the declaration of helsinki. procedures the intervention group followed a 6-week balance training program using action songs (two times a week, total 12 sessions, 30 min/session). each activity in the action songs (i.e. movement steps and poses of the choreography) had three progression levels (table 2). the intensity of the exercises was progressively increased (a) by reducing the base of support, (b) by using dynamic arm/leg movements to perturb the centre of mass, and (c) by using different surfaces (hard surface, soft mat). progression within the program was done with safety in mind and great attention was given to develop children’s understanding of safe practices. in order to increase the motivation of the children they could perform the activities in their own way and instructions were given with analogies (chatzopoulos et al., 2020). the tempo of the action songs was 120-130 beats/min, which is close to the preferred tempo of preschool children and the meter simple duple time of 4/4 table 1. characteristics of the children intervention control age (years) 4.82±.59 4.90±.49 weight (kg) 20.03±3.48 20.21±3.04 height (cm) 109.32±4.29 110.11±4.96 table 2. progressive balance program balance exercise level 1 reducing the base of support level 2 displacement of the centre of mass level 3 surface two leg standing balance activities double-leg balance, upright stance (variations including wide stance, narrow stance, semitandem, and tandem). dynamic arm movements (e.g. rise right hand, rise left hand, rise both hands and down). forward and backward arms sway, lateral sway (side to side). body sways (forward/backward, sideways). hard surface soft mat half foam roll single-leg balance activities on toes, heels. dynamic leg movements. combinations of arm/leg movements. 164 december 2022, volume 15, issue 2, 161-170 (chatzihidiroglou et al., 2018; chatzopoulos et al., 2018). the movement patterns included: step touch, step knee, leg curl, step kick (front, side), plieu touch, turn, toe tap, heel touch, lunges (forward, sideways) and heel-toe walk. for example the song “hokey pokey”: “you put the right foot in (toe tap front), you put your right foot out (toe tap back) and shake it all about (one leg standing and shake the other)….”. the combinations of the movement patterns were structured in 32 count blocks. the program included the following action songs: • a puppet said to me… • i’m a stork. • hokey pokey. • do the bear walk (https://www.youtube. com/watch?v=kg3ao6lj4bq). • balance on one foot (https://www. youtube.com/watch?v=aq2vco_giie). • balancing song for kids (https://www. y o u t u b e . c o m / w a t c h ? v = vc i l d q z t _ w&list=rdvcildqzt_-w&start_radio=1). • hop a little jump (https://www.youtube. com/watch?v=tmxs7rxx-xo). the intervention group was taught by a physical education (pe) teacher with a preschool teacher certification. during the 6-week intervention period, the control group was taught by their teacher and attended the regular pe lessons with fundamental movement skills (locomotor movements and object control with no specific balance exercises) (kapodistria et al., 2021). all study procedures took place between 10:00–12:00 hr. measures the tests were administered prior to and after the intervention in a quiet room at the preschool center. every child was tested individually, under the same conditions. the physical education post graduate students responsible for administering the tests were blinded to group allocation. single leg stance on forceplate the children performed one practice trial and two 10 sec trials of the one-leg stance on a force platform (kinvent, www.k-invent.com, sampling rate 75 hz), with a rest of 30 sec in between. the dependent variables were peak-to-peak amplitude of center of pressure (cop) in the anterior-posterior (cop/ap) and medial-lateral (cop/ml) direction. the mean of the two trials was used for analysis. during the measurement some children could not perceive the point of the task and after a few seconds, they lost their concentration and started to move/laugh or look at the examiner. for this reason, we decided to use a second test for static balance the flamingo test. the reliability of the test is presented in the results section and is further discussed in flamingo test the children stand on a beam (5 cm high, 3 cm wide), on their preferred leg for 60 sec. each contact of the free leg with the ground is recorded as a penalty point. the measurement is stopped at 30 points. for data analysis was used the sum of the points. the lower the number of penalty points the better the performance of the child. the reliability of the test is r≥.70 (bös et al., 2004) dynamic balance dynamic balance was assessed using the test “walking on balance beams” (6, 4.5, and 3 cm wide, 3 m long) (krombholz, 2018). after a practice trial, the children started the test with the 6 cm beam. the number of steps on each beam without losing balance (feet touches the floor) was recorded. the maximum number of steps on each beam is 8, and the maximum score 24 (8 steps x 3 beams). in a pilot study with 18 preschool children the test-retest reliability was very high (r = .75). standing long jump the muscle strength of the lower limbs was assessed using the standing long jump test. the children were instructed from a standing position to jump as far as possible. the best performance of two trials was used for statistical analysis. the test is reliable in 4-6 year old children (test-retest reliability r = .68), (wick et al., 2021). handgrip strength the handgrip strength test was used for upper-limb muscular strength (acquisition frequency 75 hz, kinvent, www.k-invent.com). the children squeezed the dynamometer at maximum effort for 3 sec. the best performance of two trials was used for analysis. the test is reliable in 4-6 year old children (icc=.91) (wick et al., 2021). statistical analyses the data was analysed using a mixed anova with the between-subject factor “group” (intervention vs. control), and the within-subject factor “time” (pre vs. post-testing). homogeneity of variances (levene) tests were conducted for all dependent variables. in case the assumption of sphericity was violated, the greenhouse-geisser correction was used. in the case of significant interaction, follow up t-tests were conducted to identify significant differences. the correlation between the variables and intrasession reliability was assessed using pearson’s r 165 effects of balance training / chatzopoulos, varsamis, papadopoulos, giannakos, lykesas & papadopoulou correlation coefficient. moreover, effect sizes of anova are presented as partial eta square and for t tests as cohen's d values. all statistical analyses were conducted using spss version 22 (ibm corporation). statistical significance was set at p ≤. 05. results mean and standard deviation of the variables are presented in table 3. abbreviations: *significant difference between the two groups (p < .05), dynamic balance (db), centre of pressure (cop) displacement in mediolateral (cop/ml) and anterioposterior direction (cop/ap). dynamic balance mixed anova demonstrated a significant interaction effect between group (intervention vs. control) and time of measurement (pre vs. post-testing) (f 1,49 = 4.59, p =.03, ηp2=.08). at post-testing, follow up t-tests showed that the intervention group improved significantly compared to control (t = 2.10, p = .04, d = .59). one leg stance on forceplate center of pressure (cop) in the medio/lateral (cop/ml) no significant interaction effect was observed between time of measurement and group (f 1,49 = 3.79, p = .06, η p 2 = .07). moreover, there were no statistically significant differences between the groups neither at pretesting (t = 1.32, p = .19, d = .37) nor at postintervention measurement (t = .41, p =.68, d = .11). the intra-session reliability was moderate (r = .53, p < .001). center of pressure (cop) in the anterior/posterior direction (cop/ap). there was no statistically significant interaction (f 1,49 = 3.26, p = .08, η p 2=.06). follow up t-tests showed no significant differences between the two groups neither at preintervention (t = .84, p = .40, d = .23) nor at postintervention testing (t = .77, p = .44, d = .21). the intra-session reliability was moderate (r = .598, < .001). flamingo test mixed anova showed a significant interaction effect between group and time of measurement (f 1,49 = 15.96, p = .001, η p 2=.24). at pretesting, there were no differences between the groups (t = 1.27, p = .20, d = .35). at the post-testing, the intervention group performed significantly better in comparison to control (t = 2.09, p = .04, d = .60). the intra-session reliability was high (pearson’s r = .702, p < .001). standing long jump distance there was no significant interaction effect (f 1,49 = .69, p = .69, η p 2 = .41), and follow up t-tests showed no significant group differences neither at pretesting (t = .94, p = .34, d = .26) nor at postintervention testing (t = .68, p = .09, d = .47). the intra-session reliability was high (r = .917, p < .001). hand-grip strength according to repeated measures anova there was no significant interaction (f 1,49 = .02, p =. 87, η p 2 = .001). moreover, no significant group differences were observed neither at the beginning (t = 1.57, p = .12, d = .44) nor at postintervention measurement (t = 1.60, p = .11, d = .44). the intra-session reliability was strong (r = .92, p < .001). correlations table 4 summarizes the correlations between the variables (pearson’s r). there was no correlation between balance (dynamic or static) and strength (jump or handgrip). jump and handgrip showed a moderate correlation (r = .517). table 3. mean and sd of the variables in preand post-tests intervention group control group pre post pre post db (steps) 11.56 ±8.06 17.56 ±9.52* 10.19 ±6.29 12.26 ±8.36 cop/ml (mm) 30.16 ±7.47 26.86 ±5.26 27.61 ±6.27 27.47 ±5.21 cop/ap (mm) 43.89 ±16.06 40.45 ±12.99 41.09 ±13.63 44.62 ±18.31 flamingo (contacts) 20.36 ±6.64 13.82 ±6.76* 18.30 ±4.67 17.28 ±4.96 jump (cm) 70.22 ±20.49 73.76 ±18.31 65.38 ±15.82 66.09 ±13.92 handgrip (kg) 5.55 ±1.44 5.73 ±1.38 4.95 ±1.25 5.17 ±1.12 166 december 2022, volume 15, issue 2, 161-170 table 4. correlation matrix between dynamic balance (db), flamingo test, centre of pressure (cop) displacement in the mediolateral (cop/ml) and anterioposterior direction (cop/ap), jump and grip flamingo cop/ml cop/ap jump grip db -.545** -.377** -.276* -.074 .171 flamingo .351* .230 .098 .138 cop/ml .798** .178 .188 cop/ap .117 .183 jump .517** abbreviations: pearson's correlation *p < .05, **p < .01. discussion the purpose of the present study was to investigate the effects of bt using action songs on the balance and strength of preschool children. six weeks of bt resulted in improvements in static and dynamic balance in 4 to 6-year-old children but not in strength. moreover, no correlations were observed between postural control and muscle strength. given that no study has been done to examine the impact of bt using action songs on balance and muscle strength in preschool children in a school setting, the findings of the present study have to be compared with those few studies with conventional balance training programs, i.e. with specific balance equipment and competitive activities (wälchli et al., 2018). shim et al. (2021), reported that 15-20 minutes of continuous movement of pedal-less bicycle riding (4 weeks duration, 3 times a week), resulted in postural control improvements in preschool children (aged 3-5-years old). the advantage of the present study compared to shim et al. (2021) is that no particular equipment is required for the balance training program. moreover, specific balance equipment like bicycles or other unstable training devices (e.g. air cushions, ankle disks) not only are expensive, but also for safety reasons require a ratio of one teacher to one preschool child during the training, which is impossible within a regular preschool physical education setting. although past literature seems to agree that bt is effective in promoting postural control in adolescents and adults, there are conflicting results regarding the effects of bt on pre-adolescent children (granacher et al., 2011; wälchli et al., 2018). granacher et al. (2011), reported that 4 weeks (3 times a week) of bt using unstable support surfaces with a predefined number of sets and durations (i.e. typical bt) did not improve balance parameters of elementary school children. the authors attributed the insignificant improvements to the maturational deficits in the postural control system of prepubertal children, and speculated that prepubertal children lacked important neurophysiologic prerequisites for adaptive processes after bt. on the contrary, wälchli et al. (2018) reported balance improvements in elementary school children after a bt with unstable devices (e.g. pedalo) and competitive balance games. similar balance improvements in 10-year-old children were reported by donath et al. (2013) after slackline training and muehlbauer et al. (2013) after inline skating (11 years old). it seems that apart from children’s maturation stage, the content of the bt may greatly influence the outcome in postural control. highly structured balance training regimens with many repetitions and rest times are not appealing to young children. especially preschool children lose their interest in highly structured activities quickly, due to their limited attention span (oliveira et al., 2019). therefore, for future studies with preschool children great caution should be given to the attractiveness of the intervention content. moreover, for future research the investigation of children’s interest for balance activities would be particularly enlightening. in the present study, balance training using action songs had no effect on children’s strength. since this study is the first that examined the effects of balance training on preschool children’s muscle strength, the findings are discussed in relation to studies with cohorts of older children. a few recent studies on older children (8-12 years) showed that balance training with specific devices generated significant increases in strength (wälchli et al., 2018). kocjan et al. (2020) reported that unicycling riding proved to be an effective tool to develop strength in 12-year-old children (six weeks, 12 sessions, 2 training sessions per week, 45 min per-session). in addition, muehlbauer et al. (2013) demonstrated significant strength improvements (jump height) after a 4-week inline skating program (2 times/week, 90 min. each). the divergent findings of these studies compared to our study could be attributed to the different training content of the studies. learning to ride a unicycle (or inline skating) and jumping with it around the marks requires much more strength resources than standing on one leg and moving arms/legs as in the present study. apparently, different balance exercises have different effects on strength. another reason for the insignificant strength results in the present study could be the maturation stage of the cohort in combination with the duration of the intervention. it is well established that interand intramuscular coordination (i.e. neural parameters, that are decisive for strength improvements), is not fully developed in preschool children (assaiante et al. 2005). moreover, wick et al. (2021) demonstrated muscle strength improvement (in standing long jump performance) in preschool children after 30 sessions of a strength-dominated exercise program involving countermovement jumps, lunges, single leg jumps, plank and table position. similar improvements in muscle strength in preschool 167 effects of balance training / chatzopoulos, varsamis, papadopoulos, giannakos, lykesas & papadopoulou children were reported by andrieieva et al. (2021) after a fitness program in a sport club that lasted 9 months. perhaps due to the low maturation stage of preschool children, a longer intervention duration than the 12 sessions applied in the present study, is needed for strength improvements. further research is necessary to elucidate the strength issue after balance training in terms of exercise content and intervention duration. according to the results of the present study there were no correlations between strength and balance variables. to the authors’ knowledge there are no other studies available that investigated the relation between balance and muscle strength variables in preschool children. thus, the findings are discussed in relation to studies with older age groups. in contrast to our results gafner et al. (2018) reported associations between muscle strength (hip muscle and hand-grip strength) and postural control in older participants (>65 years old). moreover, jeon et al. (2022) demonstrated that the muscle strength of knee extensors was positively related to the star excursion balance test (sebt) in young adults. the controversy results between the aforementioned studies and the present study could be attributed to the different measurements applied in the studies. in the study of jeon et al. (2022) was applied the sebt, whereas in the present study the one-legged stance test. in the sebt the participant is required to maintain standing with one leg while maximally reaching in several directions with the other leg. the more one participant has strong lower limb muscles and can flex the hip and knee of the standing leg the longer he/she can reach with the opposite leg. therefore, the muscle strength of the stance-leg is important for maximum reaching distance in the sebt and it is plausible to find correlations between muscle strength and sebt (filipa et al., 2010). the lack of correlation between muscle strength and balance variables of the present study are consistent with the results of granacher et al. (2011), who reported no association between one-legged stance and countermovement jump in adolescents participants. moreover, mccurdy et al. (2006) demonstrated no relationship between one-legged stork stand and squat strength in young adult participants (mean age 22 ± 2 years). our data agree with those of granacher et al. (2011) and mccurdy et al. (2006) and extend their findings by adding an additional age group (preschool children) to the existing knowledge on adolescents and adults. based on these results it can be argued that muscle strength and postural control are independent factors that may have to be trained with specific activities. this speculation could be reinforced by the results of andrieieva et al (2021), who reported balance and muscle strength improvements in preschool children after a fitness program of nine months. the single-leg stance test on the forceplate is one of the most frequently used tests in the literature for postural control evaluation (zumbrunn et al., 2011). however, we observed that this task was not appealing to young children. they could not comprehend the point of the task, and after a few seconds they lost their concentration and started to play (e.g. hopping, looking at the evaluator, laughing). the medium reliability of cop/ml and cop/am (r = .530 and r = .598 respectively) could be attributed to the children's lack of interest in performing the task. on the other hand, the reliability coefficient in the flamingo test was very high (r = .702). based on our experience, our impression is that the flamingo test was much more interesting than the single-leg stance test. therefore, for future studies with young children, we suggest the implementation of attractive tests where children can evaluate their own performance. limitation a limitation of the present study relates to the concentration of young children during the measurements. although the tests were conducted in a quiet room and were presented like games, we cannot be sure about the level of children’s concentration and that they really gave their best. conclusion the results of the present study demonstrated that a short-term balance training program (12 lessons) using action songs produces significant balance improvements in preschool children. however, it seems that this short period is not sufficient to induce significant muscle strength improvements. moreover, there were no correlations between balance and muscle strength variables in preschool children. this finding could be an indication that these variables are independent of each other and maybe they should be trained with complementary activities in preschool children. based on the findings, the implementation of action songs with balance requirements in the regular preschool curriculum is a safe and feasible training modality for the improvement of postural control of young children. acknowledgements we would like to thank the schools, children and parents involved in the study. references abuín-porras, v., jiménez antona, c., blancomorales, m., palacios, a., romero-morales, c., lópez-lópez, d., villafañe, j. h., & rodríguezcosta, i. (2020). could a multitask balance training program complement the balance training in healthy preschool children: a quasiexperimental study. applied sciences, 10(12), 4100. https://doi.org/10.3390/app10124100 168 december 2022, volume 15, issue 2, 161-170 altınkök, m. (2015). examining the effects of “activity education with coordination” on the development of balance and arm power in 6-year-old primary school children international online journal of educational sciences, 7(4), 140–147. andrieieva, o., kashuba, v., yarmak, o., cheverda, a., dobrodub, e., & zakharina, a. (2021). efficiency of children's fitness training program with elements of sport dances in improving balance, strength and posture. journal of physical education and sport, 21, 2872–2879. https://efsupit.ro/images/ stories/octombrie2021/art%20382.pdf assaiante, c., mallau, s., viel, s., jover, m., & schmitz, c. (2005). development of postural control in healthy children: a functional approach. neural plasticity, 12(2-3), 109–118. assaiante, c., woollacott, m., & amblard, b. (2000). development of postural adjustment during gait initiation: kinematic and emg analysis. journal of motor behavior, 32(3), 211–226. https:// doi.org/10.1080/00222890009601373 bös, k., bappert, s., tttlbach, s., & woll, a. (2004). karlsruher motorik-screening für kindergartenkinder (kms 3-6). (karlsruhe's motoric screening for nursery school children (kms 3-6), trans.). sportunterricht, 53(3), 79–87. chatzihidiroglou, p., chatzopoulos, d., lykesas, g., & doganis, g. (2018). dancing effects on preschoolers' sensorimotor synchronization, balance, and movement reaction time. perceptual and motor skills, 125(3), 463–477. https://doi.org/10.1177/0031512518765545 chatzopoulos, d. (2019). effects of ballet training on proprioception, balance, and rhythmic synchronization of young children. journal of exercise physiologyonline, 22(2), 26–37. chatzopoulos, d., doganis, g., & kollias, i. (2018). effects of creative dance on proprioception, rhythm and balance of preschool children. early child development and care, 13, 1–11. https://doi.org/ 10.1080/03004430.2017.1423484 chatzopoulos, d., foka, e., doganis, g., lykesas, g., & nikodelis, t. (2020). effects of analogy learning on locomotor skills and balance of preschool children. early child development and care, 1–9. https://doi.org/10.1080/03004430.2020.1739 029 donath, l., roth, r., rueegge, a., groppa, m., zahner, l [l.], & faude, o. (2013). effects of slackline training on balance, jump performance & muscle activity in young children. international journal of sports medicine, 34(12), 1093–1098. https://doi. org/10.1055/s-0033-1337949 faul, f., erdfelder, e., lang, a.-g., & buchner, a. (2007). g*power 3: a flexible statistical power analysis program for the social, behavioral, and biomedical sciences. behavior research methods, 39, 175–191. filipa, a., byrnes, r., paterno, m. v., myer, g. d., & hewett, t. e. (2010). neuromuscular training improves performance on the star excursion balance test in young female athletes. the journal of orthopaedic and sports physical therapy, 40(9), 551–558. https://doi.org/10.2519/ jospt.2010.3325 gafner, s. c., bastiaenen, c. h., ferrari, s., gold, g., terrier, p., hilfiker, r., & allet, l. (2018). hip muscle and hand-grip strength to differentiate between older fallers and non-fallers: a crosssectional validity study. clinical interventions in aging, 13, 1–8. https://doi.org/10.2147/cia. s146834 gebel, a., lesinski, m., behm, d. g., & granacher, u. (2018). effects and dose-response relationship of balance training on balance performance in youth: a systematic review and meta-analysis. sports medicine (auckland, n.z.), 48(9), 2067– 2089. https://doi.org/10.1007/s40279-018-0926-0 gebel, a., prieske, o., behm, d. g., & granacher, u. (2020). effects of balance training on physical fitness in youth and young athletes: a narrative review. strength & conditioning journal, 42(6), 35–44. https://doi.org/10.1519/ssc.0000000000000548 granacher, u., & gollhofer, a. (2011). is there an association between variables of postural control and strength in adolescents? journal of strength and conditioning research, 25(6), 1718–1725. https://doi.org/10.1519/ jsc.0b013e3181dbdb08 granacher, u., gollhofer, a., & kriemler, s. (2010). effects of balance training on postural sway, leg extensor strength, and jumping height in adolescents. research quarterly for exercise and sport, 81(3), 245–251. https://doi.org/10.1080 /02701367.2010.10599672 169 effects of balance training / chatzopoulos, varsamis, papadopoulos, giannakos, lykesas & papadopoulou granacher, u., muehlbauer, t., maestrini, l., zahner, l [lukas], & gollhofer, a. (2011). can balance training promote balance and strength in prepubertal children? journal of strength and conditioning research, 25(6), 1759–1766. https:// doi.org/10.1519/jsc.0b013e3181da7886 hammami, r., chaouachi, a., makhlouf, i., granacher, u., & behm, d. g. (2016). associations between balance and muscle strength, power performance in male youth athletes of different maturity status. pediatric exercise science, 28(4), 521–534. https://doi.org/10.1123/pes.20150231 hastie, p. a., johnson, j. l., & rudisill, m. e. (2018). an analysis of the attraction and holding power of motor skill stations used in a mastery motivational physical education climate for preschool children. physical education and sport pedagogy, 23(1), 37–53. https://doi.org/10. 1080/17408989.2017.1341476 jeon, j., lee, j., hong, j., yu, j., kim, j., & lee, d. (2022). correlation of eccentric strength of the knee extensors and knee proprioception with dynamic postural control. journal of back and musculoskeletal rehabilitation, 35(2), 309–316. https://doi.org/10.3233/bmr-200106 kannass, k. n., oakes, l. m., & shaddy, d. j. (2006). a longitudinal investigation of the development of attention and distractibility. journal of cognition and development, 7(3), 381–409. https://doi.org/10.1207/s15327647jcd0703_8 kapodistria, l., chatzopoulos, d., chomoriti, k., lykesas, g., & lola, a. (2021). effects of a greek traditional dance program on sensorimotor synchronization and response time of young children. international electronic journal of elementary education, 14(1), 1–8. https://doi. org/10.26822/iejee.2021.224 kocjan, a., & sarabon, n. (2020). the effect of unicycle riding course on trunk strength and trunk stability functions in children. journal of strength and conditioning research, 34(12), 3560–3568. https://doi.org/10.1519/jsc.0000000000002151 krombholz, h. (2018). development of motor talents and nontalents in preschool age – an exploratory study. cogent psychology, 5(1), 395. https://doi. org/10.1080/23311908.2018.1434059 lykesas, g., giossos, i., chatzopoulos, d., koutsouba, m., douka, s., & nikolaki, e. (2020). effects of several warm-up protocols (static, dynamic, no stretching, greek traditional dance) on motor skill performance in primary school students. international electronic journal of elementary education, 12(5), 481–487. https://www.iejee. com/index.php/iejee/article/view/1155 mahone, e. m., & schneider, h. e. (2012). assessment of attention in preschoolers. neuropsychology review, 22(4), 361–383. https://doi.org/10.1007/ s11065-012-9217-y mccurdy, k., & langford, g. (2006). the relationship between maximum unilateral squat strength and balance in young adult men and women. journal of sports science & medicine, 2(5). https://doi.org/10.5194/gi-2016-11-rc2 muehlbauer, t., kuehnen, m., & granacher, u. (2013). inline skating for balance and strength promotion in children during physical education. perceptual and motor skills, 117(3), 665–681. https://doi.org/10.2466/30.06.pms.117x29z9 murphy, d. f., connolly, d. a. j., & beynnon, b. d. (2003). risk factors for lower extremity injury: a review of the literature. british journal of sports medicine, 37(1), 13–29. https://doi.org/10.1136/bjsm.37.1.13 oliveira, j. a. de, rigoli, d., kane, r., mclaren, s., goulardins, j. b., straker, l. m., dender, a., rooney, r., & piek, j. p. (2019). does 'animal fun' improve aiming and catching, and balance skills in young children? research in developmental disabilities, 84, 122–130. https://doi.org/10.1016/j. ridd.2018.07.004 schedler, s., brock, k., fleischhauer, f., kiss, r., & muehlbauer, t. (2020). effects of balance training on balance performance in youth: are there age differences? research quarterly for exercise and sport, 91(3), 405–414. https://doi.or g/10.1080/02701367.2019.1676371 shim, a., davis, w., newman, d., abbey, b., & garafalopeterson, j. (2021). the effects of a pedal-less bicycle intervention on stability scores among preschool aged children. journal of motor behavior, 53(2), 185–190. https://doi.org/10.1080/ 00222895.2020.1748859 sumantri, m. s., rahmawati, y., jalal, f., yetti, e., & syarah, e. s. (2021). dancing is thinking for children: working memory, inhibition, and mental flexibility. international electronic journal of elementary education, 14(2), 135–146. https://doi.org/10.26822/iejee.2022.234 170 december 2022, volume 15, issue 2, 161-170 wälchli, m., ruffieux, j., mouthon, a., keller, m., & taube, w. (2018). is young age a limiting factor when training balance? effects of childoriented balance training in children and adolescents. pediatric exercise science, 30(1), 176–184. https://doi.org/10.1123/pes.2017-0061 wick, k., kriemler, s., & granacher, u. (2021). effects of a strength-dominated exercise program on physical fitness and cognitive performance in preschool children. journal of strength and conditioning research, 35(4), 983–990. https:// doi.org/10.1519/jsc.0000000000003942 zumbrunn, t., macwilliams, b. a., & johnson, b. a. (2011). evaluation of a single leg stance balance test in children. gait & posture, 34(2), 174–177. https:// doi.org/10.1016/j.gaitpost.2011.04.005 international electronic journal of elementary education, 9(3), 667-680, march, 2017 issn:1307-9298 copyright © iejee www.iejee.com the effect of simulation on middle school students’ perceptions of classroom activities and their foreign language achievement: a mixed-methods approach akram sharifi a afsaneh ghanizadeh a  safoura jahedizadeh b a imam reza international university, iran b ferdowsi university of mashhad, iran received: 21 november 2016 / revised: 11 january 2017 / accepted: 8 february 2017 abstract the present study delved into a language learning model in the domain of english as a foreign language (efl), i.e., simulation. the term simulation is used to describe the activity of producing conditions which are similar to real ones. we hypothesized that simulation plays a role in middle school students’ perceptions of classroom activities (i.e., interest, challenge, choice, and joy). it was also conjectured that simulation affects foreign language achievement. to do so, the study utilized an experimental design consisting of 51 female participants (25 learners in the control group and 26 students in the experimental group). the results demonstrated the positive role of simulation in students’ perceptions of classroom activities and their language achievement. a semistructured interview was also held at the end of the study with a number of students in experimental group to investigate student’s attitudes and emotional reactions towards simulation. keywords: simulation, perceptions of classroom activities, language achievement, mixed-methods approach. introduction regarding the historical perspective of simulation and its success in english language teaching (elt), it was originally used as a learning technique in military training and business (sam, 1990). simulation can be defined as the reality of function in a simulated environment (jones, 1986). in other words, simulation is a structured set of circumstances which mirror real life situations (sam, 1990). it is the act of stimulating the behavior of a situation or a process by the use of a suitably analogous phenomenon. simulation is a teaching technique in which the behavior is not controlled and participants can bring their own experience, knowledge, and skills to the situation and consequently enrich the learning process, change the academic setting to a real life situation, and provide an effective and efficient language learning experience. simulation can also be considered as a problemsolving activity to which learners bring their own distinct opinions, feelings, and  corresponding author: afsaneh ghanizadeh, assistant professor of tefl. imam reza international university, mashhad, iran. phone: +9153048141 e-mail: a.ghanizadeh@imamreza.ac.ir http://www.iejee.com/ international electronic journal of elementary education, 9(3), 667-680, march, 2017 668 personalities (livingstone, 1983). hyland (2009) has also declared that, “a simulation is a problem driven event that occurs in a clearly described realistic situation” (p. 10). teachers can use simulations as an ideal technique to use language communicatively and creatively (hyland, 2009). there are various kinds of simulations used in different fields of knowledge (e.g., marine simulations, the medical simulations, flight simulations, military simulations, computer and educational simulations). the training and educational simulations in turn fall in three categories: a) live simulations in which real people use simulated equipment in the real world, b) virtual simulations in which real people use simulated equipment in a virtual environment or a simulated world, or c) constructive simulation in which simulated people use simulated equipment in a simulated environment. although simulations share a lot in common with role-plays, there are major differences between the two techniques (lyu, 2006). in simulations, for instance, the necessary facts are provided to the participants for the functional part such as their age, job, gender, etc., while in role plays, participants have to invent facts or act out scenes based on a provided specific description or script. moreover, participants in simulations take on functions, responsibilities, and duties according to their own preferences and personalities, instead of acting or playing the role. in addition to these differences, imagination may be involved in simulation, however avoidance of invention of key facts is a must. consequently, as bambrough (1994) points out, simulations are different from other role activities in which the roles function within a structure and the elements of this structure are represented in a dynamic way. another distinction can be made between simulations and games. considering the similarities, both simulations and games are autonomous. in other words, the participants in a simulation and the players in a game are in charge of their roles within their particular environment (jones, 1986). however, the participants of a game have only one role who has the same duty, they are always players and their duty is to win the game. the difference between the two techniques lies within the degree of reality of function. simulations provide reality (i.e., simulations always present a high degree of reality of function, otherwise they are not simulations) whereas games present little or no reality of function. in this regard, there is no clear-cut division between simulations and games; rather it is a continuum. most of the studies on simulation have been conducted to assess the effectiveness of such a teaching technique and find its benefits related to some skills. such studies prove that using simulation as a kind of language assessment model promotes cross-cultural communication (crookall, coote, dumas, & le gat, 1987; crookall & oxford, 1990) and instrumental motivation by making the coursework more engaging (jones, 1986, p. 10). other studies support the fact that simulation lowers affective barriers to acquisition by reducing the fear of making mistakes (nemitcheva, 1995) and presents real time scenarios and instantaneous feedback (jones, 1986). it also benefits language learners to use language in highly specific contexts (brown, collins, & duguid, 1989) and provides a meaningful way of learning a language (sam, 1990). moreover, simulation promotes metacognitive strategy use (bullard, 1990), improves students’ desires to learn (davis, 1996), and makes learning and teaching process a rewarding experience for both students and teachers (tompkins, 1998). similarly, it guides learners progressively towards the final goal of learning as well as assessment purposes and lets the teacher to play the role of a facilitator who share students’ knowledge and opinions among themselves (albert, 1999). besides such benefits it helps learners to set realistic and communicative goals, assesses the features of a language use situation, and helps students to plan responses and have control on the execution of such plans (ranalli, 2008). it can also provide realistic sociocultural contexts for language learning by bridging classroom activities and foreign language achievement /ghanizadeh, sharifi & jahedizadeh 669 the gap between students and the foreign culture (schwienhorst, 2002) and motivate shy learners to take part in discussions more actively (freiermuth, 2002). in a similar vein, simulation gives learners the opportunity to solve problems without the authoritative persuasion of the teacher by providing a learner-centered context (freiermuth, 2002, p. 187) and provides the content for language learning via meaningful and cohesive contexts (purushotma, 2005, p. 84). it is also capable of enhancing learners’ grammar and vocabulary knowledge (miller & hegelheimer, 2006), as well as causing them to reflect more positive attitudes towards collaboration and providing positive reaction to the modifications among the users (ranalli, 2008). simulation has been also found to be significant in promoting vocabulary skills (ranalli, 2008) and students’ second language vocabulary recall (dehaan, reed, & kuwada, 2010). finally, it makes the teachers more flexible to take students’ individual differences into account (wang, 2010), motivates students by providing challenging opportunities for authentic discussions (dehaan, 2011), improve students’ oral communicative skills (javid, 2013), and promotes english vocabulary and pronunciation learning of esp students (meihami, meihami, & varmaghani, 2013). simulation can have a salient influence on the way students perceive their classroom activities and environment. such perceptions reflect one’s needs, emotions, and expectations. in the domain of education, students’ perceptions of classroom activities consist of four main constructs namely; interest, challenge, choice, and joy (gentry, gable, & rizza, 2002) each of which has a significant association with other student-related issues, such as, student involvement (lee, yin & zhang, 2009); self-regulating learning and motivational beliefs (kharrazi & kareshki, 2010; ghanizadeh & alishahi, 201); students’ goal-orientations (jahedizadeh, ghanizadeh, & ghonsooly, 2016); epistemological beliefs and learning approaches (ozkal, tekkaya, cakiroglu, & sunsur, 2008), and cognitive strategy use (young, 1997). due to the encompassing position of simulation in teaching practices, the present study aimed at delving into the effect of simulation on students’ perceptions of classroom activities. in this study, efl middle school learners were investigated primarily due to the researchers’ professional and educational expertise in the field. furthermore, the effect of simulation on language learning achievement was explored. methodology participants the participants of the present study comprised 51 middle school students (all females) that were divided into two equal groups, i.e., experimental and control. they were learning english at the second grade of secondary school as an obligatory course. they were around 13 and 14 years old. the study used an intact-groups design with 25 participants in the control group and 26 participants in the experimental group. the design was quasi experimental. the participants were selected among efl students learning english in an official context in mashhad, a city in north east of iran. to meet the requirement of experimental research and to ensure that the learners were homogenous in the point of their english proficiency level and their perceptions of classroom activities, a language test and a questionnaire were administered as pretest. instruments the paper version of babel english language placement tests (belpt). the paper version of babel english language placement tests (belpt) was utilized to determine language proficiency. it is closely based on the nelson quick check placement tests. the testing cycle international electronic journal of elementary education, 9(3), 667-680, march, 2017 670 should require no more than 70 minutes of trainee time and does not require any specialist testers to administer it. the tests were designed for the ease of administration in observing stringent test design standards. the tests were in multiple-choice format and consisted of items measuring the recognition of correct responses to reading prompts, grammatical forms and lexical choices in context. students perceptions of classroom activities scale. to determine students' perceptions about their classroom, this study employed 'students perceptions of classroom activities' scale which was designed by gentry and gable (2001) and was translated to persian and validated by ghanizadeh and jahedizadeh (2015). the 'students perceptions of classroom activities' instrument contains 31 statements evaluating four dimensions (interest, challenge, choice, and enjoyment). the scale measures the four dimensions via a 5-point likert-type response format (never, seldom, sometimes, often, and always). the questionnaire provided the participants with directions on how to complete the scale. as reported by gentry, gable, and rizza (2002), the instrument was piloted and a confirmatory study was undertaken for a national sample. in iranian context, the reliability and validity of the scale was confirmed through cfa and cronbach’s alpha estimates (ghanizadeh & jahedizadeh, 2015). sample items for 'interest' dimension included: 1) the teacher involves me in interesting learning activities; and 2) what i do in my class gives me interesting and new ideas. sample items for 'challenge' dimension are: 1) i have to think to solve problems in my class, and 2) what we do in class fits my abilities. sample items included in 'choice' dimension are: 1) when we work together, i can choose my partners, and 2) when there are many jobs, i can choose the ones that suit me. sample items for 'joy' dimension are: 1) the teacher makes learning fun and 2) i like what i do in my class. interview. in order to capture the quality of simulation technique and explore students’ attitudes and reactions towards the technique, a semi-structured interview was held with 6 participants of the experimental group whom were chosen randomly. they were asked to talk about the experience of simulation, whether they liked it, and how it facilitated their learning. the interviews were recorded and analyzed accordingly. procedure the data collection of this study took place in december 2015 and lasted to april 2016. in this study, for collecting the data, the babel test along with the aforementioned questionnaires were distributed among the participants of the two groups at the beginning of the study to ensure they were at the same level of language proficiency and had relatively identical perceptions before the implementation of simulation. to gather reliable data, the purpose of completing the questionnaire was explained and the participants were assured that their views would be confidential. the test and the questionnaire were also administered at the end of the term to examine whether significant differences would be observed in the perceptions and language achievement of students of experimental group after the introduction of simulation. the two classes were conducted by the same teacher and the same materials were used for the instruction. the only difference was that in the experimental group, the conversations were taught through simulation technique. to do so, different scenarios were introduced at the beginning of the term and students were asked to register for each one according to their own interest and preferences. classroom activities and foreign language achievement /ghanizadeh, sharifi & jahedizadeh 671 the researcher searched about the most frequent ambiguous and problematic linguistic areas with which efl learners face in middle schools. it was found out that those problems pertain to areas such as, adverbs of frequency, count and non-count nouns; some and any, possessive nouns; this/ that/ these/ those, comparative adjectives, preposition of time: in, on, at, and imperatives. in accordance with these gaps, the scenarios included themes mostly covered in their books, such as, talking about habits-related to appropriate use of adverbs of frequency--, having the meals --related to recognition of counts and non-count nouns-, telling about clothing – related to the use of some and any--, comparing the music –related to understanding the use of possessive nouns--, reporting leisure activities-directly referred to the use of different kind of comparative adjectives-, and giving direction-dealing with the preposition of time usage. for this aim, the students in the experimental group were asked to work on the topic they selected a week before their presentation. they, then, performed it in the class in the form of a role-play. the students in the control group, nevertheless, worked on the conversion in the traditional mode of memorization and repetition. due to the administration constrains, ten sessions, each about 30 minutes out of the 90 minuteclass time, were devoted to simulation. as an illustration, one session is briefly presented in the followings. it is worthwhile to mention in order to retain the fundamental element of simulation in creating a real situation, students put on costumes and outfits in line with the topic of conversation used and real objects such as microphone, and created real-like scenes, such as street. session 1. according to the students' need, a conversation was selected. four students volunteered to perform the conversation according to their interest. the conversation was going to cover countable and uncountable nouns in the frame of asking about eating habits. they were asked to prepare every material they needed in order to simulate a real situation on the determined session, they generated the simulated conversation. one of the students acting as jessica had a microphone as an interviewer and asked each person what she usually had for breakfast. she said hello to everyone. she saw a teenage girl walking fast and asked: excuse me, lady, do you eat breakfast? the girl stopped and answered: yes, more or less. jassica: what do you have? girl: i generally have a bagel and a cup of tea. jessica: that’s all? do you have any juice or anything else to drink? girl: not usually. once in a while i have coffee instead of tea. i'm always in a hurry. bye jessica: ok, thanks, bye. the interviewer took a look around her and said, “now, here's our next person”. meanwhile, she started her conversation with another student who looked as if she wanted to cross the street with her own child and asked: mam, what do you have for breakfast? woman1: i never eat breakfast. jessica: nothing at all? woman1: no. i'm on a diet. i'm always on a diet. jessica: ok. thank you… international electronic journal of elementary education, 9(3), 667-680, march, 2017 672 at this time, the interviewer faced to the other student as a woman with a shopping bag containing some kind of fruits, vegetables, and so on. then she asked: jessica: and what about you, ma'am? what do you have for breakfast? woman 2: oh, i usually have a bowl of cereal and some yogurt with fruit-a banana, a peach, or an orange, or some strawberries. and i have eggs and toast and a glass of juice. jessica: hmm. that sounds healthy. woman 2: yes, i always eat a good breakfast. jessica: all right, thanks. let's see what our next person says... the team terminated their scenario alongside the other students as the audience applauded. after ending the first scenario conducted by the students in the experimental group, the head of the above team who had selected other members, worked on the pronunciation of some fruits and foods such as a bagel, a cup of tea, some strawberries and a banana, a bag of cereal, a can of yogurt, and some slices of toasts. she also encouraged the use of a, an, any, and some. next, she challenged them to talk about their eating habits, which food they like and which they don’t like. the students answered and used the appropriate forms of the articles before each food. then, she assured whether the students comprehended the parts of the conversation or not, so she wrote some questions about the conversation on the board as follows: 1. the man has …… a. a big breakfast b. a small breakfast c. no breakfast at all 2. he usually drinks ….. a. water b. coffee c. tea 3. the first woman ….. has breakfast. a. never b. rarely c. sometimes 4. jessica thinks the ….has a healthy meal. a. man b. first woman c. second woman the students were actively and enthusiastically involved in the activities. they were occupied with understanding the use of count versus non-count nouns, use of quantifiers, and use of singular verbs with non-count nouns. classroom activities and foreign language achievement /ghanizadeh, sharifi & jahedizadeh 673 interview six students from the experimental group were randomly selected to sit for an interview to talk about their experience of simulating conversations in the class. the interview pursued a similar task to the questionnaires in that it sought students’ opinions on classroom activities in the light of simulation. the distinctive feature of the interview was that it allowed us to obtain more in-depth information from the respondents on the broader context surrounding the process of language learning. they were asked to articulate about the experience of simulation, whether they liked it, and how it facilitated their learning. results the results of pretest on language proficiency to examine whether there is any significant difference between control and experimental group regarding their english proficiency level, an independent samples t-test was run. table 1 below summarizes the descriptive results of english proficiency level measured by the babel test in the two groups. as the table shows, the mean scores of proficiency across participants in control and experimental groups are slightly different: control (m=14.96, sd=3.57), experimental (m=15.19, sd=2.57). table 1. descriptive statistics of english proficiency level across control and experimental groups n mean std. deviation std. error mean pre-test proficiency experimental 26 15.1923 2.95323 .57918 control 25 14.9600 3.57631 .71526 to see whether this observed difference is statistically significant, an independent samples t-test was run. table 2 presents the results of t-test run on english proficiency level. as can be seen, there is not a statistically significant difference between the two groups regarding the degree of their proficiency (t= .253, p<.05). in other words, the two groups are homogenous regarding their english proficiency level before the study. table 2. independent samples t-test showing the results of pretest on english proficiency t df sig. (2-tailed) mean difference std. error difference pre-test proficiency .253 49 .801 .2323 .91687 the results of pretest on perceptions of classroom activities to examine whether there is any significant difference between control and experimental groups regarding their perceptions of classroom activities (interest, challenge, choice, and joy), an independent samples t-test was run on each perception. table 3 below summarizes the descriptive results of perceptions in two groups. as the table shows, the mean scores of perceptions across participants in control and experimental groups are different. international electronic journal of elementary education, 9(3), 667-680, march, 2017 674 table 3. descriptive statistics of perceptions of classroom activities across control and experimental groups in pretest groups n mean std. deviation std. error mean interest exp 26 20.2308 5.87406 1.15200 con 25 17.1600 6.66258 1.33252 challenge exp 26 15.9231 2.71180 .53183 con 25 14.7600 4.05463 .81093 choice exp 26 16.6538 3.24891 .63716 con 25 16.1200 3.35807 .67161 joy exp 26 17.5000 5.20192 1.02018 con 25 15.8000 6.39010 1.27802 to see whether these observed differences are statistically significant, independent samples t-tests were run. table 4 presents the results of t-tests run on perceptions of classroom activities. as it can be seen, there are not statistically significant differences between the two groups regarding their perceptions: interest (t= 1.74, p<.05), challenge (t= 1.20, p<.05), choice (t= .57, p<.05), joy (t= 1.04, p<.05). in other words, the two groups were homogenous regarding their perceptions of classroom activities before the study. table 4. independent samples t-test showing the results of pretest on student’s perceptions of classroom activities t df sig. (2-tailed) mean difference std. error difference interest 1.748 49 .087 3.07077 1.75704 challenge 1.209 49 .233 1.16308 .96238 choice .577 49 .567 .53385 .92515 joy 1.044 49 .302 1.70000 1.62864 the results of posttest on language proficiency to investigate the effect of simulation on students' language proficiency, the differences between the two groups on babel test were calculated in post-test. the means of the both groups in the post-test were shown to be different. as it can be seen in table 5, the mean of the experimental groups (m= 16.6154, sd= 2.5624) is higher than that of control groups (m= 14.6400, sd= 3.7403). table 5. independent samples t-test showing the results of post-test on english proficiency n mean std. deviation std. error mean post-test proficiency experimental 26 16.6154 2.5624 .50254 control 25 14.6400 3.7403 .74806 to investigate whether this observed difference is statistically significant, an independentsamples t-test was run. as table 6 shows, there is a statistically significant difference between experimental and control groups (t= 2.20, p<.05). in other words, it can be implied that experimental group gained higher scores in babel test and this is an indication of the efficiency of simulation. the effect size calculated via cohen's d was found to be 0.61 which is a large value according to cohen's index value. classroom activities and foreign language achievement /ghanizadeh, sharifi & jahedizadeh 675 table 6. independent samples t-test showing the results of post-test on english proficiency the results of posttest on perceptions of classroom activities to examine whether there is any significant difference between control and experimental groups regarding their perceptions of classroom activities (interest, challenge, choice, and joy), an independent samples t-test was run on each perception. table 7 below summarizes the descriptive results of perceptions in two groups. as the table shows, the mean scores of perceptions across participants in control and experimental groups are different. table 7. descriptive statistics of perceptions of classroom activities across control and experimental groups in post-test groups n mean std. deviation std. error mean interest exp 26 25.6154 4.85038 .95124 con 25 18.6000 7.59934 1.51987 challenge exp 26 21.9231 6.07897 1.19218 con 25 16.0800 5.35350 1.07070 choice exp 26 23.3846 5.49965 1.07857 con 25 15.7600 3.16596 .63319 joy exp 26 24.0769 4.48038 .87868 con 25 15.5200 5.65774 1.13155 to see whether these observed differences are statistically significant, independent samples t-tests were run. table 8 presents the results of t-tests run on perceptions of classroom activities. as it can be seen, there are statistically significant differences between the two groups regarding all four perceptions as follows: interest (t= 3.946, p<.05), challenge (t= 3.637, p<.05), choice (t= 6.035, p<.05), joy (t=6.000, p<.05). in other words, simulation positively influenced student's perceptions of their classroom activities. the effect sizes for each t-value were then computed via cohen's d. the magnitudes for interest, challenge, choice, and joy are 1.10, 1.02, 1.69, and 1.67, respectively, which are quite large values according to cohen's index value. table 8. independent samples t-test showing the results of post-test on student’s perceptions of classroom activities t df sig. (2-tailed) mean difference std. error difference interest 3.946 49 .000 7.01538 1.77796 challenge 3.637 49 .001 5.84308 1.60646 choice 6.035 49 .000 7.62462 1.26333 joy 6.000 49 .000 8.55692 1.42609 t df sig. (2tailed) mean difference std. error difference post-test proficiency 2.208 49 .032 1.975 .89470 international electronic journal of elementary education, 9(3), 667-680, march, 2017 676 interview results the interview questions were guided by research questions and evaluating students’ attitudes and feedback towards simulation. following are the interview questions: interview questions: 1) what were your overall thoughts/feelings about the simulation? 2) how did you like it? 3) does it have any role in your interest to do the classroom activities? in which ways? 4) how did conducting conversation in this creative way affected your learning? in the followings, some responses extracted from students’ interviews are provided: mona: it is a new and attractive method in learning english language for me. i love being in those sessions whether as an actor or an audience. i feel that i am in a real situation of english conversion. i had a good feeling when i performed a role as a native speaker. english is so sweet. faeze: it is so challengeable. we had to check our intonation and pronunciation before implementation of simulation in order to act our roles naturally and in a native-like accent. talking like a native speaker of english is very interesting. bita: i hated english because i didn’t like the english classroom in school at all. but through watching my classmates' performances i thought we were in a scene of cinema instead of a rigid english classroom. fateme: i don’t like to finish the course of conversation through simulation, it was so interesting. mohadese: i took pleasure from this course. my friends and i were fun through the session. i never enjoyed english classrooms whether in school or institute, because i think all of them administrate a unit rule. we must learn whatever they want, we must act whatever they choose, and we must be evaluated whatever they arrange. i liked these kind of methodology because i was free to choose my role, my partner,… . i learned the conversations of english for using in daily life with my friends not for getting score to pass the course. azam: it was a good experience for me to choose what role i've liked. i wish educational system in our country consider this kind of practical and authentic activity in our textbooks so that we can put ourselves in a real situation like this and perform the text as naturally as possible. discussion and conclusions the present study is the amalgamation of quantitative and quantitative research in that it employed an experimental design with control and experimental groups alongside the follow up interview. it aimed at exploring the effect of simulation on efl learners' perception of classroom activities, their language achievement, and their attitudes toward this innovative technique. primarily, this study probed students’ improvement regarding their perceptions of classroom activities (interest, challenge, choice, and joy). the results indicated that simulation influenced all four perceptions positively and significantly. findings after implementing this creative technique indicated that all four perceptions were significantly higher in experimental group showing that language classes got enjoyable, interesting, challenging, and fluid for most of the students as there were some limitations and deficiencies for teaching english programs in iranian middle schools. in particular, it was found that students in experimental group displayed more interest in their classroom activities under the influence of implementing simulation in their class. this is in line with haertel, walberg, and haertel (1981) and fraser (1991) who reported that stimulating, protective, and challenging learning atmosphere can interestingly develop performance and increase the level of interest and involvement in every classroom context. consistent with the above finding, ainley, hidi, and berndorff (2002) as well as hidi and renninger (2006) contended that interest promotes effort, attention, recall, and classroom activities and foreign language achievement /ghanizadeh, sharifi & jahedizadeh 677 achievement, and the characterestics of relevance and a sense of reality in simulation are desirable and interesting because simulation allows students to use the target language to express themselves. moreover, the results of both quantitative and qualitative data showed that through learning in the light of the simulation technique, learners perceived their classroom activities joyful in that as stated earlier it inherently tends to promote real-life and authentic communication (crookall, coote, dumas, & le gat, 1987; crookall & oxford, 1990; nemitcheva, 1995), enhances instrumental motivation by making the coursework more engaging (jones, 1986, p. 10), lowers affective barriers to acquisition by reducing the fear of making mistakes (nemitcheva, 1995), and presents real time scenarios and instantaneous feedback (jones, 1986). the findings also suggested that students in experimental group reckoned that they had more opportunities to choose the type of term project, rules, roles, etc. in their classroom. this is due to the fact that as mentioned earlier, simulation gives students the freedom to make their own choices and decisions, and allows students to base their choices and decisions on their own experience. furthermore, since students were not considered as active agents in class in the traditional system of learning english language in schools, they did not have any choice in classrooms while performing a given task. it is suggested that this lack of choice is related to the teacher-centeredness of classes in our country and teacher makes decisions instead of students about everything related to classroom; that is, designing a course, teaching, and curriculum (ghanizadeh, 2016). simulation, nevertheless, seems to provide ample opportunities for students to choose language fucntions, their roles, scenarios, … etc. cray and currie (1996) maintained that teachers do not have to act on behalf of their learners, but with their learners which is parallel to the discoveries of this research, as classes were teacher-centered and they did not take students’ needs and preferences into account. therefore, there should be a combination of teachers’ and learners’ views to improve the quality of learning. furthermore, because the technique gives students the chance to perform a task or solve a problem together, it has been perceived as challenging. so, the learners were more involved and enthusiastic in the activities which were perceived as more challenging and fluid. in other words, simulation positively influenced students' engagement in challenging tasks. in this study, it was also found that simulation contributed to language learning. we can argue that since the purpose of the simulation is to assist students’ conceptual learning, the actors through the conversations who were selected based on their needs and interests, can make improvements in their language use via verbal prompts, cues, and/or questions to guide or scaffold the students’ language proficiency. meanwhile, their cognitive understanding and consequently their language proficiency improved. it can also be argued perceptions of classroom activities mirror subsequent learning in the class. so, when these perceptions are enhanced, student learning experiences and the effort they dedicate to learning will in turn improve (csizer & kormos, 2009) resulting in higher language achievement. the interview protocols suggested that all interviewees had favorable attitudes towards simulation and enjoyed the experience. initial feelings that all participants reminded before the simulation were being intimidated, anxious, uninterested, and not motivated. these initial feelings were more related to a phobia about their language ability to use english language for the extended period of time. simulation changed this phobia to a high desire to learning english, leading to an attitudinal change in their motivation to learn. international electronic journal of elementary education, 9(3), 667-680, march, 2017 678 conclusions and implications the findings of the present study can have important implications for second language learning research in general, and efl teacher education in particular. results of this study, in accordance with previous research, revealed that using simulation technique in performing conversations in the classrooms provide the perception phenomena in the students. these perceptions in turn assist learners shape their conceptualization toward their learning english. in the other word, if students perceive their classroom activities as challenging, interesting, and joyful with various options to choose a particular task, their first priority would be adapting motivation and making attempts to learn effectively. taken together, it can be concluded that the benefits of short-term implementation of the simulation technique to supplement traditional classroom instruction yields a cost effective means for students to acquire a greater use of the target language while building confidence and motivation. even a short ten-session period with holding the classroom activities through simulation, as shown by this study, can build positive perceptions, motivation, and language proficiency. all in all, simulation not only is a tool for influencing students' attitudes to language learning, but also a facilitator of students’ learning at increasingly higher levels of conceptualization. as mentioned earlier, student's views and perceptions are salient areas to consider when designing effective educational experiences. in school reform efforts, emphasis is often placed on achievement measures, whereas student attitude also plays a large role in school success simultaneously. one of the prominent roles of efl teachers is to enhance learning outcomes and help students adapt good habits toward learning environment by implementing innovative and appealing techniques. teachers in an efl context can shape these perceptions by providing challenging activities, interesting tasks with vast ranges of options, and enjoyable learning activities (ghanizadeh & royaei, 2015). designing such activities need time, energy, cost, motivation, and support of educational authorities, but the point is that it’s worth doing. although, making all efl learners interested in learning process is a tough job, it is possible to lead students toward using real conversations by emphasizing on their proficiency in learning, not their marks at the end of the course. the implications mentioned here are totally practical if the ultimate objective of motivated methodology of presenting the subjects is to ameliorate students’ functionality rather than superficial teaching of the course. • • • references ainley, m., hidi, s., & berndorff, d. (2002). interest, learning, and the psychological processes that mediate their relationship. journal of educational psychology, 94, 545-561. albert, i. p. (1999). web-based simulation generator: empowering teaching and learning media in political science. paper presented at asciltie99. bambrough, p. (1994). simulations in english teaching. buckingham: open university press. brown, j. s., collins, a., & duguid, p. (1989). situated cognition and the culture of learning. educational researcher, 18(1), 32-42. classroom activities and foreign language achievement /ghanizadeh, sharifi & jahedizadeh 679 bullard, n. (1990). briefing and debriefing. in d. crookall & r. oxford (eds.), simulation, gaming and language learning (pp. 55–67). new york: newbury house. cray, e. & currie, p. (1996). linking adult learners with the education of l2 teachers. tesol quarterly, 30 (1), 113-130. crookall, d., coote, a., dumas, d., & le gat, a. (1987). the isaga game: inquisitive speaking and gameful acquaintance: a mix of tongues and communicating across cultures. in d. crookall, c. s. greenblatt, a. coote, j. klabbers & d. watson (eds.), simulation-gaming in the late 1980s (pp. 5763). oxford: pergamon. crookall, d., & oxford, r. (1990). simulation, gaming, and language learning. new york: newbury house publishers. csizer, k., & kormos, j. (2009). learning experiences, selves and motivated learning behaviour : a comparative analysis of structural models for hungarian secondary and university learners of english. in z dörnyei & e ushioda (eds), motivation, language identity and the l2 self.. multilingual mattters, bristol, pp. 98-117. davis, r. s. (1996). simulations: a tool for testing "virtual reality" in the language classroom. in jalt '95: curriculum and evaluation. tokyo: japan association for language teaching dehaan j. (2011). teaching and learning english through digital game projects. digital culture & education, 3(1), 46-55. dehaan j., reed w. m., kuwada k. (2010). the effect of interactivity with a music video game on second language vocabulary recall. language learning & technology, 14(2), 74-94. dougil, j. (1987). drama activities for language teaching. london: macmillan. frasr, b. j. (1991). two decades of classroom environment research, in b.j. fraser & h.j. walberg (eds) educational environments: evaluations, antecedents and consequence, pp 3-27. london: pergamon. freiermuth, m. (2002). connecting with computer science students by building bridges. situation and gaming, 3(3), 299-315. gentry, m., & gable, r. k. (2001). my class activities: a survey instrument to assess students’ perceptions of interest, challenge, choice, and enjoyment in their classrooms. mansfield center, ct: creative learning press. gentry, m., gable, r. k., & rizza, m. g. (2002) students perceptions of classroom activities: are there grade-level and gender differences. journal of educational psychology, 94, 539-544. ghanizadeh, a. (2016). the interplay between reflective thinking, critical thinking, self-monitoring, and academic achievement in higher education. higher education, doi 10.1007/s10734-0160031-y. ghanizadeh, a., & alishahi, m.h. (2016). the bonds between efl learners' perceptions of classroom activities, self-regulatory skills, and language achievement. international journal of educational investigations, 3(2), 72-85. ghanizadeh, a., & jahedizadeh, s. (2015). an exploration of efl learners' perceptions of classroom activities and their achievement goal orientations. international journal of research studies in education, 4(3), 33-45. ghanizadeh, a., & royaei, n. (2015). emotional facet of language teaching: emotion regulation and emotional labor strategies as predictors of teacher burnout. international journal of pedagogies and learning, 10(2), 139–150. doi:10.1080/22040552.2015.1113847. haertel, g. d., h. j. walberg, and e. h. haertel. (1981). “social-psychological environments and learning: a quantitative synthesis.” british educational research journal 7, 27–36. hidi, s., & renninger, k. a. (2006). the four-phase model of interest development. educational psychologist, 41, 111–127. international electronic journal of elementary education, 9(3), 667-680, march, 2017 680 hyland, k. (2009). language-learning simulations: a practical guide. in: english teaching forum, 31(4). jahedizadeh, s., ghanizadeh, a., & ghonsooly, b. (2016). the role of efl learners’ demotivation, perceptions of classroom activities, and mastery goal in predicting their language achievement and burnout. asian-pacific journal of second and foreign language education, 1-16. doi 10.1186/s40862-016-0021-8 javid, c. (2013). an investigation of effectiveness of simulation in developing oral skills: a case study. european scientific journal, 9(32), 254-270. jones, g. (1986). computer simulations in language teaching -the kingdom experiment. system, 14(2), 171-178. kharrazi, a., & kareshki, h. (2010). environmental perceptions, motivational beliefs and selfregulating learning by iranian high school students. procedia social and behavioral sciences, 5, 2160-2164. lee, j., yin, h., & zhang, z. (2009). exploring the influence of the classroom environment on students’ motivation and self-regulated learning in hong kong. the asia-pacific education researcher, 18 (2), 219-232. livingstone, c. (1983). role-play in language learning. singapore, longman. lyu, y. (2006). simulations and second/foreign language learning: improving communication skills through simulations. theses and dissertations. paper 1364. meihami, h., meihami, b., & varmaghani, z. (2013). call in the form of simulation games: teaching english vocabulary and pronunciation through sims. international letters of social and humanistic sciences, 8, 57-65. miller, m., & hegelheimer, v. (2006). the sims meet esl: incorporating authentic computer simulation games into the language classroom. international journal of interactive technology and smart education, 3(4), 311–328. nemitcheva, n. (1995). the psychologist and games in the intensive foreign language game-based course. in d. crookall & k. arai (eds.), simulation and gaming across disciplines and cultures (pp. 70-74). thousand oaks, ca: sage publications. ozkal, k., tekkaya, c., cakiroglu, j., & sungur, s. (2008). a conceptual model of relationships among constructivist learning environment perceptions, epistemological beliefs, and learning approaches. learning and individual differences, 19, 71-79. purushotma, r. (2005). commentary: you’re not studying, you’re just . . . . language learning & technology, 9(1), 80–96. ranalli, j. (2008). learning english with the sims: exploiting authentic computer simulation games for l2 learning. computer assisted language learning, 21(5), 441-455. sam, y. w. (1990). drama in teaching english as a second language a communicative approach. in: the english teacher (xix); melta. schwienhorst, k. (2002). why virtual, why environments? implementing virtual reality concepts in computer-assisted language learning. simulation & gaming, 33(2), 196–209. tompkins, k. p. (1998). role playing/simulation. the internet tesl journal, 4(8). wang, y. h. (2010). using communicative language games in teaching and learning english in taiwanese primary schools. journal of engineering technology and education, 7(1), 126-142. young, a. j. (1997). i think, therefore i’m motivated: the relations among cognitive strategy use, motivational orientation and classroom perceptions over time. learning and individual differences, 9(3), 249-283. international electronic journal of elementary education, 2014, 6(3), 463-478. issn:1307-9298 copyright © iejee www.iejee.com using the branching story approach to motivate students’ interest in reading1 hanan alduraby  imam mohammed bin saud university, saudi arabia jane liu eastern washington university, usa abstract this action research was to answer the pedagogical question: how effective can a branching story approach be as a motivational tool for elementary reading instruction? a branching story was created with hyperlinks built into a powerpoint story. the young readers could interact with options at different turning points of the story, which activated the readers’ curiosity, and promoted their decision making and thinking skills. the study was carried out in an all girl school, saudi arabia. a class of 20 fourth-graders participated in the study. for data collection and analyses, the students were assigned into four groups based on their previous reading achievements and interests. students’ responses to comprehension questions were tabulated. the unit-end survey results were analyzed. a majority of the participating students overwhelmingly enjoyed the experience. they were motivated to achieve the project goals sooner than planned. kinesthetic learners demonstrated their academic potentials well beyond their performance under the traditional reading instruction. keywords: branching story, narrative reading, multimedia technology, reading instruction introduction reading instruction reading contributes to improvement of individuals’ minds and expansion of human beings’ imagination. people are travelling far away in their minds during their reading activities (sofsian, 2006). reading improves vocabulary and the readers’ power of word. learning how to read at an early age helps with a higher level of intelligence and greater academic achievements as well as emotional development (ciampa, 2012; edgington, 1998; sofsian, 2006). young readers have more self-confidence than the nonreaders (international reading association (ira), 2012; national association for 1 this manuscript is from a master’s thesis.  hanan alduraby, department of education eastern washington university cheney, wa, 99004 phone: 509-359-7023 email: jliu2@ewu.edu international electronic journal of elementary education vol.6, issue 3, 463-478, 2014 the education of young children (naeyc), 2009). students’ reading achievements correlate with the time they spent in reading activities (gambrell, 2011; swartz & hendricks, 2000). therefore, all children must have the needed time and opportunities to practice reading. moss and terrell (2012) said that independent reading is an essential element of quality reading programs. independent reading offers students a good opportunity to practice their reading skills. however, reading independently does not begin as an independent effort. it is a collaborative effort among students, educators and also parents (sofsian, 2006). ira and naeyc (2012) confirmed that effective teachers depend on their reading knowledge, current studies, appropriate expectations and understanding of children’s strengths and needs to build their instructional strategies. they create developmentally appropriate goals and then find the appropriate instructions to achieve the goals. the developmentally appropriate instruction should be challenging, but achievable. the practice focuses on students’ overall development in all domains of cognition, emotion, language, morale, and kinesthesis skills (abu-jaber, al-shawareb, & gheith, 2010). this instruction includes daily dependent and independent reading experiences with stories and informational books. narrative reading there are many kinds of reading, including historical, scientific and narrative reading. narrative reading is one of the most attractive and effective reading activities (blyer & perkins, 1999). narrative is defined as a form of spoken or written communication, which tells a story. the word, narrative, comes from the latin word genarare meaning ‘to know’ (nathanson, 2006). willingham (2004) believed that narratives have a privileged status in the learning process. human minds deal with narratives in a different way compared with other types of reading discourse. there are four fundamental advantages about narratives. they can be used to encourage everyone to love reading. children prefer narratives (fiction) to non-fiction books. secondly, narratives are easy to comprehend because there is repetition of names, phrases and vocabulary. reading stories is generally a quicker experience than non-fiction reading (nathanson, 2006). thirdly, narratives build a story structure that facilitates making connection. narratives should be viewed as an intrinsically teleological form, in which events can be explained by the prior events. narrative stories are a chain of events that lead to conflicts and complications (cronon, 1992). in addition, narratives can be facilitated through on-line processing and inference-making. moreover, narratives are persuasive tools that impact values and morals that may help to change the beliefs and ideas of individuals toward the world (mar, djikic, & oatley, 2008). fictional narratives may reinforce moral development and improve empathy. reading narratives can influence one’s character. the stories create shared meaning, beliefs, and visions that readers can associate with (hakemulder, 2001; kelly & zak, 1999). according to kelly and zak (1999), narratives enable readers to dive into the story world, make readers part of the story world. narratives in islam are equally valued in its education. the narratives are about human or nonhuman creatures that demonstrate representative good or bad behaviors (yusef, 2011). the role of the creatures is to cultivate among people morally sound behaviors and avoid immoral ones. aljefri (2008) argued that the islamic narratives are used for religious, intellectual, behavioral, and social skills’ development. the narratives represent truth of divinity, universe, human beings and lives, instill values, as well as teach how to construct a better society. they also carry as in any literature of branching story approach / alduraby & liu 465 other cultures the elements of stories, such as plots, setting, events, and characters to achieve educational goals. because of the human needs to create meaning and make connections between knowledge and experiences, narratives are part of human traditions in all cultures. they help to protect the culture by telling, retelling and exploiting a culture’s heroes against the threat of their enemies (kelly & zak, 1999). therefore, this literacy mode is popularly used in all countries and cultures. motivation to read reading is considered as the first duty that prophet mohammad had received from allah through the angel gabriel. in muslim culture, people, old and young, are expected to read the qur’an as well as other books so that people may live a fuller life. with so many advantages of reading and obligation to become literate citizens, people are not necessarily interested in reading. some muslims today are not voluntary readers, that is, they do not have the desire to read different books. unesco (cited in alaga, n.d) recently reported that arab countries have the highest percentage of illiteracy in the world. reading activity is one of the least interesting hobbies for quite a number of people in the arab world. a study of 80 arab people (afifi, 2012) indicated that the participants only read one book a year. an individual in the arab countries spends an average of only 6 minutes in reading (outside the qur’an) per year while individuals in european countries and america read for an average of 200 hours a year. while 35,000 books are published in america and 85,000 books in japan a year, only 5,000 books are published every year in the arabic world. obviously, lack of reading resources and little motivation to read are problems in the arab countries, including in saudi arabia. gambrell (2011a) described motivation to read as “the likelihood of engaging in reading or choosing to read” (p. 5). people with high motivation to read find time to read, consequentially improve their reading skills. social cognitive theory can help explain the reason for lack of motivation (gambrell, 2011a). learners do not imitate their teachers thoughtlessly. they need to cognitively realize the importance of reading while they follow the teacher’s modeling. it asserts the role of self-efficacy in learning. motivated readers read more than unmotivated readers, achieved higher levels in their reading classes, performed better on standardized tests of reading and had higher school grades (applegate, 2011; gambrell, 2011). deci and ryan (as cited in ciampa, 2012) believed that motivation is critical to engage young students in the reading process. it is also a clear predictor of the students’ future skills in reading. students’ motivation appears in their thoughts about themselves as readers and their opinion about the reading process (ciampa, 2012). the most important reason for high motivation to read is that the more one reads, the better readers he/she becomes (gambrell, 2011a). lai (2013) found many useful ways to improve students’ reading skills; however, students are not able to benefit from these ways unless they have the drive to read. teachers at elementary schools are viewed as the most influential in students’ motivation to read than the teachers after elementary schools (ulper, 2011). a powerful strategy for motivation is to offer opportunities for students to choose what and how to read. retting and hendricks (2000) asserted that choice is a strong force that urges students to become autonomous for their own learning. when students are given the chance to choose their reading assignments, they are more intrinsically motivated and have better performance (ciampa, 2012). using technology for reading activities over the past decade, with technological advancement, saudis are becoming the biggest buyers in the world of technology (alzubedi, 2012). although saudi arabia has the fewest readers in the arab world, saudies are the best readers in fast reading by international electronic journal of elementary education vol.6, issue 3, 463-478, 2014 using the new technology tools (alhasan, 2010). the strong economic conditions in saudi arabia enable the saudi people to use these devices easily. people are interested in multimedia tools. they prefer to use technological tools like smartphones for fast reading through social networks. this technology revolution in saudi arabia is affecting the youth’s learning style, too. conventional language learning methods are being challenged since this technology-savvy generation of students grows up with the 21st century technologies. there was no explicit data by far about time saudi k-12 students spend with technology tools, but it was reported that american youth spent between 6 and 8 hours daily using technology tools (petkove & rogers, 2011). this generation of children considers e-book as “a new and unique medium” and many of them prefer digital reading to the conventional books (ciampa, 2012; reinking & watkins, 2000). innovative educators are using digital stories as an alternate tool to serve a powerful role of reviving language learning and culture as well (rivera & reuney, 2010). digital stories usually contain some multi-media features, such as videos, sounds, graphics, animations as well as choices for readers to practice their problem solving skills. all the features jointly form into a strong motivational force that attracts students to be responsible for their own learning and improve their self-independence (austen, 2001; oakly & jay, 2008; ciampa, 2012; swartz & hendricks, 2000). among digital stories, a branching story is structured with its unique features. branching stories are narratives. they have different paths that encourage the reader to interact with the plot. it contains “directed graph of nodes connected by arcs that represent user choices. every possible path through the graph represents a story that can be told to the user” (riedl & young, 2006, p. 26). the choices in a branching story engage readers into making a decision among the options and looking forward to the next event of the story. branching stories can be compiled into different genres, such as fantasy, adventure, and even video games. they can be used for the purposes of entertainment as well as instructional practice (lai, 2013). there are many technological tools to create branching stories. one of the easiest and most common tools is microsoft powerpoint. jones (2003) described powerpoint as a presentation program, which was initially used widely in the business world, and then eventually brought into educational field. if used effectively, powerpoint can (a) promote teaching and learning process for both students and teachers, (b) encourage teachers to facilitate their instruction in a professional matter, and (c) distribute the electronic file for and to students where the viewer of powerpoint is free (jones, 2003; nouri & shahid, 2005). the program can be a powerful motivational medium. purpose of the project the purpose of this study was to examine how branching stories could effect students’ motivation to read. most people in saudi arabia do not have the desire to read beyond their religious obligations. the problem starts during their childhood when they are not encouraged to read. the notion of reading is related to the notion of studying. this generation of technology-dependent students enjoy reading using technological tools. therefore, a computer-based branching story was created in an effort to enhance students’ intrinsic motivation in reading. research design this project went through several major preparatory stages: designing a branching story and related reading activities, obtaining approval from the university institutional research board and also from ministry of education in saudi arabia, and then choosing a school that accepted this experiment. branching story approach / alduraby & liu 467 branching story design in the process of designing a branching story, identification of its reader was the first to consider. all schools in saudi arabia are gender segregated, so the researchers knew that the study would be carried out in an all-girl school. fourth or fifth graders were the target readers because they should have possessed the skill in independent reading and an ability to answer comprehension questions with evidence to support their answers. thus a narrative branching story was planned to design for forth and/or fifth grade female students. in order to ensure that the story was developmentally appropriate, the native saudi researcher first explored saudi young girls’ interest in stories they had read. almost all saudi female elementary children asked mentioned cinderella as their favorite. as a result, a branching story was generated, titled as “cinderella wants a child”, using a popular character from a popular fairy tale. the story starts where cinderella’s original version ends. it unfolds after cinderella’s happy marriage to the prince. cinderella has difficulty in getting pregnant. she faces the pressure of either being divorced from the royal family due to no help with the western medicine, or going to the top of a far-away mountain to secure a magic herb. the story is not just for fun, but also to teach children moral lessons of love, bravery, and perseverance. the branching story was structured as a narrative mediation tree (figure 1). it includes 44 slides in total with graphics and scripts on each slide. hyperlinks are built into 40% of the slides that take the readers to eight different paths and/or twists of the story. the eight paths loop and are intertwined. two buttons are provided on each of 18 slides. students can choose to click on either button and then be taken to the chosen path of the story development. a path contains between six and sixteen slides. the language on each slide is formal in order to improve students’ vocabulary. options are offered to maintain student’s interest, but not provided on each slide because the purpose of this project was to promote students’ desire in reading instead of playing a game. the script was written in both islam and english. the words on each slide varied in length in order to improve students’ reading skills and avoid students’ fatigue with continuous reading of the lengthy text. powerpoint presentation program was used to carry through the story. pink color was chosen as a background because the saudi girls who were inquired about their favorable figure associated cinderella with the color of pink. black was the chosen font color for sharp visionary contrast. pictures and images were integrated to the slides to maintain young readers’ interest and to promote their imagination. the feature of “rehearse time” was also built into each slide. this enables educators or researchers to record the time length during which a reader stays on each slide. international electronic journal of elementary education vol.6, issue 3, 463-478, 2014 figure 1. narrative mediation tree project activities this branching story approach included four components for students to perform: 1) understanding of computer rules, 2) independent reading time of the branching story, 3) individual time to answer the comprehension questions on the worksheet, and 4) whole class sharing time. the rules students should understand and comply with during independent reading time were:  do not start the activity without the teacher’s permission.  choose only one option from the two provided.  do not go back to the previous slide after clicking the button.  use only computer mouse, not the keyboard.  stop reading at the end of the scheduled reading time.  do not use the computer while they answer the comprehension questions.  start reading on the second, third, and fourth day from where they stopped the day before.  tell the teacher when they reach the end of the branching story. branching story approach / alduraby & liu 469 comprehension is an essential component of the branching story approach. ten comprehension questions were developed for students to answer after reading the story. the questions covered all levels of cognitive development. the knowledge-level questions were: what is cinderella doing? what did cinderella see? the prediction questions included: what do you think she would do? what do you think would happen to cinderella next? what is your prediction of the story? and synthesis questions are: how would cinderella find the treatment for her pregnancy? why do you like cinderella and some other figure(s) in the story? the last section of the whole class discussion was scheduled to provide students the opportunity to orally communicate their understanding of the story. participants twenty fourth-grade females were the participants. complying with the research policies of the american university and the ministry of education in saudi arabia, the native saudi researcher first needed to choose a participating school out of the ministry’s approval list. because all schools in saudi arabia are gender segregated and the native saudi researcher is a female, the school chosen had to be an all-girl elementary school. furthermore, the researcher needed to ensure that the school should have enough computers for each participating student. as a result, a private school with a well-equipped computer lab was chosen. the principal there embraced this innovative study warmly. she made the decision on the participating teacher and her entire intact class because the teacher was viewed as a strong reader herself and a professional with a sincere interest in reading strategies. the fourth graders between the ages of nine and ten in the entire class were all involved in the experiment. the students were divided into four groups based on their previously demonstrated interest in reading and reading achievements: four high readers (h 1-4), seven average readers (a 1-7), five low readers (l 1-5), and four highly active students (ha 1-4) with low academic performance. project implementation the entire study was planned for five days. the islam version of the story was used. the native saudi researcher performed as activity facilitator and observer. the teacher maintained to be the direct communicator with her students. the activity started at the beginning of the week, saturday and continued on sunday, and other days of the week. thursdays and fridays are weekends in saudi arabia. the activity lasted for 45 minutes on the first day and 40 minutes for the remaining days. on each of the days, all the fourth graders transited from their classroom to the school computer lab after the regular school day. they each took their assigned seat in front of a computer that had a cover page of the branching story on the screen. on the first day, the teacher started by asking the students about the story of cinderella, then informing them that the activity would let them know more about cinderella after her happy marriage. the teacher explained the rules and procedures of the branching story approach, and then the students started independent exploration and reading of the story. at the beginning, the students were very eager to start the process. several of them watched their classmates for the first few minutes, and then got into the discourse themselves. a few just clicked on the option buttons to satisfy their curiosity of this new practice. the students all knew quite well how to use a computer, and did not have much difficulty with computer operation. most of the questions they asked were related with what to do with the options. it was interesting enough to notice that the questions mainly related to what was “right” to do, such as “what is the correct button?” “should i international electronic journal of elementary education vol.6, issue 3, 463-478, 2014 click on this button instead of that?” “is this a correct option?” some students repeatedly asked the similar questions. they seemed to have a hard time in realizing that they were given options and autonomy to choose. with the days progressing, more students looked more interested and engaged in the activity. they asked to come to the computer lab earlier. they quickly got into the quiet reading time after entering the lab. there was less informal chatting between the students during the reading time. they were so attentive to reading with their bodies leaning closely toward the computer screens. more students were willing to share their answers to the comprehension questions in written and orally. they were excited to tell about what they had read. on the fourth day, students came to the computer lab and started immediately working on the story. it was apparent that a majority of the students accomplished the story and answered the comprehension questions well on the day. the researcher and the teacher decided to end the study one day earlier. however, it was encouraging to observe the few students who did not finish the entire story continue reading because they were still eager to know the ending of the story. data report the empirical data was collected. the researcher took anecdotal notes while observing the students. “rehearse time” was saved daily and recorded on the computers each day after students left the computer lab. the students answered the comprehension questions, demonstrating their understanding of the narrative they had read that day and predicted what would happen next. the whole class sharing time turned out to be a pleasant interactive opportunity for verbal story telling. the students completed the unit-end survey to express their opinions about this reading event. table 1. data report on student’s reading time. (time was recorded in seconds.) slides slide 2 64 words with options slide 12 43 words no options slide 4 70 words with options slide 27 83 words no options slide 20 28 words with options slide 33 5 words without options students h1 49 s 35 s 6 25 11 s 10 h2 51 s 35 s 50 92 25 s 46 h3 19 s 41 s 0 read 64 15 s 9 h4 45 s 55 s 24 0 32 s a1 58 s 24 s 0 16 7 s 8 a2 58 s 24 s 19 24 10 s 16 a3 52 s 39 s 0 51 12 s 35 a4 12 s 32 s 41 26 10 s 9 *a6 28 s 23 s 19 9 18 s 28 a7 17 s 24 s 26 41 15 s 0 l1 76 s 107 s 0 75 41 s 0 l2 139 s 31 s 0 75 42 s 27 branching story approach / alduraby & liu 471 table 1 (cont.). data report on student’s reading time. (time was recorded in seconds.) slides slide 2 64 words with slide 12 43 words no slide 4 70 words with slide 27 83 words no options slide 20 28 words with slide 33 5 words without students l3 10 s 80 s 49 0 11 s 0 l4 30 s 12 s 32 83 11 s 0 l5 80 s 55 s 121 s 60 33 s 40 ha1 16 s 42 s 0 52 5 s 8 ha2 63 s 78 s 126 s 24 36 s 38 ha3 21 s 3 7 14 4 s 6 ha4 8 28 s 19 4 21 s 15 * a5 was absent for two days. so her record was not included. the feature of powerpoint “rehearse time” enabled the researchers to track how long a student spent on each slide. this feature showed progression of students’ reading time. it helped the researcher be aware of each student’s reading capacity. table 1 tabulated the time the students spent in reading six selected slides. slide 27 has the most words of 83 and slide 33 contains only five words. students’ reading time for those six slides can be divided into three different levels: long, average and short. the long time used would inform a teacher that a particular student might have difficulty in reading and/or comprehension when her answers to the comprehension questions were not an accurate summary of the narrative. the average time used could imply the time length a teacher should schedule for later. the short time used may lead to two possibilities that the student either did not read or a fast reader. their comprehension answers could be associated as an indicator of either possibility. four students (two has, one a, one l) took longer than 60 seconds for reading slide 2, beginning of the branching story. six students (one h, two a, one l, two has) spent shorter than 20 seconds. the remaining students spent between 30-50 seconds to read slide. three students (two ls, one ha) spent more than 70 seconds to read slide 12, which concluded the branching story. two students (one l, one ha) spent shorter than 13 seconds on it. majority of the students spent 20-50 seconds to read it. for reading slide 4, two students (one l, two has) spent more than 120 seconds reading the slide. five students (one h, two as, two has) took shorter than 20 seconds. the remaining students took from 20 to 50 seconds to read it. two l students spent longer than 40 seconds to read 28 words in slide 20, which had two options. three students (one a, two has) spent shorter than 8 seconds to read slide 20. the remaining students used 13-30 seconds to finish reading the slide. four students (one h, three ls) took more than 70 seconds to read slide 27, which included 83 words with no options. four students (two a, two has) spent shorter than 20 seconds. it took the remaining students 30-60 seconds to read this slide. for reading slide 33, which included 5 words with no options, four students (one h, one a, one l, one ha) spent more than 35 seconds. four students (one a, one l, two has) spent shorter than 8 seconds. the remaining students spent the time between 10-30 seconds. international electronic journal of elementary education vol.6, issue 3, 463-478, 2014 examining the students’ reading time of the six chosen slides, it was conclusive that more low students (l) and one highly active student (ha) needed a longer time to read the slides. the feature of “rehearse time” apparently tracked the time when a student stayed on each slide, but the researchers are cognizant that it entails more practice of this feature in the future to generate appropriate time length during which a student needs to read each of the scripts. the time gap between the students can imply multiple aspects of instruction regarding students’ reading ability, attention span and interest in reading, etc. table 2 recorded the number of words and sentences the students put in their answers to the comprehension questions. some questions required descriptive responses, such as what was cinderella doing in order to get to the mountain? or why do you think cinderella would find the treatment? students’ answers were used to inform the researchers of their comprehension level of the branching story. students from different groups demonstrated different levels of understanding. students’ responses to the comprehension answers without description were categorized as generic answers. a large number of words and descriptive sentences used in the answers belonged to the high level of comprehension. table 2. students’ answers to comprehension questions student number of words number of sentences descriptive answers generic answers h1 41 5 2 8 h2 41 8 4 6 h3 37 6 3 7 h4 40 4 3 5 a1 32 6 3 7 a2 74 9 7 1 a3 17 4 1 9 a4 36 7 2 8 a5 40 7 2 8 a6 71 12 6 4 l1 16 3 2 5 l2 4 1 1 6 l3 30 4 4 1 l4 25 6 4 3 l5 43 9 4 6 ha1 61 9 3 7 ha2 63 10 8 2 ha3 57 9 6 4 ha4 74 9 5 5 branching story approach / alduraby & liu 473 all the four ha and two a students responded with rather long answers of more than 55 words. they each wrote more than 10 sentences. their answers were more descriptive than other classmates. two l (4 and 16 words used) should be categorized as “low” because they wrote the least number of words for the comprehension questions. their answers were more generic than descriptive. the answers from the remaining ten students 4 hs, 3 as, and 3 ls were categorized as “average”. the average number of words in their answers was 38 words. the number of descriptive answers was appropriately three sentences from each student. to the teacher’s greatest surprise, the ha students answered the comprehension questions with a larger number of descriptive sentences than their peers of any other groups did. they demonstrated their understanding of the story clearly. the recorded release time revealed that three of them spent a relatively short time in reading the story. the teacher assigned them as highly active learners due to her previous difficulty in keeping them engaged into an activity or maintaining their attention for a longer period of time in class. this study offered them an alternate opportunity to display their cognitive potentials. their minds responded much better with the interactive kinesthetic activity. once the learning approach matched their learning styles, they could focus and achieve a lot better. table 3 displayed students’ responses on the unit-end survey. seventeen students participated in the survey. due to family activity, three students did not attend school that day. the data revealed that 90% of the students (15 out of 17) viewed highly their experience with the branching story approach. they also expressed their desire to read more stories like this. fifteen students preferred reading on a computer to a printed book. thirteen students liked to have the feature of options in the branching story. table 3. result of unit-end survey survey questions answers 4 high students 6 average students 3 low students 4 highly active how was your experience with the activity of the branching story? great 100% (4) 83.3% (5) 100% (3) 75% (3) weak 0% 16.6% (1) 0% 25% (1) do you like the cinderella story? yes 100% (4) 83.3% (5) 100% (3) 75% (3) no 0% 16.6% (1) 0% 25% (1) were the instructions of reading the branching story clear? yes 75% (3) 66.6% (4) 66.6% (2) 75% (3) not too much 25% (1) 16.6% (1) 33.3% (1) 25% (1) no 0% 16.6% (1) 0% 0% how were the questions about the story? easy 100% (4) 50% (3) 66.6% (2) 75% (3) average 0% 50% (3) 33.3% (1) 25% (1) do you like the options of the story? yes 100% (4) 50% (3) 100% (3) 75% (3) not too much 0% 50% (3) 0% 25% (1) international electronic journal of elementary education vol.6, issue 3, 463-478, 2014 table 3 (cont.). result of unit-end survey the whole class sharing time was the culminating point of the branching story approach. the students were unexpectedly active in sharing their interpretation of the story with the classmates. the students’ participation level was much higher than in the conventional reading activities according to the teacher. some students responded the comprehension questions with the vocabulary at a more advanced cognitive level than before. interacting with the story, making their own options for different paths, and using narrative writing to express their own comprehension of the story activated the students’ minds and offered the students the opportunities to display their academic potentials. their positive attitude in the entire discourse and demonstrated engaging behaviors manifested the effect of technology integration in motivating students’ passion for reading. the most amazing changes occurred on the four students who were labeled as “highly active” because they had a hard time in focusing, which consequently affected their reading grades negatively with the traditional instructional practice. their behaviors and performance levels with this branching story approach were astonishing. they were engaged into the activities without any behavior problems in all the four days. three of them showed their best cognitive skills never before in writing and verbal sharing. even the other ha student who spent a little longer time in reading the story also outperformed many other peers in her answers to the comprehension questions. discussions in summary, this project accomplished its original goals: the branching story approach interested students and motivated them to read. the experience energized students’ enthusiasm in reading. they were eager to read the varied plots of the story and volunteered to share their understanding of the story. they were passionate about informing their classmates about what they had read due to the different path they each chose on the different days. they felt empowered with the options, and gained satisfaction of learning more twists of the story, even if one day earlier than some of their classmates. the students’ participation level exceeded their performance with the conventional reading activities. the recorded “rehearse time” reflected the students’ increased focus on reading. by the end of the event, the time the students spent on each slide became longer. they became more serious about reading and understanding the content with decreased causal chatting among the students while reading. this branching story design created opportunities for students’ engagement, and the engaged students worked hard. the do you like to read a hypertext story or a printed book story? hypertext story 100% (4) 66.6% (4) 100% (3) 100% (4) printed story 0% 16.6 (1) 0% 0% do you like to have the same activity with different stories? yes 100% (4) 66.6% (4) 100% (3) 100% (4) no 0% 33.3% (2) 0% 0% do you like the pictures in the story? yes 100% (4) 66.6% (4) 100% (3) 75% (3) not too much 0% 16.6% (1) 0% 25% (1) no 0% 16.6% (1) 0% 0% branching story approach / alduraby & liu 475 students’ attitudes, learning and performance improved with more active self-efficacy and motivation. the results of this project once again reinforce the necessity to differentiate instruction to promote all students’ interest in reading. traditional methods have proved unsuccessful in getting all students involved into an effective learning, especially with this e-generation. the kinesthetic learners require hands-on experiences. doing, touching and taking part in physical activities promote their success in learning (hutton, 2007). transformed instructional strategies can function to facilitate students’ learning and bring their potentials into full play. it is vital to point out that adoption of technology into reading instruction will not totally replace the traditional methods. it should be a part of the entire pedagogy. teachers need to have knowledge and proficiency to employ all kinds of teaching strategies that it takes to meet all students’ needs. teachers need to consider students’ cognitive and affective level as well as culture of the entire school’s instructional practice. using technology tools should be age appropriate and match students’ skills and abilities. limitations of the study all educators should be gender sensitive, and not stereotyping. as an initial effort in applying the branching story approach into reading instruction, cinderella was chosen as the main character and the color of pink as a background. however, the researchers realized that saudi young female students should be exposed to more branching stories with broadened cultural and social perspectives. recommendations for future application the positive effects of this branching story approach hold implication for the future efforts to integrate the branching story approach in a wider scope of arabic reading instruction. therefore, the following recommendations were made: 1) more branching stories should be developed for instructional purpose and also for fostering life-long readers. stories/narratives from the printed books can be transformed into multi-media branching format. educators, storywriters, even students should be encouraged to design branching stories as much as possible. 2) it takes time and some expertise to design a branching story, especially narratives that developmentally appropriate. thus teachers should work collaboratively to design stories instead of waiting for the stories to come to them. 3) in order to reach and motivate all kinds of learners, the need to integrate updated technology into the regular reading instruction is obvious. branching stories are just one kind of technology. internet and social media can also be used in reading instruction to provide students with different kinds of e-books. a new generation of saudi students is awaiting more e-reading opportunities. • • • hanan alduraby, instructor, imam mohammed bin saud university. received her master of education degree at eastern washington university, usa. her research interest is in literacy instruction with media and technology. jane liu, ph. d., professor of education, eastern washington university, her research has been on teacher education and comparative education. she has presented and published nationally and internationally. international electronic journal of elementary education vol.6, issue 3, 463-478, 2014 references abu-jaber, m., al-shawareb, a., & gheith, e. (2010). kindergarten teachers’ beliefs toward developmentally appropriate practice in jordan. early childhood educational journal, 38, 6574. doi 10.1007/s10643-010-0379-z afifi, a. (2012). reading in the arab world over the edge of danger. dar alkotob. retrieved from http://www.daralkotob.com/read/press-reports/691 alaga, a. (n.d.) the reality of reading in the arab world. yanabe’a 3, 49-51 retrieved from http://www.qou.edu/arabic/publicrelation/yanabi3/no_3/sub_20.pdf alhasan, f. (2010) saudis do not read as other arabs, but they are the best readers in fasting reading. aleqtisadiah. retrieved from http://www.aleqt.com/2010/01/01/article_650746.html aljefri, h. h. (2008). islamic educating by the story and its applications in kindergarten level. retrieved from http://www.al5aatr.com/researches/13/130_tarbyah_ryaz.pdf alzubedi, e. (2012). saudi arabia has the biggest buyers in the world of smartphones. tech world. retrieved from http://www.tech-wd.com/wd/2012/04/17/phones-share-ksa/ applegate, a. j. (2011). a study of thoughtful literacy and the motivation to read. the reading teacher, 64(4), 226-234. doi:10.1598/rt.64.4.1 austen, i. (2001, january 4). rebooted any good books lately? new york times, pp. 1. blyer, n., & perkins, j. (1999). culture and the power of narrative. journal of business and technical communication, 13(3), 245-248. ciampa, k. (2012). improving grade one students' reading motivation with online electronic storybooks. journal of educational multimedia & hypermedia, 21(1), 5-28. retrieved from ebscohost.com cronon, w. (1992). a place for stories: nature, history and narrative. the journal of american history, 87(4), 1347-1376. edgington, w. d. (1998). the use of children’s literature in middle school social studies: what research does and does not show. the clearing house, 72(2),121-134. gambrell, l. b. (2011a). motivation in the school reading curriculum. journal of reading education, 37(1), 5-14. retrieved from ebscohost.com gambrell, l. b. (2011b) seven rules of engagement: what’s most important to know about motivation to read?. the reading teacher, 65(3), 172-178. doi:10.1002/trtr01024 hakemulder, j. f. (2001). how to make alle menschen bruder: literature in a multicultural and multiform society. in d. schram & g. steen (eds.), the psychological and sociology of literature: inhonor of erlund ibsch (pp.225-242). amsterdam: benjamines. hutton, s. (2007). helping kinesthetic learners succeed. education. retrieved from http://www.education.com/magazine/article/kinesthetic_learner/ international reading association for the education of young children. (2012). learning to read and write: developmentally appropriate practices for young children. retrieved from http://bit.ly/92titj jones, a. m. (2003). the use and abuse of powerpoint in teaching and learning in the life science: a personal overview. bee-j, 2. retrieved from bio.itsn.ac.uk. kelly, c., & zak, m. (1999). narrativity and professional communication: folktale and community meaning. journal of business and technical communication, 13(3), 297-317. lai, j. (2013, october 20). how to write a branching story. ehow. retrieved october 21, 2013 from http://bit.ly/zcwwvk mar, r. a., djikic, m., & oatley, k. (2008). effect of reading on knowledge, social abilities, and selfhood. in s. zyngier, m. bortolussi, a. chenkova, & j. auracher (eds.) directions in branching story approach / alduraby & liu 477 empirical studies in literature: in honor of willie van peer, pp. 127-137. amsterdam: benjamins. moss, b., & terrell, a. (2012). independent reading is not an independent effort. the reading teacher, 65(6), 378-380. doi:10.1002/trtr01056 national association for the education of young children. (2009). developmentally appropriate practice in early childhood programs serving children from birth through age 8. retrieved from http://www.naeyc.org/ files/naeyc/file/positions/position%20statement%20web.pdf nathanson, s. (2006). harnessing the power of story: using narrative reading and writing across content areas. reading horizons, 47(1), 1-26. retrieved from scholarworks.wmich.edu nouri, h., & shahid, a. (2005). the effect of powerpoint presentations on student learning and attitudes. global perspectives on accounting education, 2, 53-73. oakley, g., & jay, j. (2008). "making time" for reading: factors that influence the success of multimedia reading in the home. reading teacher, 62(3), 246-255. retrieved from ebscohost.com petkov, m., & rogers, g. e. (2011). using gaming to motivate today's technology-dependent students. journal of stem teacher education, 48(1), 7-12. retrieved from ebscohost.com rettig, m. k., & hendrick, c. g. (2000). factors that influence the book selection process of students with special needs. journal of adolescent & adult literacy, 43(7), 608-618. reinking, d. & watkins, j. (2000). a formative experiment investigating the use of multimedia book reviews to increase elementary students’ independent reading. reading research quarterly,35(3), 384-419. riedl, m. o., & young, r. m. (2006). from linear story generation to branching story graphs. leee computer graphics and applications. 26(3), 23-31. retrieved from cc.gatech.edu rivera, m. n., & reuney, k. (2010). using digital storytelling to save our language and culture. presentation at 29th annual international pibba conference, guam. sofsian, d. (2006). the importance of reading. ezine articles. retrieved from http://bit.ly/bo6gs5 swartz, m. k., & hendrick, c. g. (2000). factors that influence the book selection process of students with special needs. journal of adolescent & adult literacy, 43(7), 608-618. ulper, h. (2011). the motivational factors for reading in terms of students. educational sciences: theory and practice, 11(2), 954-960. retrieved from ebscohost.com willingham, d. t. (2004). the privileged statues of story. american educator, 28(2), 43-45; 5153. yusef, z. (2011). educational methods in qur'an. retrieved from http://bit.ly/17ppmqc international electronic journal of elementary education vol.6, issue 3, 463-478, 2014 www.iejee.com this page is intentionally left blank international electronic journal of elementary education, 9(3), 613-626, march, 2017 issn:1307-9298 copyright © iejee www.iejee.com the role of socio-cognitive variables in predicting learning satisfaction in smart schools mohammad reza firoozi a  ali kazemi a maryam jokar a a yasouj university, iran received: 1 november 2016 / revised: 10 january 2017/ accepted: 17 january 2017 abstract the present study aimed to investigate the role of socio-cognitive variables in predicting learning satisfaction in smart schools. the population was all the primary school students studying in smart schools in the city of shiraz in the school year 2014-2015. the sample, randomly chosen through multi-stage cluster sampling, was 383 primary school students studying in smart schools in shiraz. the instruments were the computer self-efficiency questionnaire developed by torkzadeh (2003), performance expectation questionnaire developed by compeau and higgins (1995), system functionality and content feature questionnaire developed by pituch and lee (2006), interaction questionnaire developed by johnston, killion and oomen (2005), learning climate questionnaire developed by chou` and liu (2005) and learning satisfaction questionnaire developed by chou and liu (2005). in order to determine the possible relationship between variables and to predict the changes in the degree of satisfaction, we made use of correlational procedures and step-wise regression analysis. the results indicated that all the socio-cognitive variables have a positive and significant correlation with learning satisfaction. out of the socio-cognitive variables in question, computer self-efficiency, performance expectation and learning climate significantly explained 53% of the variance of learning satisfaction. keywords: learning satisfaction, computer self-efficiency, performance expectation and learning climate, system functionality. introduction incorporating technology into instruction, as one of the important aspects of information technology is considered a milestone in the social, vocational and educational life of human beings in the 21st century and has opened up new avenues in educational institutions, especially in schools and universities (rahimi & yadollahi, 2011). information and communication technology has the potentials to speed up, enrich and deepen acquisition of skills. it also helps with implementing fundamental changes in schools, promotes learning and creates opportunities to facilitate communication between schools and society (davis, tearle, 1999). in addition, information and communication technology  corresponding author: mohammadreza firoozi, dept. of psychology, college of humanities, yasouj university, daneshjoo street, yasouj, iran, pc: 7591874934 phone +98 917 124 4196 e-mail: m.firoozi@yu.ac.ir http://www.iejee.com/ international electronic journal of elementary education, 9(3), 613-626, march, 2017 614 could make schools more efficient and productive as it could provide a variety of means to promote and facilitate teachers’ professional activities (kirschner, wopereis, 2003). smart schools represent a new approach to education. by combining information technology and curricula, such schools have brought about fundamental changes in the teaching-learning process (umat, 2000). technology, especially information and communication technology, has acted as a catalyst in transforming traditional schools to smart ones. in other words, information technology is inevitable if the transformation from traditional schools to smart schools is to take place (hashim & man, 1999). running and management of smart schools are based on computer technology and the internet; the content is electronically delivered and parents are connected with the principal and the teachers online and keep abreast of the educational situation of their children (sotani, 2012). in smart schools, the role of the teacher is one of guide, and not as a transmitter of knowledge and the role of the student is that of an active, creative and participating member of the class (umat, 2000). traditional schools have been criticized for promoting passive learning, ignoring individual differences and needs of learners, ignoring problem-solving skills and critical thinking (pelgrum, 2001). on the other hand, new developments in web-based technology have brought about challenges as well as opportunities in the virtual world so much so that nowadays, it is almost impossible to find higher education system in which the benefits of using technology have not been reaped (ong & ruthven, 2009; ming et al., 2010). in these schools, the main idea is taken from electronic education technology, which has had substantial impacts on learning(wu, tennyson, hsia & liao, 2008) and learner satisfaction (sung, kwon, & ryu, 2008). while smart schools are said to have such advantages as richness of instruction, availability of knowledge content, provision of social interaction, and personal responsibility (osguthorpe & graham, 2003), nevertheless, dissatisfaction with learning could act as a threat to adopting smart schools (henceforth ss) (so & brush, 2008). for example, research evidence suggests that in such schools, it is difficult for student to adjust to the smart system which may could be due to technical failures, limited learner ability, inability to use state-of-the-art technology, and the like smart schools due attention should be paid to technological and human factors. these factors may be related to student attitudes and participation, and educational technologies (wu et al., 2008; 2010). this calls for the scrutiny of learners in this system and the technologies used as these could contribute to learner satisfaction (heba & nouby, 2008). smart schools are becoming more popular throughout the world. one of the reasons for their popularity is that these schools create learning atmosphere conducive to learning. given that, it is necessary to gain insights into the factors which make smart schools popular with students and make these schools a success or failure, for that matter. this recognition could enhance learning satisfaction as well. for the purpose of evaluating the effectiveness of smart schools, it is necessary to determine the degree of learning satisfaction of learners with courses offered. gaining insights into the factors determining learning satisfaction could help educational authorities to develop effective strategies as they relate to smart schools, that will, in turn, allow administrators and instructors in educational institutions to create new educational benefits and value for learners. extensive studies have been carried out so far into the factors contributing to the learning satisfaction or dissatisfaction with learning in traditional schools. however, given the essential differences between traditional schools and smart schools, it is necessary to shed light on the cognitive, technological and social factors contributing to learning satisfaction with smart schools. more specifically, there is a dire need for more in-depth research to the role of socio-cognitive variables in predicting learning satisfaction / firoozi, kazemi & jokar 615 gain insights into what determines student learning satisfaction in a smart school environment and to investigate what determinants influence student perceptions of smart school contexts and their correlations. this study, which is consistent with the social cognitive theory (bandura, 1986), is an attempt to investigate the primary determinants affecting student learning satisfaction in a smart school environment. the origins of smart schools date back to 1996. it was first initiated in malaysia. one year later, efforts were underway to implement this system in schools. in order to accomplish this, they had to draw on the experiences of other countries. that is why they carried out a worldwide investigation to determine how the idea of smart schools had been put into use. equipped with this knowledge, they produced the first plan for the smart schools in the same year (zain, atan, & idrus, 2004). an attempt was made to draw upon latest technology and to bring the latest technology to schools (chan, 2002; sspt, 1997). the objective of smart school concept was to help accomplish the ideals of education throughout the country, which could allow the country to regain lost ground and more important to meet new challenges. doing this involved making major changes in the culture and educational practices. thus, a totally new education system was sought for in which thinking, creativity and reflectivity were prized, rather than mechanical learning. learners were to determine the pace with which they learned, to collaborate and to reflect on their own learning. in addition, teaching materials did not comprise just the printed books. rather, electronic materials of various kinds were to support learning (umat, 2000). this system could bring about fundamental changes in education. among other things, the instructional methods were not fixed; students had the opportunity to make use of various educational technologies and interact with others in new ways. in addition, given the interactive learning environment available to them, the students had the opportunity to overcome the limitations of traditional schools (pituch & lee, 2006). social cognitive theory (bandura, 1986) is the theoretical framework adopted in order to gain insight into what brings about learning satisfaction in smart schools. this theory is widely used in research endeavors which somehow attempt to investigate the predictability of human behavior, which can, in turn, allow us to see how human behavior could be changed. essentially, this theory states that human beings progress in their behaviors through step-by-step interactions with the surrounding environment and that in order for such interaction to influence one’s behavior, the surrounding environment must be exposed to the cognition. moreover, this theory considers an interrelationship among cognitive, environmental factors and human behavior (wood & bandura, 1989). by cognition the reference is to individual cognition, affect and biological phenomena. in addition, social and physical environments comprise the environmental factors, which could have an impact on one’s behavior. cognitive mechanisms are the means by which environmental factors could influence behavior. it could be seen that according to social cognitive theorists, performance expectations and self-efficacy are two determining cognitive factors, with the latter being more important as it has a more important role in influencing one’s behavior. self-efficacy is defined as the way an individual judges him/herself and how (s)he views his/her confidence and ability in accomplishing something. needless to say, if an individual is not confident enough, there is no point in talking about his/her accomplishing something. self-efficacy determines how successfully somebody can accomplish something, how much effort (s)he will put into it, how long the perseverance will last especially in the face of difficulties and how much the person is resistant. it could be seen that the relationship between self-efficacy and performance expectation is reciprocal, suggesting that self-efficacy could have a significant impact on performance expectations, which can, in turn, influence self-efficacy. international electronic journal of elementary education, 9(3), 613-626, march, 2017 616 in addition, according to research evidence, environmental factors could have a significant impact on the performance and behavior of individuals. earlier, it was the case learning environment was seen as a function of physical and social factors. learning environment was later expanded by piccoli et al. (2001). they recognized a number of environmental factors which could distinguish e-learning environment from classroom environment and include technology, content, interaction, learning model, and learner control. it is possible to classify these factors into two rather distinct categories which are directly relevant to smart schools. the first category has to with technological environment and the second with social environment. the discussion so far has delineated the social cognitive theory which forms the theoretical underpinnings applicable to smart schools. as discussed above, learning satisfaction with smart schools could be discussed in terms of learners’ cognitive factors, technological factors and social factors. cognitive factors are related to learners’ cognitive beliefs and have an impact on learners’ behaviors in the use of smart schools. it is believed that two cognitive variables, namely computer self-efficacy and performance expectations are the factors determining human behavior in using an information system (is), (compeau & higgins, 1995; compeau, higgins, & huff, 1999; venkatesh, morris, davis, & davis, 2003). according to the theorists of the social cognitive theory, performance expectations are defined as the perceived consequences of a behavior. they also hold that these expectations determine individuals’ actions. individual judgements made about the outcomes of behaviors determine performance expectations. it is more likely that individuals will be involved in behaviors that they have every reason to believe that they will result in positive benefits. thus, it is less likely that they will perform behaviors which they believe will not produce favorable consequences. performance expectations have been defined as the degree to which learners believe that electronic learning system could help them to accomplish the performance goals they have in mind. the definition is consistent with the related concept of perceived usefulness, which is, in turn, taken from davis’s (1989) technology acceptance model (venkatesh et al., 2003). it is necessary to point out that there is some research evidence suggesting the influence of performance expectations on individual behavior when it comes to using computer systems (compeau & higgins, 1995, compeau et al., 1999, & venkatesh et al., 2003). specifically, it has already been shown that performance expectations positively correlate with students’ learning performance (bolt, killough & koh, 2001; kazu, demirkol, 2014) and learning satisfaction (martins & kellermanns, 2004; martirosyan, saxon & wanjohi, 2014; shih, 2006). personal beliefs lead to individual attitudes. for example, behavioral beliefs are directly related to a person’s intention to perform a certain behavior (ajzen & fishbein, 1980). consistent with the theory of reasoned action (taylor & todd, 1995), positive attitudes of users toward the information system could be measured by the degree of user acceptance, which could be used for the purpose of predicting the behaviours of the users while using the system. learning satisfaction is taken to be a good measure for user acceptance and that is why in computer-mediated learning studies, it is frequently used to gauge learners’ attitudes (chou & liu, 2005; piccoli et al., 2001). thus, it is possible to conceptualize learners’ attitudes toward smart schools as the learning satisfaction with smart schools. this satisfaction is defined as the totality of beliefs and attitudes of learners resulting from putting together all the benefits that learners receive from smart schools. another related cognitive factor relevant to the current study is self-efficacy, which has been defined as the beliefs which individuals hold about their capabilities to successfully carry out certain behaviors (bandura, 1986). social cognitive theorists opine that the role of socio-cognitive variables in predicting learning satisfaction / firoozi, kazemi & jokar 617 perceptions of self-efficacy toward a task are formed based on the cue received from several information sources: (1) past experience and familiarity with similar activities, (2) vicarious learning, (3) social support and encouragement, and (4) attitudes toward the task. according to bandura (1986), self-efficacy is specific to certain tasks and when it comes to measuring it, it is necessary to pay due attention to the context in which it occurs. consistent with the characterization given of self-efficacy, several studies have been carried out in which self-efficacy beliefs towards tasks such as computers and isrelated behaviors have been investigated (compeau & higgins, 1995; compeau et al., 1999). in line with this general definition given for self-efficacy, computer self-efficiency has been defined and the ability of individuals in using information technology to successfully carry out computer-related tasks or jobs (marakas, yi, & johnson, 1998). in the context of smart schools, self-efficacy could be defined as individuals’ confidence in using their ability to perform particular learning tasks. research findings have indicated that increased computer self-efficacy could help improve students’ initiative and persistence, which could, in turn, result in improved performance or outcome expectations (johnston, killion, & oomen, 2005; kuo, walker, schroder & belland, 2014; piccoli et al., 2001; wu et al., 2010), including attitude and behavioral intention(venkatesh & davis, 2000). as far as computer-mediated learning is concerned, research evidence suggests that improved computer self-efficacy could help promote students’ confidence in their computer-related abilities, which could in turn result in perceived positive performance expectations to the learning courses (bolt et al., 2001; jawahar & elango, 2001; santhanam, sasidharan & webster, 2008; shen, cho, tsai & marra, 2013; shih, 2006; wu et al., 2010). effective learning in smart schools could be attributed to such factors as the quality and dependability of an e-learning system, easy access to effective educational technologies, material content, and course-related information (piccoli et al., 2001). thus, it could be seen that the functionality of this system along with content features can be considered as important technological environment factors related to smart schools. these features are very influential in learners’ acceptance and use of smart schools. previous research findings have invariably shown that system functionality could significantly impact user beliefs in different computer-related contexts (igbaria, gamers, & davis, 1995; venkatesh & davis, 2000). for example, research evidence was clearly indicative of the fact that specific system functionality is a determinant, influencing the use of e-learning system (hong, thong, wong, & tam, 2002; pituch & lee, 2006; wu et al., 2010). system functionality has been defined as an individual’s perception of ability of an e-learning system in facilitating flexible access to instructional and related media (pituch & lee, 2006). in the context of the current study, system functionality could be defined as the perceived ability of smart schools in providing flexible access to instructional and the related media. these media allow students to have access to course materials and content, submit their homework assignments, to complete tests given and take quizzes online. content is utilized to recognize different formats and types of information (wu et al., 2010; zhang, keeling, & pavur, 2000). in the current study, content is defined as the technologybased materials and the information related to various courses which could be used by students in the context of smart schools. in smart schools, educational goals are accomplished through sharing and delivering course content, using different media such as tutorials, or web-based courses. given the variety of delivery methods, a major concern is to decided how to design and present content. in addition, the formats and types need to correspond to delivery or access in smart schools (so & brush, 2008). when it comes to designing smart schools, appropriate content features, and the effective design are essential considerations in such design (piccoli et al., 2001). drawing on the previous research (zhang et al., 2000), in the current study, content features are defined as the international electronic journal of elementary education, 9(3), 613-626, march, 2017 618 characteristics and presentation of content of courses and information in smart schools. some examples of content features in the environment of smart schools are: text, hypertext, graphics, audio and video, computer animations and simulations, embedded tests, and multimedia information. given the discussion so far, it could be seen that system functionality and content feature could directly impact perceived usefulness of is (hong et al., 2002; pituch & lee, 2006). in a number of empirical studies, it has been found that both content features (tajuddin, baharudin & hoon, 2013; wu et al., 2010; zhang et al., 2000) and system functionality (pituch & lee, 2006; shen et al., 2013; wu et al., 2010) directly affect the effectiveness of computer-mediated learning. in designing smart schools in which instruction is mediated through computers, designers increasingly pay close attention to facilitating interaction between humans and computers. this could be accomplished in the form of online collaboration, creation of virtual communities, and allowing instant messaging in the context of smart schools (graham, 2006). as far as group interaction is concerned, social environment factors, such as collaborative learning (francescato et al., 2006), learning climate (chou & liu, 2005; wu et al., 2010) and social interaction (johnston et al., 2005) are important determinants of beliefs students hold about using an e-learning system. in the same vein, it has been found that social interaction has a direct impact on the utilization of an e-learning system (kuo et al., 2014; pituch & lee, 2006; tajuddin et al., 2013). likewise, from a learning perspective, the interactions among students themselves, between teachers and students and learning collaboration are the keys to the effectiveness of a learning process. method the research design adopted was descriptive and predictive in nature. in the present study, satisfaction was considered as the criterion variable and computer self-efficiency, expected performance, cognitive variables, system efficiency and content characteristics (technological environment), interaction and learning environment (social environment) were considered as predictive variables of satisfaction. the population was all the male and female students studying in smart schools in shiraz in the school year 2014-2015. the sample was 400 students of primary schools, chosen through random multistage cluster sampling as follows. first out of the education districts in shiraz, district 2 and 3 were chosen randomly. out of district 2, 2 schools for males and females, and in each school, four classes were randomly chosen. the questionnaires were administered to all of them. the questionnaires were checked to make sure they were completed correctly and contained the essential information. given that 17 questionnaires were found to be incomplete, they were left out. thus, 383 questionnaires remained for further analysis. seven questionnaires were used to collect the data. except the computer self-efficiency questionnaire, which was in persian, all the other questionnaires were originally constructed in english. the scales were translated from english to persian and then from persian back to english by two university professors majoring in tefl. the translated versions were piloted on 100 students similar to the target group, which resulted in some modification of the form. in addition some items were deleted and some more were added. in addition, the scales were examined by four university professors majoring and psychology and educational sciences for content validity. the construct validity was established through correlating the score of each item with the total score. the following questionnaires were used in the study. computer self-efficiency questionnaire this questionnaire was developed by torkzadeh, koufteros and pflughoeft (2003). the original form of the questionnaire consists of 37 items, measuring the ability or inability of the role of socio-cognitive variables in predicting learning satisfaction / firoozi, kazemi & jokar 619 the respondents in using computers on a 5-point scale, ranging from 5 (completely agree) to 1 (completely disagree). zamanpour, khani and moradiani deizehrud (2013) established the reliability to be 0.80 and validity as satisfactory. in the current study, cronbach alpha was used to establish the reliability of the instrument, which was 0.92. performance expectancy questionnaire compiu and higging’s questionnaire (1995) was used to investigate the performance expectancy. this questionnaire consists of 4 items, measuring the success or failure of the respondents in performance expectancy. chiu, hsu, sun, lin, & sun (2005) reported the reliability index of the measure as 0.90 and validity as satisfactory. in the present study, the reliability of the scales, established through cronbach alpha was 0.73. system capability questionnaire pitich and lee’s capability questionnaire (2006) was used to measure system capability. this questionnaire has 6 items and measures the respondent’s ability in performance expectancy. pitich and lee (2006) reported the reliability of the questionnaire as 0.83 and validity as satisfactory. in the present study, cronbach alpha was used to establish the reliability of the scale, which was established to be 0.82. content specifications questionnaire pitich and lee’s content specifications questionnaire (2006) was used for this purpose. this questionnaire consists of 8 items. wu et al. (2010) established the reliability of the questionnaire as 0.89 and the validity as satisfactory. in the present study, the reliability, established through cronbach alpha, was 0.92. interaction questionnaire johnson et al.’s interaction questionnaire (2005) was used for this purpose. it has four items and measures interaction on a sale of 5. the reliability of the questionnaire was established by pitich and lee (2006), which was 0.90. they also reported the validity as satisfactory. in the current study, the reliability of the questionnaire established through cronbach alpha was 0.62. learning climate questionnaire chou and liu’s learning climate questionnaire (2005) was used for this purpose. it has 11 items and measures learning climate on a sale of 5. the reliability of the questionnaire was established by wu et al. (2010), which was 0.92. they also reported the validity as satisfactory. in the current study, the reliability of the questionnaire established through cronbach alpha was 0.83. learning satisfaction questionnaire chou and liu’s learning satisfaction questionnaire (2005) was used for this purpose. it has 10 items and measures satisfaction with learning on a sale of 5. the reliability of the questionnaire was established by chou and liu (2005), which was 0.86. they also reported the validity as satisfactory. in the current study, the reliability of the questionnaire established through cronbach alpha was 0.85. given that the impact of intelligent schools is more pronounced in some courses, the students were asked to consider the course of physical sciences in answering the questions. in addition to descriptive statistics of frequency, mean and standard deviation, use was made of pearson correlation coefficient and stepwise regression. international electronic journal of elementary education, 9(3), 613-626, march, 2017 620 results the first research question posed in the current study was “what are the most important predictive psychosocial variables of satisfaction with learning in smart schools?” table 1. zero -order correlation matrix between the variables variable md sd 1 2 3 4 5 6 7 computer selfefficacy 73.32 13.51 1 performance expectations 14.86 3.48 0.38** 1 system functionality 22.10 6.08 0.29** 0.55** 1 content feature 29.57 7.29 0.26** 0.63** 0.84** 1 learning climate 7.62 1.90 0.27** 0.56** 0.64** 0.64** 1 interaction 42.47 7.65 0.26** 0.54** 0.56** 0.55** 0.73** 1 learning satisfaction 40.30 7.12 0.60** 0.56** 0.42** 0.45** 0.50** 0.44** 1 note: **p < 0.01. table 1 depicts the results concerning the relationship between predictive and criterion variables. according to this table, cognitive factors (i.e., computer self-efficacy and performance expectancy), technological environment (i.e., system capability and content specifications) and social environment (i.e., interaction and learning atmosphere) have significant and positive relationship with learning satisfaction. the correlation between learning satisfaction and the variables in the study ranges from 0.41 to 0.59. the significant correlations are marked with an asterisk. given that cognitive factors(i.e., computer self-efficacy and performance expectancy), technological environment (i.e., system capability and content specifications) and social environment (i.e., interaction and learning atmosphere) have significant correlation with learning satisfaction, in further analysis, use was made of step-wise regression analysis, which required entering cognitive factors, followed by social factors (technological environment and social environment) into the analysis. the results in table 2 indicate that out of predictive variables, computer self-efficacy, performance expectancy and learning atmosphere could predict learning satisfaction. the other factors were left out as they could not predict learning satisfaction. it could be seen that the f value is significant for the variable of computer self-efficacy (p<0.01). this variable alone can account for 36% of the variance of learning satisfaction. adding performance expectancy to the regression analysis, a major portion of variance could be accounted for (i.e., 49%). it could thus be seen that almost 13% of the variance of learning satisfaction could be accounted for by performance expectancy, which is significant (p<0.01). finally, adding learning atmosphere, which belongs to social environment, the variance amounts to 53%, 4% of which belongs to the social environment. table 2. results of stepwise regression to predict satisfaction with learning step variable r r2 sd f p b β t p 1 computer self-efficacy 0.60 0.36 5.72 211.38 0.001 17.20 0.60 10.66 0.001 2 performance expectations 0.70 0.49 5.11 180.99 0.001 11.01 0.45 11.37 0.001 3 learning climate 0.73 0.53 4.93 139.64 0.001 6.18 0.43 11.34 0.001 the role of socio-cognitive variables in predicting learning satisfaction / firoozi, kazemi & jokar 621 discussion and conclusion the findings of the present study demonstrated that learning satisfaction in smart schools is the result of an interplay between various cognitive, technological and social factors. while technology is a pre-requisite for learning, it is in no way adequate and cannot bring about learning satisfaction. this finding is in line with the theorizing in social cognitive theory, which, essentially, is about the reciprocal relationship between cognitive, environmental and behavioural factors (bandura, 1986). the findings of the study suggest that computer self-efficacy, performance expectancy and learning environment, respectively, are the best predictors of learning satisfaction with smart schools. students with higher self-efficacy expose themselves more to the computer and the internet. they utilize their skills to achieve more in less time. in the current study, one of the findings was that computer self-efficacy, as an important cognitive variable, is significantly and positively correlated with learning satisfaction. this finding is in line with a number of studies (chou & liu, 2005; jin & lin, 2012; kuo et al., 2014; shen et al., 2013; wu et al., 2010). this finding is indicative of the fact that self-efficacy could be accounted for based on bandura’s anthropological concepts. this could be attributed to the fact that in the socio-cognitive approach, the emphasis is laid upon motivational processes, perception and feeling of individual about his or her merits. the findings of the study also showed that in addition to computer self-efficacy, performance expectancy is one of the strongest predictors of learning satisfaction (compeau & higgins, 1995; shen et al., 2013; wu et al., 2010). it is highly likely that individuals get involved in activities which, they believe, bring about their satisfaction. by developing their expectancy of the possible results of their actions, students are enabled to see the outcomes of their actions before actually doing them. they can also predict the possible rewards and punishments. students with higher efficacy have higher expectations to overcome the possible problems they are likely to encounter, and thus, demonstrate more learning satisfaction. this essentially means that the higher the self-efficacy, the more confident the student is that what he is doing is important and useful. this confidence could, in turn, bring about behaviors which lead to further progress and success. in the present study, learning climate was the third most important predictor of learning satisfaction. in some studies carried out in the same vein, it has been found that learning atmosphere does indeed lead to learning satisfaction (kuo et al., 2014; wu, wu, & tasi, 2014; wu et al., 2010). in the classroom situation, teachers are primarily responsible for establishing law and order, optimum use of resources and instructional materials, use of learning/teaching strategies and the evaluation of learning achievement. accomplishing all of these requires proper classroom management which is possible through proper planning and organization, leadership, supervision, control and evaluation. the result of the interaction among all these variables is learning environment which is conducive to learning. the learning atmosphere is a tangible feature which could be felt upon entrance to a classroom. the proper management finally determines achievement and final output in the classroom. it is necessary to point out that to make the most out of smart schools, learners should have the required computer competence. otherwise, they could fail in exercising control over their learning activities. given the indispensable role that computer competency plays in this regard, it is highly necessary for the authorities in smart schools to provide the necessary encouragement for learners to attend the relevant courses and acquire the necessary computer self-efficacy for effective academic functioning. needless to say, the specific needs of language learners in different settings should also be taken into account. international electronic journal of elementary education, 9(3), 613-626, march, 2017 622 active participation of learners in smart school environment could create a learning atmosphere conducive to easy and natural learning. if smart schools manage to create such a positive social environment, the odds are they could bring about interaction among the students themselves and with their teachers. this way, it is highly likely that they will interact more with their peers and their teachers, which can, in turn, improve the learning climate. learners in such a situation will perceive better performance on the part of smart schools and this can enhance learning satisfaction. in spite of the fact that the present study provides important insights into the factors contributing to learning satisfaction in smart schools, given the limitations of the study, the findings should be interpreted with caution. first, the research instrument was validated using participants in smart schools in the iranian context. given that, the findings could not be generalized to other contexts. it is, thus, necessary to use samples from other contexts to see whether the same findings will be replicated. second, measuring computer self-efficacy and performance expectancy required the use of selfreport instruments. given the shortcomings associated with such measures, the findings should be interpreted with caution. third, while the study sheds lights on some of the factors contributing to learning satisfaction in smart schools, it is in no way exhaustive in addressing all the possible contributing factors. future research might unravel other determinants of learning satisfaction in smart schools. opportunities for further research, mean that the findings of the study should be interpreted with caution. first, the research was validated using sample data gathered from the smart schools in the iranian context. given that the participants were chosen from a single country, the generalization of the findings should be done with utmost care. other samples from different nations, cultures, and contexts could be gathered in future research endeavors to confirm or refine the findings of the present study. second, as is the case with all self-report measures, in the current study, use was made of self-report instruments to measure computer self-efficacy and performance expectations. this needs to be taken into account when it comes to interpreting the findings. third, even if the current study sets a timely stage for future research in understanding the determinants of learning satisfaction in a smart schools environment, it is advisable to adopt a longitudinal design and to examine the relationships among the identified research variables. this could be a useful extension of the current study. finally, the list given of the determinants of learning satisfaction in smart schools is no way exhaustive. future studies could unravel additional determinants of student learning satisfaction with smart schools. • • • references bandura, a. (1986). social foundations of thought and action: a social cognitive theory. englewood cliffs, nj: prentice hall. bolt, m. a., killough, l. n., & koh, h. c. (2001). testing the interaction effects of task complexity in computer training using the social cognitive model. decision sciences, 32(1), 1-20. chan, f. m. (2002). ict in malaysian schools: policy and strategies. in a workshop on the promotion of ict in education to narrow the digital divide, tokyo, japan, 15–22 october. the role of socio-cognitive variables in predicting learning satisfaction / firoozi, kazemi & jokar 623 chiu, c. m., hsu, m. h., sun, s. y., lin, t. c., & sun, p. c. (2005). usability, quality, value and e-learning continuance decisions. computers & education, 45(4), 399-416. chou, s. w., & liu, c. h. (2005). learning effectiveness in a web‐based virtual learning environment: a learner control perspective. journal of computer assisted learning, 21(1), 65-76. compeau, d. r., & higgins, c. a. (1995). computer self-efficacy: development of a measure and initial test. mis quarterly, 19(2), 189-211. compeau, d., higgins, c. a., & huff, s. (1999). social cognitive theory and individual reactions to computing technology: a longitudinal study. mis quarterly, 23(2), 145-158. davis, f. d. (1989). perceived usefulness, perceived ease of use, and user acceptance of information technology. mis quarterly, 13(3), 319-340. davis, n. e., & tearle, p. (1999). a core curriculum for telematics in teacher training. tele-teaching 98 conference; vienna. retrieved november 28, 2003, from http://www.ex.ac.uk/telamatics/t3/corecurr/tteach98.htm francescato, d., porcelli, r., mebane, m., cuddetta, m., klobas, j., & renzi, p. (2006). evaluation of the efficacy of collaborative learning in face-to-face and computer-supported university contexts. computers in human behavior, 22(2), 163-176. graham, c.r. (2006). blended learning systems: definitions, current trends and future directions. in c. j. bonk, & c. r. graham (eds.), the handbook of blended learning: global perspectives, local designs (pp. 3-21). san francisco: pfeiffer. hashim, y., & man, r. (1999). multimedia-based teaching and learning: issue and application in smart education. instructional technology and smart education: preparation and challenges in the new millennium, malaysian association of educational technology, penang, 45-64. heba, e. d., & nouby, a. (2008). effectiveness of a blended e-learning cooperative approach in an egyptian teacher education programme. computers & education, 51(3), 988-1006. hong, w., thong, j. y., & wai-man wong, k. y. t. (2002). determinants of user acceptance of digital libraries: an empirical examination of individual differences and system characteristics. journal of management information systems, 18(3), 97-124. igbaria, m., guimaraes, t., & davis, g. b. (1995). testing the determinants of microcomputer usage via a structural equation model. journal of management information systems, 11(4), 87-114. jawahar, i. m., & elango, b. (2001). the effect of attitudes, goal setting and self-efficacy on end user performance. journal of organizational and end user computing, 13(2), 40-45. jin, d., & lin, s. (eds.). (2012). advances in future computer and control systems (vol. 1). springer science & business media. johnston, j., killion, j., & oomen, j. (2005). student satisfaction in the virtual classroom. internet journal of allied health sciences and practice, 3(2), 1-7. kazu, i. y., & demirkol, m. (2014). effect of blended learning environment model on high school students' academic achievement. tojet: the turkish online journal of educational technology, 13(1) 78-87. kirschner, p., & wopereis, i. g. (2003). mindtools for teacher communities: a european perspective. technology, pedagogy and education, 12(1), 105-124. kuo, y. c., walker, a. e., schroder, k. e., & belland, b. r. (2014). interaction, internet self-efficacy, and self-regulated learning as predictors of student satisfaction in online education courses. the internet and higher education, 20, 35-50. malaysian ministry of education, (moe), (1997). the malaysian smart schools implementation plan. international electronic journal of elementary education, 9(3), 613-626, march, 2017 624 marakas, g. m., yi, m. y., & johnson, r. d. (1998). the multilevel and multifaceted character of computer self-efficacy: toward clarification of the construct and an integrative framework for research. information systems research, 9(2), 126-163. martins, l. l., & kellermanns, f. w. (2004). a model of business school students' acceptance of a web-based course management system. academy of management learning & education, 3(1), 726. martirosyan, n. m., saxon, d. p., & wanjohi, r. (2014). student satisfaction and academic performance in armenian higher education. american international journal of contemporary research, 4(2), 1-5. ming, t. s., murugaiah, p., wah, l. k., azman, h., yean, t. l., & sim, l. y. (2010). grappling with technology: a case of supporting malaysian smart school teachers’ professional development. australasian journal of educational technology, 26(3), 400-416. ong, e. t., & ruthven, k. (2009). the effectiveness of smart schooling on students’ attitudes towards science. eurasia journal of mathematics, science & technology education, 5(1), 35-45. osguthorpe, r. t., & graham, c. r. (2003). blended learning environments: definitions and directions. quarterly review of distance education, 4(3), 227-33. pelgrum, w. j. (2001). obstacles to the integration of ict in education: results from a worldwide educational assessment. computers & education, 37(2), 163-178. piccoli, g., ahmad, r., & ives, b. (2001). web-based virtual learning environments: a research framework and a preliminary assessment of effectiveness in basic it skills training. mis quarterly, 25(4), 401-426. pituch, k. a., & lee, y. k. (2006). the influence of system characteristics on e-learning use. computers & education, 47(2), 222-244. rahimi, m., & yadollahi, s. (2011). the anxiety of high school students and its relationship with the use of computers and personal computer ownership. in proceedings of the fourth conference on e-learning, university of technology, tehran (vol. 18). santhanam, r., sasidharan, s., & webster, j. (2008). using self-regulatory learning to enhance elearning-based information technology training. information systems research, 19(1), 26-47. shen, d., cho, m. h., tsai, c. l., & marra, r. (2013). unpacking online learning experiences: online learning self-efficacy and learning satisfaction. the internet and higher education, 19, 10-17. shih, h. p. (2006). assessing the effects of self-efficacy and competence on individual satisfaction with computer use: an it student perspective. computers in human behavior, 22(6), 1012-1026. so, h. j., & brush, t. a. (2008). student perceptions of collaborative learning, social presence and satisfaction in a blended learning environment: relationships and critical factors. computers & education, 51(1), 318-336. soltani, m. (2012). the structure of smart schools in the educational system. journal of basic and applied scientific research, 2(6), 6250-6254. sspt, smart school project team, (1997). the malaysian smart school: conceptual blueprint. ministry of education. malaysia, kuala lumpur. sung, y. h., kwon, i. g., & ryu, e. (2008). blended learning on medication administration for new nurses: integration of e-learning and face-to-face instruction in the classroom. nurse education today, 28(8), 943-952. tajuddin, r. a., baharudin, m., & hoon, t. s. (2013). system quality and its influence on students’ learning satisfaction in uitm shah alam. procedia-social and behavioral sciences, 90, 677-685. taylor, s., & todd, p. (1995). decomposition and crossover effects in the theory of planned behavior: a study of consumer adoption intentions. international journal of research in marketing, 12(2), 137-155. the role of socio-cognitive variables in predicting learning satisfaction / firoozi, kazemi & jokar 625 torkzadeh, g., koufteros, x., & pflughoeft, k. (2003). confirmatory analysis of computer selfefficacy. structural equation modeling, 10(2), 263-275. umat, j. (2000, september). web-based dissemination and utilization of learning resources: tigerweb project. in asia and pacific seminar/workshop on educational technology (pp. 6-12). venkatesh, v., & davis, f. d. (2000). a theoretical extension of the technology acceptance model: four longitudinal field studies. management science, 46(2), 186-204. venkatesh, v., morris, m. g., davis, g. b., & davis, f. d. (2003). user acceptance of information technology: toward a unified view. mis quarterly, 27(3), 425-478. wood, r., & bandura, a. (1989). social cognitive theory of organizational management. academy of management review, 14(3), 361-384. wu, j. h., tennyson, r. d., & hsia, t. l. (2010). a study of student satisfaction in a blended e-learning system environment. computers & education, 55(1), 155-164. wu, j. h., tennyson, r. d., hsia, t. l., & liao, y. w. (2008). analysis of e-learning innovation and core capability using a hypercube model. computers in human behavior, 24(5), 1851-1866. wu, m. l., wu, p. l., & tasi, p. v. (2014). the relationship between classroom climate and learning satisfaction senior high school students. asian journal of management sciences & education, 3(1), 58-68. zain, m. z., atan, h., & idrus, r. m. (2004). the impact of information and communication technology (ict) on the management practices of malaysian smart schools. international journal of educational development, 24(2), 201-211. zamanpour, e., khani, m, h., & moradiani deizehrud, s, kh. (2013). the effect of computer anxiety on attitude towards e-learning: the mediating role of attitude and self-efficacy of computer and internet. journal of educational psychology, 28(9),78-98. zhang, x., keeling, k. b., & pavur, r. j. (2000). information quality of commercial web site home pages: an explorative analysis. in proceedings of the twenty first international conference on information systems (pp. 164-175). association for information systems. international electronic journal of elementary education, 9(3), 613-626, march, 2017 626 this page is intentionally left blank. www.iejee.com http://www.iejee.com/ international electronic journal of elementary education, 2012, 4(3), 473-487. issn:1307-9298 copyright © iejee www.iejee.com due to technical problems our readers had some problems with accessing this article. therefore the editorial board decided to republish it in a new volume of iejee (march, 2013 vol.5, issue 2 p. 109-124 ). student academic performance outcomes of a classroom physical activity intervention: a pilot study heather erwin  university of kentucky, usa alicia fedewa university of kentucky, usa soyeon ahn university of miami, usa received: june, 2012 / revised: june, 2012 / accepted: july, 2012 abstract a physical activity is beneficial to children’s health, yet academic pressures limit opportunities for students throughout the school day. the purpose of this study was to determine the effect of a classroom pa intervention on student academic performance outcomes. intervention participants (n=15) received daily pa breaks. reading and mathematics fluency, pa, grades, and standardized test scores were collected. effects of the intervention were examined using mixed-design anovas. intervention students had significantly higher reading fluency and mathematics scores post-intervention and higher means for standardized reading and mathematics scores as well as grades. short bouts of pa are important for improving cbm math and reading fluency scores. classroom teachers should be encouraged to devote time during academic learning to incorporate pa. keywords: curricular intervention, academic achievement, child health, curriculum-based measurement introduction throughout the last three decades, children have become increasingly more sedentary given the changes in our modernized environment (centers for disease control and prevention [cdc], 2009; stevens, to, stevenson & lochbaum, 2008). schools have been identified as locations in which physical activity (pa) promotion should occur (pate, davis, robinson, stone, mckenzie & young, 2006). no child left behind legislation has led to budget cuts and increased pressure for schools to increase standardized test scores, thereby leaving schools to reduce or even eliminate programs that could enhance pa in children (chomitz, slining, mcgowan, mitchell,  heather erwin, university of kentucky, 115 seaton bldg. lexington, kentucky, usa 40506-0219, phone: 859.257.5311 fax: 859.323.1090 e-mail: heather.erwin@uky.edu http://www.iejee.com/ mailto:heather.erwin@uky.edu international electronic journal of elementary education vol.4, issue 3, 473-487, 2012 474 dawson & hacker, 2009; coe, pivarnik, womack, reeves & malina, 2006; sibley & etnier, 2003). during school hours, the decrease of pa through limited time spent in physical education class or recess breaks contributes to the significant increase of sedentary behaviors in children. fewer children walk or ride their bicycles to school, and pa is increasingly being replaced with television watching, time spent on the internet, and the ubiquitous playing of video games (cdc, 2009; stevens et al., 2008; world health organization [who], 2009). experts recommend that children engage in 60 minutes or more of moderate to vigorous pa per day (strong, malina, blimkie, daniels, dishman, gutin…trudeau, 2005), yet studies have found that only 42% of children ages 6-11 years obtain this goal (troiano, berrigan, dodd, masse, tilert & mcdowell, 2008). when addressing health outcomes, typically the physical benefits are discussed; however, participating in physical activities has also shown a significant and positive effect on children’s cognitive functioning (fedewa & ahn, 2011; trudeau & shephard, 2010) and academic outcomes, with no detrimental effects to learning when time is taken away from instruction (sibley & etnier, 2003). researchers theorize that children receive cognitive benefits from participating in pa through a number of mediating processes (basch, 2010; trudeau & shephard, 2010). in a review of the literature, trudeau and shephard (2010) identified physiological influences such as greater arousal and enhanced levels of neurotrophins that stimulate neural connections in the hippocampus or learning center of children’s brains. further, additional psychosocial influences were also found in the literature, including an increased level of self-esteem and connectedness in schools, likely enhancing children’s ability to learn (trudeau & shephard, 2010). research attempting to identify the mediating relationships between children’s levels of pa and cognitive outcomes are limited by methodology employed in most of the studies (see fedewa & ahn, 2011), and thus the specific causal pathways between pa and children’s cognitions have yet to be identified. to date, most of the research examining the academic and cognitive effects of children’s pa has been measured through traditional, standardized tests or grades. although helpful in assessing the long-term effects of pa interventions on children’s cognitive outcomes, these traditional measures are not useful in assessing short-term gains or improvement as a result of the intervention (bricker, yovanoff, capt & allen, 2003; pretti-frontczak, 2002). given that pa interventions are not typically implemented over long durations of time (i.e., greater than one academic year), it is likely that effects of these interventions may be missed due to the measurements used to assess academic or cognitive gains (see macy, bricker & squires, 2005). curriculum-based measurements (cbms) one way of assessing academic gains over short periods of time is through the use of cbms. cbms are research-based assessments used in schools to ascertain student achievement on basic skills such as reading, math, writing, or spelling. in response to the limitations of traditional, standardized tests, cbms were developed in the 1970s as a means of monitoring children’s response to an intervention (see reschly, busch, betts, deno & long, 2009). these measures are well known and utilized by many teachers, school psychologists and other school personnel, as they are sensitive to small growth over time, are inexpensive, and translate into targeted goals for student achievement (macy et al., 2005; reschly et al., 2009). in a recent meta-analysis, reschly and colleagues (2009) were able to demonstrate the strong predictive validity (r = .67) of a particular type of cbm — oral reading fluency measures — on children’s future reading achievement and high-stakes standardized assessments. the cumulative evidence over the past three decades has been remarkable for these student academic performance outcomes of a classroom physical activity intervention: a pilot study / heather & erwin 475 measures given the relatively minimal resources in terms of cost and administration time. as pressures for high stakes testing increase and the time children spend engaged in pa decreases, considerable evidence is needed to demonstrate the effectiveness of classroom-based interventions that promote pa during the school day. yet despite this need, there is a dearth of research assessing the effectiveness of classroom-based pa interventions on children’s learning outcomes. these types of interventions have, however, been shown to significantly increase student pa levels and intensity in the classroom (cardon, de clercq, de bourdeaudhuij & breithecker, 2004; erwin, abel, beighle & beets, 2009; erwin, beighle, morgan & noland, 2011; gibson, smith, dubose, greene, bailey, williams…donnelly, 2008; liu, hu, ma, cui, pan, chang, et al., 2007; mahar, murphy, rowe, golden, shields & raedeke, 2006; stewart, dennison, kohl & doyle, 2004), as well as result in enhanced health outcomes such as improved bmi (liu et al., 2007), decreased back/neck pain (cardon et al., 2004), increased bone strength (macdonald, kontulainen, khan & mckay, 2007; macdonald, kontulainen, beck, khan & mckay, 2008), and noise reduction in the classroom. all of these positive outcomes result in an increased ability to concentrate (norlander, moas & archer, 2005). in the handful of studies assessing the impact of classroom-based pa on children’s academic performance, a number of benefits have been found. in particular, students have improved their behaviors (maeda & randall, 2003; mahar et al., 2006), concentration (lowden, powney, davidson & james, 2001; norlander et al., 2005), recognition and memory (della valle, dunn, dunn, geisert, sinatra & zenhausern, 1986), and reading and mathematical skills (fredericks, kokot & krog, 2006; uhrich & swalm, 2007) from physical activities performed in the classroom setting. embedded within the need to establish effective pa classroom interventions are measures that are sensitive to incremental changes in students’ academic growth. cbms will not only allow for progress monitoring but also assesses students on content in which they are being exposed through their instruction. by using measures that detect small changes in academic growth, it may be possible to more accurately detect whether pa is exerting a positive effect on children’s rate of learning or ability to retain material. thus, the purpose of the current pilot study was twofold. first, the study aimed to evaluate whether implementing curricular pa positively influenced children’s reading and mathematics achievement. second, the relationship of cbms with other standardized measures and grades used in assessing children’s reading and mathematics achievement will be measured in order to examine its potential for further use as an academic assessment tool in monitoring the effectiveness of pa interventions. because cbms have not been used before as a tool for measuring the impact of curricular pa on children’s academic outcomes, the present study serves as a pilot in investigating these questions. methods participants participants included 29 3rd grade students (mage = 8.87, sd = .54) from one southeastern elementary school (two classrooms). students were assigned to intervention (n = 16) and control (n = 13) conditions via a quasi-experimental design (by homeroom class) over a 20-week intervention period. one classroom served as the treatment, while another classroom served as the control. procedures were approved by the lead author’s institutional review board, and all parents/guardians signed an international electronic journal of elementary education vol.4, issue 3, 473-487, 2012 476 informed consent form, while all child participants completed an assent form to participate. instrument measurements of reading fluency, mathematics aptitude, grades, standardized test scores, classroom behavior, and school day pa were collected for all participants. two cbms.reading and mathematics fluency. specifically, curriculum-based reading fluency and mathematics measures are short progress measures designed to assess children’s reading and mathematical fluency (stecker & lembke, 2005). criterion validity coefficients for curriculum-based measurements are .80-.90 for reading and .between .60-.80 for mathematics (foegen, jiban & deno, 2007; jitendra, sczesniak & deatline-buchman, 2005). the oral reading fluency measures consisted of three reading passages wherein the child would read aloud for one minute, with the examiner recording the number of words correctly read for each passage. the median score out of the three reading passages was used for the child’s oral reading fluency score at each of the three time points. for mathematical fluency, grade-appropriate mathematical problems consisting of addition, subtraction, and basic multiplication were given on a classwide level to the students every two weeks. the students were given one minute to complete as many problems as they could with the number of correct responses used as their mathematical fluency score for each of the three time points. the psychometric properties of these instruments are described in the results section. grades. each classroom teacher also provided student grades for reading and mathematics at each of the three designated time points throughout the school year (december 2009, march 2010, may 2010). these were recorded as percentages (out of 100). standardized test scores. a number of different standardized tests were administered at different points throughout the school year. at the beginning and end of the school year, students took the test of primary reading outcomes (t-pro), which assesses phonics, vocabulary, comprehension, and research skills, as well as standardized testing and reporting (star) reading tests (r=0.93; http://www.cde.ca.gov/ta/tg/sr/technicalrpts.asp), which coincide with the accelerated reader program. at three time points (august 2009, december 2009, and march 2010), the students completed the discovery education assessment which assesses reading/language arts and mathematics. the outputs rate the students at levels, which are determined by the number of correct responses. these levels were recorded as: novice = 1, apprentice = 2, proficient = 3, and distinguished = 4. physical activity. to measure school day pa, participants wore a pedometer (walk4life, ls 2500, plainfield, il) for five consecutive school days, which is consistent with recommendations of monitoring periods for this age of children (vincent & pangrazi, 2002). this pedometer brand and model has been found to produce reliable and valid scores when used with children (beets, patton & edwards, 2005). procedures curriculum-based reading and mathematics fluency. during the baseline week (september, 2010), trained researchers administered the reading fluency probes and each classroom teacher administered the mathematics assessments for all students. the same procedures were followed once every two weeks using different forms (alternate passages and worksheets validated for the purposes of alternate use) of the reading and math standardized assessments designed to measure small progress over time (stecker & lembke, 2005). http://www.cde.ca.gov/ta/tg/sr/technicalrpts.asp student academic performance outcomes of a classroom physical activity intervention: a pilot study / heather & erwin 477 physical activity. to prevent reactivity with the pedometers, participants were given the opportunity to handle the pedometer, open it, and practice applying and removing it from their waistband prior to data collection. on the first day of data collection, each participant was assigned a pedometer to be used for the duration of the study. upon entering the classroom first thing in the morning, students were instructed to wear the pedometer on their waistband for the entire school day. immediately prior to dismissal, students returned their pedometer to the assigned bin. their data were recorded on a data sheet and reset for use the next day. this occurred during five days of baseline and one random day per week during the intervention. the classroom teacher of the intervention group led pa breaks for 20+ minutes per day. she maintained a log of all pa breaks she provided including the name and nature of the pa break as well as the duration and time period. each integrated pa break related to the math and reading content that was currently being taught. she participated in a 30-minute classroom pa training provided by an expert in classroombased pa. the training took place prior to baseline data collection. during the training, the definition of pa, the importance of pa in the classroom, and the connection between pa and academic performance were presented. additionally, managing children in pa settings and instructional means for presenting activity breaks to the students were emphasized. the intervention teacher was provided with promoting physical activity and health in the classroom activity break cards (pangrazi, beighle & pangrazi, 2009) and other web resources for classroom physical activities (i.e., energizers, pe central). in addition to the training and resources, the year prior to implementation of the intervention, the intervention classroom teacher took two graduate courses related to pa promotion with youth and teaching effectiveness in pa settings. the courses each addressed classroom pa breaks. the classroom teacher of the control group did not provide these pa breaks to her students. in lieu of the pa breaks, students in the control group continued with traditional, in-seat learning of the content. this included teacher-directed instruction, individual student seatwork, and partner or group work at desks. all students had the same amount of time allotted for physical education (two 30-minute classes per week) and recess (one 30-minute session per day). data analysis the validity of curriculum-based measurement (research question 1) was addressed by examining the extent to which a particular test (i.e., cbm) correlates with previously validated measures (i.e., standardized test scores and teacher-reported grades). therefore, scores from cbm’s, standardized test scores, and teacher reported grades were correlated and compared separately for reading and mathematics achievement. of nine repeated cbm measures, only scores at baseline, time 5, and time 8—which were collected at the same time points as standardized test scores and teacher grades of mathematics and reading—were correlated with the other two measures such that differences in the number of repeated measures were controlled and further students’ performance on different measures were compared concurrently. next, the intervention effects of pa on mathematical and reading performances (research question 2) were examined, using a series of mixed-design anovas. to control for differences in the number of repeated scores, the authors chose three cbm scores at baseline, time 5, and time 8. therefore, for reading and mathematics achievement, two sets of mixed-design anova—using time and measures as withinsubject factors and the type of intervention as a between-subject factor—were performed. international electronic journal of elementary education vol.4, issue 3, 473-487, 2012 478 results validity of curriculum-based measurement the concurrent validity of the curriculum-based measurement was evaluated based on inter-correlations among all three measures (i.e., cbm, standardized test scores, and teacher ratings of students’ grades) on reading and mathematics, separately. table 1 and table 2 show correlations among scores from cbm, standardized test scores, and teachers’ reported grades for the control group in the upper diagonal of the matrix and for the treatment group in the lower diagonal of the correlation matrix for mathematics and reading, respectively. t a b le 1 . c o rr e la ti o n s a m o n g c b m , s ta n d a rd iz e d t e s t s c o re s , a n d g ra d e s o n m a th e m a ti c s c b m t im e 1 c b m t im e 2 c b m t im e 3 t e s t t im e 1 t e s t t im e 2 t e s t t im e 3 g ra d e s t im e 1 g ra d e s t im e 2 g ra d e s t im e 3 n o te . ** p < . 0 1 ; * p < . 0 5 c b m : t im e 1 1 .3 5 9 .8 4 3 ** .4 2 3 .5 8 5 * .5 8 1 * .4 2 0 .2 1 6 .5 4 4 * c b m : t im e 2 .8 9 5 ** 1 .4 2 4 .3 1 1 .5 2 6 .1 6 5 .0 1 9 -. 2 5 4 .1 6 7 c b m : t im e 3 .8 8 3 ** .7 5 3 ** 1 .4 8 3 .5 4 8 * .2 8 0 .5 0 5 .1 0 7 .6 6 6 ** t e s t: t im e 1 .5 4 4 .4 6 4 .5 7 9 * 1 .7 8 2 ** .3 5 2 .0 0 2 .5 8 2 * .3 9 4 t e s t: t im e 2 .6 4 6 * .6 8 5 * .6 9 0 * .6 8 6 * 1 .4 0 2 .4 6 3 -. 0 1 3 .6 3 2 * t e s t: t im e 3 .7 4 1 ** .8 4 2 ** .7 2 2 ** .5 6 1 .8 5 1 ** 1 -. 2 7 7 .3 3 7 .0 6 7 g ra d e s : t im e 1 .3 7 0 .3 0 6 .1 3 0 .5 1 6 .2 1 9 .3 2 7 1 .0 7 0 .6 7 0 ** g ra d e s : t im e 2 .3 8 6 .4 2 2 .2 7 3 .6 6 1 * .7 0 8 ** .4 9 8 .5 0 8 1 .0 1 1 g ra d e s : t im e 3 .4 4 5 .2 3 5 .3 7 9 .6 9 7 ** .4 8 8 .3 3 6 .6 4 7 * .4 0 8 1 student academic performance outcomes of a classroom physical activity intervention: a pilot study / heather & erwin 479 t a b le 2 . c o rr e la ti o n s a m o n g c b m , s ta n d a rd iz e d t e s t s c o re s , a n d g ra d e s o n r e a d in g 0 c b m t im e 1 c b m t im e 2 c b m t im e 3 t e s t t im e 1 t e s t t im e 2 t e s t t im e 3 g ra d e s t im e 1 g ra d e s t im e 2 g ra d e s t im e 3 n o te . ** p < . 0 1 ; * p < . 0 5 c b m : t im e 1 1 .3 5 9 .8 4 3 ** .4 2 3 .5 8 5 * .5 8 1 * .4 2 0 .2 1 6 .5 4 4 * c b m : t im e 2 .8 9 5 ** 1 .4 2 4 .3 1 1 .5 2 6 .1 6 5 .0 1 9 -. 2 5 4 .1 6 7 c b m : t im e 3 .8 8 3 ** .7 5 3 ** 1 .4 8 3 .5 4 8 * .2 8 0 .5 0 5 .1 0 7 .6 6 6 ** t e s t: t im e 1 .5 4 4 .4 6 4 .5 7 9 * 1 .7 8 2 ** .3 5 2 .0 0 2 .5 8 2 * .3 9 4 t e s t: t im e 2 .6 4 6 * .6 8 5 * .6 9 0 * .6 8 6 * 1 .4 0 2 .4 6 3 -. 0 1 3 .6 3 2 * t e s t: t im e 3 .7 4 1 ** .8 4 2 ** .7 2 2 ** .5 6 1 .8 5 1 ** 1 -. 2 7 7 .3 3 7 .0 6 7 g ra d e s : t im e 1 .3 7 0 .3 0 6 .1 3 0 .5 1 6 .2 1 9 .3 2 7 1 .0 7 0 .6 7 0 ** g ra d e s : t im e 2 .3 8 6 .4 2 2 .2 7 3 .6 6 1 * .7 0 8 ** .4 9 8 .5 0 8 1 .0 1 1 g ra d e s : t im e 3 .4 4 5 .2 3 5 .3 7 9 .6 9 7 ** .4 8 8 .3 3 6 .6 4 7 * .4 0 8 1 international electronic journal of elementary education vol.4, issue 3, 473-487, 2012 480 as shown in the shaded areas of table 1, mathematics scores from cbm had small to large correlations with standardized test scores on mathematics for both control and intervention groups. however, the correlations between cbm scores and grades were small and insignificant. as shown in the shaded areas of table 2, reading scores from cbm, standardized scores, and grades were correlated with a small to large magnitude. patterns of correlations among three measures on reading were similar between intervention and control groups, showing lower correlations between cbm scores and grades, yet higher correlations between cbm scores and standardized test scores. intervention effect on mathematics achievement a preliminary analysis was first performed to determine whether any preexisting differences on mathematics scores existed between control and intervention groups. results from three sets of independent t-tests showed that the intervention group was not statistically different from the control group on cbm scores (t(27) = -.87, p = .39), standardized test scores (t(25) = -.24, p = .81), or teacher’s reporting of students’ grades (t(25)=-2.52, p = .05), indicating no statistically significant pre-existing differences at the baseline measures of mathematics between the two groups. mauchly’s tests indicated that the assumption of sphericity was violated for the main effects of measure, χ2(2) = 10.94, p = .004, and interaction effect between measure and time, χ2(9) = 23.58, p = .005, but not for the main effect of time (χ2(2) = 2.43, p = .30). therefore, degrees of freedom were corrected using the huynh-feldt estimates of sphericity (gamst, myers & guarino, 2008) for measure (ε = .78) and interaction between measure and time (ε = .77). as shown in table 3, a mixed-design anova showed that the main effects of time (f(2, 44) = 15.52, p < .01, partial 2  = .41), measures (f(1.56, 34.36) = 2716.32, p < .01, partial 2  =.99), and intervention (f(1, 22) = 7.49, p = .01, partial 2  =.25) were statistically significant. further, two-way interactions between time and measure (f(3.08, 67.84) = 8.67, p < .01, partial 2  = .28) and three-way interactions among time, measure, and intervention (f(3.08, 67.84) = 6.49, p < .01, partial 2  = .23) were statistically significant. because a higher-order interaction supersedes lower-order effects (gamst, myers & guarino, 2008), follow-up tests were performed to further investigate the three-way interaction among time, measure, and intervention in detail. tests of simple effects showed a significant two-way interaction effect between time and intervention for cbm scores (f(2,26) = 10.31, p <.01), but not for standardized test scores (f (2,21) = 2.63, p = .10) or teachers’ reported grades (f(2,23) = 1.59, p = .23) . as shown in figure 1, the intervention group (m = 24.56, sd = 2.21) scored significantly higher on cbm scores than the control group (m = 13.69, sd = 2.45) at time 3 (mdiff = 10.87, p = .003), but not time 1 (mdiff = 2.75, p = .39) or time 2 (mdiff = 2.16, p = .49). table 3. results from mixed-design anova on mathematics source ss df ms f p partial 2  time 563.18 2 281.59 15.52 <.01 .41 time * intervention 76.95 2 38.48 2.12 .13 .09 error (time) 798.31 44 18.14 student academic performance outcomes of a classroom physical activity intervention: a pilot study / heather & erwin 481 table 3 (continue). results from mixed-design anova on mathematics source ss df ms f p partial 2  measure 330133.93 1.56 211367.38 2716.32 <.01 .99 measure * intervention 341.37 1.56 218.56 2.81 .09 .11 error (measure) 2673.81 34.36 77.81 time * measure 432.63 3.08 140.30 8.67 <.01 .28 time * measure * intervention 323.74 3.08 104.99 6.49 .<.01 .23 error (time * measure) 1097.19 67.84 16.17 intervention 872.02 1 872.02 7.49 .01 0.25 error 2562.85 22 116.49 figure 1. mathematics across time by intervention groups intervention effect on reading achievement results from three sets of an independent t-test indicated no pre-existing differences between control and intervention groups on all three measures of reading achievement, cbm standardized test scores teachers’ reported grades international electronic journal of elementary education vol.4, issue 3, 473-487, 2012 482 t(27) = -1.48, p = .15 for the cbm scores, t(26) = -.97, p = .34 for standardized test scores, or t(25) = -1.39, p = .18 teacher’s rating of students’ grades. mauchly’s tests indicated that the assumption of sphericity had been violated for the main effects of measure, χ2(2) = 80.06, p < .01, and interaction effect between measure and time, χ2(9) = 48.35, p < .01, but not for the main effect of time (χ2(2) = .22, p = .90). therefore, degrees of freedom were corrected using the huynh-feldt estimates of sphericity (gamst, myers & guarino, 2008) for both measure (ε = .53) and interaction between measure and time ( ε = .67). a mixed-design anova showed statistically significant main effects of time (f(2, 40) = 14.39, p < .01, partial 2  = .42), intervention (f(1, 22) = 353.51, p < .01, partial 2  = .95) and measures (f(1.06, 21.23) = 95.27, p < .01, partial 2  =.83) as well as two-way interactions between time and measure, f(2.67, 53.29) = 7.66, p < .01, partial 2  =.28. the significant two-way interaction between time and measure was examined by testing the simple effects of measures at each time point. pairwise comparisons using a bonferroni adjustment indicated that the cbm scores were statistically higher than standardized test scores for all three time points (mdiff = 79.46, p <.01 for time 1; mdiff = 87.41, p <.01 for time 2; mdiff = 92.46, p <.01 for time 3). similarly, students scored higher on standardized test scores than teachers’ reported grades for all three time points (mdiff = 88.50, p <.01 for time 1; mdiff = 90.73, p <.01 for time 2; mdiff = 91.91, p <.01 for time 3). however, no differences were found between cbm scores and teachers’ reported grades for any of the three time points. table 4. results from mixed-design anova on reading source ss df ms f p partial 2  time 1015.12 2 507.56 14.39 <.01 .42 time * intervention 11.40 2 5.70 0.16 .85 .01 error (time) 1410.59 40 35.26 measure 344423.48 1.06 324414.61 95.27 .<.01 .83 measure * intervention 7829.28 1.06 7374.45 2.17 .16 .10 error (measure) 72308.34 21.23 3405.38 time * measure 1015.48 2.66 381.11 7.66 <.01 .28 time * measure * intervention 11.02 2.66 4.14 0.08 .96 .00 error (time * measure) 2649.72 53.29 49.72 intervention 757396.55 1.00 757396.55 353.51 <.01 0.95 error 5912.75 1.00 5912.75 2.76 .11 0.12 student academic performance outcomes of a classroom physical activity intervention: a pilot study / heather & erwin 483 figure 2. reading across time by intervention groups discussion the present study sought to evaluate the potential effectiveness of implementing curricular pa on children’s reading and mathematics achievement. second, the authors examined the validity of curriculum-based measures with other standardized measures and grades in assessing children’s reading and mathematics achievement. each of these questions will be discussed with respect to the findings of the current study as well as implications for classroom teachers. the results of the current study suggest that curricular pa had a significantly positive effect on children’s cbm reading and mathematics scores. given the short increments of time in which these measures were administered, it is likely that cbms were better able to pick up small increments of growth in children’s achievement than were standardized test scores. the results of the additional pa on children’s reading and mathematics scores that was implemented in the treatment group confirm the general body of research in this area, suggesting that pa may enhance children’s cognitive outcomes (fedewa & ahn, 2011; sibley & etnier, 2003; trudeau & shephard, 2010). when examining mathematics scores, in particular, cbm scores for the control group peaked at time 2 and dropped at time 3, whereas the intervention group continued to improve upon their outcomes. with regard to standardized test scores, both groups scored similarly at time 1; the control group dropped at time 2, and both groups improved at time 3. again, the intervention group showed a consistent trend of improvement. teachers’ reported grades showed a jump at time 2 and a slight drop at time 3 for both groups. these trends suggest that pa enhanced learning for those students in the intervention group. cbm standardized test scores teachers’ reported grades international electronic journal of elementary education vol.4, issue 3, 473-487, 2012 484 for reading, students in both groups showed improvement from time 1 to time 3 on cbm scores and teachers’ reported grades. standardized test scores for reading peaked at time 2 and dropped for both groups at time 3; however, the control group demonstrated a greater drop in scores than the intervention group. thus, the pa intervention appeared to be more beneficial for mathematics. one possible explanation is that the pa breaks may have been more geared towards mathematics content thus leading to greater improvements in that area. the last hypothesis examined whether cbms are valid measures of assessing students’ achievement over time in comparison to standardized test scores and teacher grades. as mentioned earlier, although standardized test scores may be helpful in assessing the long-term effects of pa interventions on children’s cognitive outcomes, these traditional measures are not useful in assessing short-term gains or improvement as a result of the intervention (bricker et al., 2003; pretti-frontczak, 2002). in the vast majority of pa intervention research, durations of curricular interventions are not typically implemented for longer than one academic year, thus necessitating a measure that can capture small increments of achievement growth. in the present study, it was hypothesized that cbms would provide a more accurate indicator of student academic progress than standardized test scores given the short 20 week duration of the study. it was discovered that students in the treatment group had significantly higher scores in reading and mathematics when assessed by cbms, but that this difference did not reach significance when compared by standardized test scores or teacher grades. further, the reading and mathematics cbms were moderately to largely correlated with the standardized test scores while teacher grades were not correlated with either cbm or standardized test scores. these are very promising findings, as the inclusion of cbm assessments in measuring the effects of curricular pa interventions should be strongly considered. cbm assessments are short, accurate, and reliable measures that have been used to assess student academic progress for over four decades (reschly et al., 2009). perhaps by using standardized test scores or teacher grades as indicators of student academic progress, the beneficial effects of pa interventions have been missed in the literature. the current study provides preliminary evidence for the benefits of using cbms in measuring students’ academic growth as a result of curricular pa interventions. in conclusion, allotting 20+ minutes per day to provide curricular-based pa breaks to students does not appear to detract from student performance outcomes, behavior or pa levels. in fact, reading and math scores (as measured by cbms) significantly improved, while pa levels showed a trend of increasing due to this type of intervention. elementary teachers should be encouraged to incorporate pa during their lessons in the classroom setting due to the multiple positive student outcomes. • • • acknowledgements the authors would like to thank lydia childress and her students for participating in this research project. we enjoyed her passion for doing physical activity with her students. references student academic performance outcomes of a classroom physical activity intervention: a pilot study / heather & erwin 485 ahamed, y., macdonald, h., reed, k., naylor, p. j., liu-ambrose, t., & mckay, h. (2007). school-based physical activity does not compromise children's academic performance. medicine and science in sports and exercise, 39(2), 371-376. barros, r. m., silver, e. j., & stein, r. e. k. (2009). school recess and group classroom behavior. pediatrics, 123, 431-436. basch, c.e. (2010). healthier students are better learners: a missing link in school reforms to close the achievement gap. research review (6), 1-107. beets, m. w., patton, m. m., & edwards, s. (2005). the accuracy of pedometer steps and time during walking in children. medicine and science in sports and exercise, 37(3), 513-520. bricker, d., yovanoff, p., capt, b., & allen, d. (2003). use of a curriculum-based measure to corroborate eligibility decisions. journal of early intervention, 26, 20-30. cardon, g., de clercq, d., de bourdeaudhuij, i., & breithecker, d. (2004). sitting habits in elementary schoolchildren: a traditional versus a "moving school". patient education and counseling, 54(2), 133-142. centers for disease control and prevention. (2010). association between school-based physical activity, including physical education, and academic performance. atlanta, ga: author. centers for disease control and prevention. (2009). physical activity for everyone. retrieved march 23, 2012 from http://www.cdc.gov/physicalactivity/everyone/guidelines/index.html chomitz, v. r., slining, m. m., mcgowan, r. j., mitchell, s. e., dawson, g. f., & hacker, k. a. (2009). is there a relationship between physical fitness and academic achievement? positive results from public school children in the northeastern united states. journal of school health, 79, 30-37. coe, d. p., pivarnik, j. m., womack, c. j., reeves, m. j., & malina, r. m. (2006). effect of physical education and activity levels on academic achievement in children. medicine and science in sports and exercise, 38(8), 1515-1519. della valle, j., dunn, k., dunn, r., geisert, g., sinatra, r., & zenhausern, r. (1986). the effects of matching and mismatching students' mobility preferences on recognition and memory tasks. journal of educational research, 79(5), 267-272. erwin, h. e., abel, m. g., beighle, a., & beets, m. w. (2009). promoting children's health through physically active math classes: a pilot study. health promotion practice, 12(2), 244251. erwin, h. e., beighle, a., morgan, c. f., & noland, m. p. (2011). effect of a low-cost, teacherdirected classroom intervention on elementary students' physical activity. journal of school health, 81, 455-461. fedewa, a.l., & ahn, s. (2011). the effects of physical activity and physical fitness on children’s cognitive outcomes: a meta-analysis. research quarterly for exercise and sport, 82(3), 521-535. foegen, a., jiban, c., & deno, s. (2007). progress monitoring measures in mathematics: a review of the literature. journal of special education, 41(2), 121-139. fredericks, c. r., kokot, s. k., & krog, s. (2006). using a developmental movement programme to enhance academic skills in grade 1 learners. south african journal for research in sport, physical education, and recreation, 28(1), 29-42. gamst, g., myers, l. s., & guarino, a. j. (2008). analysis of variance designs: a conceptual and computational approach with spss and sas. new york, ny: cambridge university press. http://www.cdc.gov/physicalactivity/everyone/guidelines/index.html javascript:__dolinkpostback('','mdb%7e%7epbh%7c%7cjdb%7e%7epbhjnh%7c%7css%7e%7ejn%20%22journal%20of%20special%20education%22%7c%7csl%7e%7ejh',''); international electronic journal of elementary education vol.4, issue 3, 473-487, 2012 486 gibson, c. a., smith, b. k., dubose, k. d., greene, j. l., bailey, b. w., williams, s. l., et al. (2008). physical activity across the curriculum: year one process evaluation results. international journal of behavioral nutrition and physical activity, 5, 36. jitendra, a. k., sczesniak, e., & deatline-buchman, a. (2005). an exploratory validation of curriculum-based mathematical word problem solving tasks as indicators of mathematics proficiency for third graders. school psychology review, 34, 358–371. liu, a. l., hu, x. q., ma, g. s., cui, z. h., pan, y. p., chang, s. y., et al. (2007). report on childhood obesity in china (6) evaluation of a classroom-based physical activity promotion program. biomedical environmental science, 20(1), 19-23. lowden, k., powney, j., davidson, j., & james, c. (2001). the class moves! pilot in scotland and wales: university of glasgow. macdonald, h. m., coupe, d. m. l., kontulainen, s. a., & mckay, h. a. (2007). a school-based physical activity intervention positively affects change in imax in preand early pubertal boys. journal of bone and mineral research, 22, s12-s12. macdonald, h. m., kontulainen, s. a., petit, m. a., beck, t. j., khan, k. m., & mckay, h. a. (2008). does a novel school-based physical activity model benefit femoral neck bone strength in preand early pubertal children? osteoporosis international, 19(10), 1445-1456. macy, m.g., bricker, d.d., & squires, j.k. (2005). validity and reliability of a curriculum-based assessment approach to determine eligibility for part c services. journal of early intervention, 28(1), 1-16. maeda, j. k., & randall, l. m. (2003). can academic success come from five minutes of physical activity? brock education journal, 13(1), 14-22. mahar, m. t., murphy, s. k., rowe, d. a., golden, j., shields, a. t., & raedeke, t. d. (2006). effects of a classroom-based program on physical activity and on-task behavior. medicine and science in sports and exercise, 38(12), 2086-2094. national center for health statistics. (2009). health, united states, 2008. hyattsville, md: u.s. department of health and human services. norlander, t., moas, l., & archer, t. (2005). noise and stress in primary and secondary school children: noise reduction and increased concentration ability through a short but regular exercise and relaxation program. school effectiveness and school improvement, 16(1), 9199. ogden, c. l., yanovski, s. z., carroll, m. d., & flegal, k. m. (2007). the epidemiology of obesity. gastroenerology, 132, 2807-2102. pangrazi r,, beighle a,, & pangrazi d. (2009). promoting physical activity and health in the classroom. san francisco, ca: pearson benjamin cummings. pate, r.r., davis, m. g., robinson, t. n., stone, e. j., mckenzie, t. l., & young, j. c. (2006). promoting physical activity in children and youth: a leadership role for schools. circulation, 114, 1214-1224. pretti-frontczak, k.l. (2002). using curriculum-based measures to promote a linked system approach. assessment for effective intervention, 27(4), 15-21. reschly, a.l., busch, t.w., betts, j., deno, s.l., & long, j.d. (2009). curriculum-based measurement oral reading as an indicator of reading achievement: a meta-analysis of the correlational evidence. journal of school psychology, 47, 427-469. sibley, b. a., & etnier, j. l. (2003). the relationship between physical activity and cognition in children: a meta-analysis. pediatric exercise science, 15, 243-256. stecker, p. m., & lembke, e. s. (2005). advanced applications of cbm in reading: instructional decision making manual. retrieved march 20, 2010 from http://www.studentprogress.org/library/training/cbmmath/advancedreading/advrdgmanual formattedsept29.pdf student academic performance outcomes of a classroom physical activity intervention: a pilot study / heather & erwin 487 stevens, t. a., to. y., stevenson, s. j., & lochbaum, m. r. (2008). the importance of physical activity and physical education in the prediction of academic achievement. journal of sport behavior, 31, 368-388. retrieved march 20, 2012 from http://search.ebscohost.com.ezproxy.uky.edu/login.aspx?direct=true&db=aph&an=350781& site=ehost-live&scope=site stewart, j. a., dennison, d. a., kohl, h. w., & doyle, j. a. (2004). exercise level and energy expenditure in the take 10! in-class physical activity program. journal of school health, 74(10), 397-400. strong, w. b., malina, r. m., blimkie, c. j. r., daniels, s. r., dishman, r. k., gutin, b….trudeau, f. (2005). evidence based physical activity for school-age youth. journal of pediatrics, 146, 732-737. troiano, r. p., berrigan, d., dodd, k. w., masse, l. c., tilert, t., & mcdowell, m. (2008). physical activity in the united states measured by accelerometer. measurement and science in sports and exercise, 40, 181-188. trudeau, f. & shephard, r.j. (2010). relationships of physical activity to brain health and the academic performance of schoolchildren. american journal of lifestyle medicine, 4, 138150. uhrich, t. a., swalm, r. l. (2007). a pilot study of a possible effect from a motor task on reading performance. perceptual and motor skills, 104, 1035-1041 vincent, s. d., & pangrazi, r. p. (2002). determining baseline physical activity levels in children. medicine and science in sports and exercise, 14, 432-441. world health organization. (2009). global strategy on diet, physical activity and health. retrieved march 10, 2012 from http://www.who.int/dietphysicalactivity/childhood/en/ http://search.ebscohost.com.ezproxy.uky.edu/login.aspx?direct=true&db=aph&an=350781&site=ehost-live&scope=site http://search.ebscohost.com.ezproxy.uky.edu/login.aspx?direct=true&db=aph&an=350781&site=ehost-live&scope=site http://www.who.int/dietphysicalactivity/childhood/en/ international electronic journal of elementary education december 2022, volume 15, issue 2, 75-85 75 © 2022 published by kura education & publishing. this is an open access article under the cc byncnd license. (https://creativecommons. org/licenses/by/4.0/) copyright © www.iejee.com issn: 1307-9298 ‘everybody is sick and there won’t be a cure’: the power of privileging u.s. student voices through covid-19 drawings meredith jonesa,*, carol mcnultyb abstract introduction the covid-19 pandemic and the subsequent lockdown was particularly challenging for elementary students who experienced disruptions in almost all aspects of their daily activities. our study addressed a dearth of u.s. studies documenting the adverse effects the pandemic created and continues to have for children by analyzing their drawings. this study was conducted post-lockdown after students resumed normal school operations. the mosaic approach framed this case study research that was conducted using drawings created by sixteen students in grades prekindergarten through fifth grade (ages 4 to 10 years old). students drew what came to mind when thinking of covid-19. we conducted a content analysis of the data and determined four dominant themes: (1) symptoms related to covid-19, (2) evidence of psychological or emotional responses, (3) health and safety precautions, and (4) depictions of the virus cell. we assert children’s drawings as a useful tool for educators to create opportunities to connect with students and potentially reduce their anxieties about issues, including those beyond the pandemic. this will be of interest to educators who may not have considered the power of children’s drawings as a means to provide voice about current issues. it will also add to the global knowledge base about children’s perceptions of covid-19 and the resulting lockdown. in this article, we explore the use of children’s drawings as a means of communicating meaning assigned to the pandemic, a very confusing time for many school-aged children. first, we explore the existing global literature related to the pandemic, and then discuss children’s drawings as a data source, which has long been used in the health care industry but is less prevalent in the educational community. using the mosaic approach (clark & moss, 2001) to frame the study, we employed an adapted draw and write technique (pridmore & bendelow, 1995) to analyze the drawings of sixteen participants in the united states, aged keywords: children’s drawings, covid-19, social emotional support received : 8 november 2022 revised : 7 december 2022 accepted : 19 december 2022 doi : 10.26822/iejee.2023.280 a,* corresponding author: meredith jones, university of north carolina at wilmington, wilmington, usa. e-mail: jonesmk@uncw.edu orcid: https://orcid.org/0000-0002-3469-1642 b carol mcnulty, university of north carolina at wilmington, wilmington, usa. e-mail: mcnultycp@uncw.edu orcid: https://orcid.org/0000-0003-0898-0050 december 2022, volume 15, issue 2, 75-85 76 4-10 years old. content analysis of the data revealed four overarching themes which included: symptoms, psychological, health and safety precautions, and virus cell depiction. we discuss the implications of providing drawing as a means for students to powerfully express their voice. literature review during the covid-19 pandemic, researchers worldwide studied children’s perceptions via their drawings. in spain, researchers (e.g., gonzález-carlo, varea, & garcía-monge, 2022; mondragon et al., 2022) investigated children’s drawings about the lockdown and social distancing. both studies found that children had major disruptions to their routines during lockdown that led to isolation, negatively impacting their social well-being. gonzález-calro et al. noted children reported they missed playing outside with their friends and visiting their grandparents the most. mondragon et al. call for more attention to children’s voices by placing greater emphasis on social and inclusive policies to alleviate possible effects of isolation during future lockdown situations. cornaggia et al. (2021) conducted a similar study on children’s drawings during the pandemic lockdown in italy. in their study, children experienced tranquility and emotional wellbeing while spending time with family during the pandemic. however, children also expressed feelings of sadness and loneliness due to not being able to interact with their friends (cornaggia). in france, in a study of children aged 5-17, researchers (martinerie et al., 2021) found that children had relatively accurate depictions of covid-19 and concluded that they had developed coping mechanisms for fears of covid-19. researchers in south africa explored children’s wellbeing and safety due to social isolation through drawings during the pandemic (haffejee & levine, 2020). in the united kingdom, bhandari (2021) explored areas of worry in children’s drawings during the pandemic. finally, in sweden, rydström et al. (2022) explored how children accessed and perceived information during the covid-19 pandemic. through all of these studies, it is evident that using children’s drawings to illuminate the artist’s perceptions of how they assign meaning to the world is useful. to our knowledge, very little research has been conducted in the united states. this study adds to this growing body of international research on children’s drawings during the covid-19 pandemic. children’s drawings as a data source drawing is a common activity for elementary children. what may appear to some as a simple byproduct of passing time, to others it can offer a tool for unlocking children’s communication about their worlds and the meanings they assign to these worlds. children’s drawings go well past the production of art, and instead, interweave their thoughts and emotions (deguara, 2019), providing insights into children’s way of thinking and how they make sense of the world around them. drawing provides a child-centered approach to sharing those lived experiences that provide a window into a child’s emotional and cognitive states (bland, 2012; mitchell, 2006; zlateva, 2019). drawing has been described as an effective means for children to reveal the complexities of their lived experiences (mitchell, 2006; walker, 2007) and is a natural way for children to express themselves without the typical filters of language. children’s drawings, however, are sometimes missed as a resourceful tool in understanding their social and emotional development in the elementary years. children may be asked to draw as part of a formal lesson, for example a science lesson, or in lessons with the school’s art teacher, but these drawings have a different focus. such activities emphasize academic performance. children’s drawings have been described as a means of gaining “informed insight” (bradding & horstman, 1999, p. 175), and are powerful tools for qualitative research, particularly when interpreted by the child instead of adults (bradding & horstman; leonard, 2006; günindi, 2015). meanings are often attached to drawings and though sometimes more is drawn than can be described or translated into words, children can convey significant meaning to their drawings through writing or dictation (dyson, 1986). several disciplines have used children’s drawings in a variety of ways to better understand them. in education, children’s drawings have been used to measure intelligence such as with the draw-aperson tests. in psychology, they have been used as therapeutic devices to access children’s innermost feelings to assess their reactions to stress. in sociology, researchers have turned more frequently to the use of children’s art as a means of visual communication (leonard, 2006). in the field of health care, children’s drawings have been used to reveal their perceptions of their own health issues and those of others (bradding & horstman, 1999), and their narratives offer valuable insights into children’s wants and needs (burgess et al., 2022). though certainly not universal, most children do seem to enjoy drawing and are considered competent in the activity (leonard, 2006). generally, drawing is viewed as an engaging, non-threatening and autonomous activity for children (bradding & horstman, 1999), while at the same time, it is also recognized as a reliable means to gather data (zlateva, 2019). it is considered a developmentally appropriate means of collecting data with children as opposed to more common means of data collection such as surveys or interviews (farmer et al., 2018). this study made use of children’s drawings to better understand how they were making 77 ‘everybody is sick and there won’t be a cure’: the power of privileging u.s. student voices / jones & mcnulty sense of the covid-19 virus post lockdown and upon reentry to school in 2022. though the immediate threat of covid-19 has significantly subsided, the information gleaned from these data provide important insight to educators and caregivers who continue to work with children who were indelibly marked by these events. how children conceptualize sickness the health industry has long turned to children’s drawings to better understand how children perceive illness. children generally seem to categorize illness based on three characteristics or bodies of information: biomedical (e.g., the absence or presence of diseases); psychosocial (e.g., healthy people are happy and sick people are sad); and representations of healthy lifestyles (e.g., smoking is not a healthy lifestyle, but exercising is), (mouratidi et al., 2015). undoubtedly, a child’s age plays an important factor in their perception of health and illness, and with increased maturity comes more complex, abstract, and scientific explanations of illness (mouratidi). children’s perceptions are shaped not only by their personal experiences and contexts of illness, but also by social context. some research (e.g., zaloudikova, 2010) suggests that children find it easier to define illness, which they typically see as a short-term condition, rather than health, which they typically view as a more permanent state. lasting effects of covid-19 in many aspects, the pandemic can be characterized as a trauma. trauma describes those events that overwhelm a person’s ability to adapt to life that leads to strong negative emotions that translate to perceptions of threat to self (phelps & sperry, 2020). many facets of the covid-19 pandemic and the lockdowns affected children and inflicted significant stress. children’s responses to trauma are largely determined by the surrounding adults’ reactions and supports, as well as proximity to the traumatic event, and stability of the day-today routine (fernandez, 2020). while many may believe that the pandemic is in the “rearview mirror,” we assert that understanding children’s perceptions of the covid-19 virus is imperative to our current work with them in academic settings. just because the threat of covid-19 has subsided in most parts of the world, it is important to understand the lasting effects that covid-19 may have on the mental, academic, and physical health of children, particularly as they are continually exposed to the news of emerging virus strands, new viruses such as monkeypox, and continued attention and debate on immunizations and boosters. the effects of the pandemic, though still unraveling, are beginning to be documented. mental health for both children and adults, covid-19 represents an interesting departure from many other forms of medical trauma. even children who did not experience actual illness with covid-19 often experienced anxiety, from mild to debilitating, about anything from contracting the illness (either themselves or a loved one) to interruptions of social time with their friends (samji et al., 2022). with the onset of the lockdown, social interactions and networks dissolved almost overnight and many children found this a devastating departure from their established support networks of peers and teachers (pečja et al., 2021). studies (e.g., günindi, 2022) are beginning to document the ill-effects the lockdown had on socialization skills. in regard to mental health, children and adolescents are more likely to experience higher rates of depression and anxiety during and after periods of enforced isolation and this proved true following covid-19 lockdown periods (segre et al., 2021). some research (gupta & jawanda, 2020) indicates that children reported increased anxiety and frustration about the future. there are clear associations between loneliness and mental health problems, mostly depression, even up to nine years later (segre et al., 2021). other concerning findings from günindi (2022) found that the pandemic adversely affected the social skills of preschool children, and that even though parents were able to spend more time with their children, these experiences were negatively overshadowed by worry over financial, health, and psychological issues, and decreased social and physical activities. at a time when children perhaps most needed the limited mental health services afforded to them in public schools, they were suspended due to school lockdowns. academic health academic consequences from the pandemic are equally as concerning. gupta and jawanda (2020) summarize the negative effects on education as the following (p. 2): “loss of quality, deprival of education, inadequate learning, lack of digital access, fall in educational outputs, wide learning gap between low and high socioeconomic group.” just as social networks were removed abruptly with the lockdown, so were established in-person pedagogies and peer learning opportunities (kamei & harriott, 2021). the pandemic and subsequent school closures and online learning disproportionately affected the academic development of vulnerable populations: students living in poverty, those requiring special learning services, and those who speak english as an additional language (hoofman & secord, 2021). i mportant services were interrupted, and existing december 2022, volume 15, issue 2, 75-85 78 challenges confounded. unesco (2020) in dicates these devastating impacts hit low-income females the hardest, particularly in underdeveloped countries, as the risk increases for school drop-out, sexual exploitation, and early pregnancy. physical health even though children were not as affected by the physical ailments of covid-19 they are not free from impacts of the lockdown and the pandemic. researchers (gupta & jawanda, 2020) express a variety of concerns related to physical wellbeing. these include the lack of outdoor space to which some children had access for physical activity, nutritional deficiency (particularly children who depend on school food programs), as well as routine health care visits that were often suspended as a result of overwhelmed health care facilities. they also point to negative effects that increased online learning at a computer may have on children, including back pain, eye strain, and insomnia. theoretical framework this study relies on the mosaic approach developed by clark & moss in 2001, which describes a strengthbased approach for viewing children as experts in exploring and making meaning of their environments. it relies on positive, established relationships, and privileges active listening. eskandari et al. (2021) denote that using the mosaic approach with children to understand their views can empower children to help shape their environment. the approach holds six key elements (clark, 2017, p. 24), all of which were employed in this study. these elements include: multimethod, participatory, reflexive, adaptable, lived experiences, and embedded in practice (see table 1). purpose of current study the purpose of this study was to understand the perceptions of school-aged children (grades prekindergartenfifth grade) about the covid-19 through their drawings and dictations. we believed that by examining their drawings, we could gain a better sense of how children were processing these traumatic experiences, thus providing teachers and families with important insights as to how to help them cope. we note that the insights we gleaned apply to a specific snapshot in time, and that children continue to process, react, and reflect on their experiences. methods we employed a variation of the draw-and-write technique, or dwt, (pridmore & bendelow, 1995), which came to prominence in the uk in the 1980s and became more popular around 2006 (hartel et al., 2018). known largely in association with the children’s health care industry (hartel et al.), the technique can assume many forms, but in essence, as the name suggests, involves the child drawing in response to a theme, topic, or research question, and the child (or an adult) writing down their associated ideas. such a technique is effective because it recognizes the rights of children and views the child as the expert of their experiences and involves children as participants in the research, instead of just using them as objects of research (horstman et al., 2008). this type of research answers the call for less intrusive, child-friendly methods of collecting data (machenjedze, 2019). understanding children’s perceptions is paramount to improving the effectiveness of educational interventions. the dwt emphasizes the need for the child to feel valued, respected, and equal, at the end of the data collection (horstman et al.). table 1 elements of the mosaic approach in study element characteristics of mosaic approach as applied to current study multimethod recognizes different voices of children and creative strategies (blaisdell, 2012). students were free to draw their perceptions of covid-19 without limitation. children were viewed as collaborators in the study. participatory children are seen as experts in their own experiences and interpretations (huser, 2015). no “right” or “wrong” determinations were made about their interpretations but rather appreciated as the child’s reality. reflexive adults include the child’s interpretation and reflect on that interpretation (greenfield, 2011). dictation was provided by the adult to explain the drawing, and both dictation and drawings informed results. adaptable can be applied in a variety of other settings (clark & moss, 2001). children’s drawings are recommended as a strategy for teachers in a variety of other settings lived experiences values lived experiences over knowledge (rogers, 2020). as researchers, we were more interested in what children experienced as a result of covid-19, instead of what they knew about the virus. embedded into practice listening framework that can be an evaluative tool and potential to be used in practice (botsoglou, 2019). 79 ‘everybody is sick and there won’t be a cure’: the power of privileging u.s. student voices / jones & mcnulty participants the study was conducted with 16 participants in grades pre-kindergarten through fifth grade across the state of north carolina. in the united states, these grade levels include children ages four to ten years old. we used convenience sampling based on adult students enrolled in a graduate early childhood education class, taught by the first author. students enrolled in a diverse children and families course were invited to submit children’s drawings as part of a class assignment on children’s understanding of the covid-19 pandemic. children who participated were family members and friends of the adult university students. through the class, students were informed of the aims of the assignment and were reminded that submissions were purely voluntary and would not affect their grade in the class in any way. they were also reminded that the child, once asked to participate, could withdraw their participation at any time without consequence. children who agreed to participate in the drawings were directed, “draw me a picture of what you think about when you think of covid-19.” we encouraged the adults to provide little elaboration on the prompt and to allow children as much time as needed to complete the drawing. upon completion of the picture, the adult or child wrote a verbatim caption of the drawing. annotating children’s expressions about their drawing is a practical way of gaining insight into what the child is focused on in their representation of a particular experience (halpenny, 2021). prior to submission, the adults were asked to include the child’s current grade level. some also included the child’s age. the child’s name and gender were not collected. drawings were collected during march 2022, at the late onset of covid-19 as the lockdown was eased, and children returned to school, with the initial goal of the assignment being solely exploratory for the purposes of a graduate teacher preparation class. based on the quality of the drawings, the descriptions, and the emergence of themes, the idea of analyzing the drawings as data surfaced. parents were contacted again and asked if their child’s drawing could be used for a research study. thereafter, the second author joined the study and participated in all phases of the research. in march 2022, the omicron variant of covid-19 was widespread across north carolina and the united states (fowler & jasper, 2022). although children had largely returned to in-person learning at school and state mask mandates had been lifted in february, covid-19 remained a prominent issue on local and national news and in daily discourse both among families and the broader society. analysis of drawings we used content analysis to explore children’s drawings and annotations of experiences related to covid-19. content analysis was chosen to analyze drawings because it keeps the participant’s message as the focus rather than adults' interpretations (merriman & guerin, 2006). at the onset of data analysis, we independently reviewed the drawings and noted initial impressions of each picture. next, we independently looked for and noted focal points of drawings such as situations, people, objects, and missing items (halpenny, 2021). this led to our initial codes and categories. to ensure inter-rater reliability we analyzed the pictures and discussed our notes and initial impressions and focal points of the drawings together. examples of codes included sickness, covid-19 illustrations, precautions, emotional facial features, and personification. the codes were then grouped into four themes that are discussed below: (1) symptoms, (2) psychological, (3) health and safety precautions, and (4) virus cell depiction. results and discussion analysis revealed four themes in children’s drawings and depictions. themes included symptoms, psychological, health and safety precautions, and virus cell depiction. each theme is discussed below with supporting quotes and examples of children’s drawings of each theme. while most drawings fit into one theme, some fit into multiple themes due to the content of the drawing and description. symptoms analysis of children’s drawings and verbal descriptions indicated some drawings (n=7) centered on the symptoms related to covid-19 such as coughing or sneezing. some children spoke vaguely about “being sick” without discussing specific symptoms. for example, a child in first grade stated, “all the kids are sick.” another child in kindergarten stated, “covid-19 is not good for people. it may cause passing germs and making people sick.” other children were more specific about symptoms in their depictions and drawings. one pre-kindergarten child shared, “covid is cough. he got covid because he was sneezing. covid is not good.” interestingly, several children drew and described vomiting as a symptom of covid-19. a pre-kindergarten child described covid-19 symptoms by stating, “covid is throwing up. it’s like blue throw up. you can’t breathe.” similarly, another pre-kindergarten child shared [figure 1], “this is me throwing up. vomit is coming through my teeth.” we found this very interesting in that vomiting was not a common symptom of covid-19. we wondered if some young children may automatically associate being “sick” with vomiting, particularly if this has been their own experience in the past. december 2022, volume 15, issue 2, 75-85 80 figure 1 you can’t breathe finally, another pre-kindergarten child noted their drawing [figure 2] shows, “my brother throwing up because he got sick. he was at school and got sick.” such statements are reflective of chu et al.’s (2021) findings from research conducted with students aged 6-17 who were asked to write about their thoughts and feelings regarding covid-19. over 20% of the participants mentioned family concerns. figure 2 my brother throwing up because he got sick psychological the psychological theme captured drawings and/ or verbal depictions with identifiable emotions or moods. this could be illustrated in the drawing of people or anthropomorphic characteristics given to the virus cell. they assigned psychological aspects to their drawings which included fear, death and destruction, clearly defined emotions and moods, and anthropomorphic features to the virus cell, often depicted as angry faces. for example, one second grader noted [figure 3], “sad days. everybody is sick and there won’t be a cure.” the picture was drawn in black, which as zlateva (2019) notes, can often be interpreted as a cry for help. while we do not know if the sole use of the black marker was intentionally or merely convenient, the limited use of color in this picture, and others from our data, are similar to cornaggia et al.’s (2022) study of children’s drawings of “before” and “during” covid-19 lockdown. in their study, children used less color in drawings of “during” lockdown than “before” drawings. mandrapa (2015) notes that children who feel anxious often include clouds, rain, and flying birds in their pictures. this was also found in linder et al.’s (2017) study of drawings of children who are undergoing cancer treatment. dark and stormy weather was associated with drawings that depicted “sick days.” figure 3 sad days in one first grader’s drawing [figure 4] the virus cell was drawn somewhat realistically and had an angry face above two people. the people were drawn with the letter “x” in place of their eyes. zlateva (2019) indicates that the drawing of eyes is the most important detail of the face, and that closed eyes represent “a desire to hide from the world, avoiding visual contact” (p.92). we interpreted the teeth as an aggressive depiction, also confirmed by zlateva. the fact that the faces are close together may suggest there is close communication between people (edwards, 2016). figure 4 i don’t like covid 81 ‘everybody is sick and there won’t be a cure’: the power of privileging u.s. student voices / jones & mcnulty two first graders drew pictures of people dying (although they did not specifically say they died from covid-19). finally, one fifth grader captioned their picture, “coronavirus destroying a plant.” in their picture [figure 5], the cell is depicted with slanted eyebrows and something orange dispelled from the mouth. this picture also denotes the bold strokes drawn close together that are associated with stress, strong feelings, and even anger (mandrapa, 2015). figure 5 coronavirus destroying a plant this stood out to us as a psychological trait given the anthropomorphic features and the use of the word “destroy.” reminiscent of this study, other studies (martinerie et al., 2021; mcgellen et al. 2021) have explored children’s drawings of covid-19 and found them to include anthropomorphic features in the virus cell. they note how this can be from the messaging they receive through media or news. in studies that have examined children’s drawings related to the covid-19 pandemic, other researchers have also found children include clear emotions in their drawings. in thompson, spencer, and curtis’ (2021) study on children’s perspectives on covid-19 within the united kingdom found fear and sadness to be a primary theme of their findings. children in their study reported being scared of getting the virus, anxiety over loved ones dying, and an abrupt change to their daily lives. undoubtedly, covid-19 elicits strong emotions in children which may continue to be conjured when discussed or heard on the news. health and safety precautions the third theme, health and safety precautions, included drawings and/or verbal depictions specific to the public messaging during the pandemic. five drawings and descriptions illustrated specific precautions related to the pandemic including wearing masks, social distancing and lockdown procedures, and getting vaccinated. one fourth grader said, “going to school, but have to wear a mask.” in the picture [figure 6], the child labeled the person “me” and added, “mask to be safe.” another fourth grader drew a picture [figure 7] of a mask and a needle, adding the caption, “my picture is a shot and mask. you should always wear your mask. you should get the vaccine which is the needle shot.” figure 6 going to school but have to wear a mask image 7 a shot and a mask finally, a pre-kindergarten child also drew a picture of masks and noted, “you wear a mask so you don’t get covid-19. you have to go to the store and buy stuff and stay home.” we found it interesting that in all sixteen drawings, no child drew a picture of handwashing. we speculated that children may have been more exposed to this heath habit, and thus did not mention it because it was less novel than wearing masks, social distancing, or isolating. december 2022, volume 15, issue 2, 75-85 82 virus cell depiction the virus cell depiction theme included drawings that had a representation of the actual coronavirus cell that was circulated widely in the media. it is recognized as a gray spherical mass with red triangle-tipped spikes. several children included their interpretations of the virus cell with a sphere and protruding spikes. for example, figure 8 illustrates a fifth grader’s representation of covid-19, with a caption that reads, “covid spreading covid.” figure 8 covid spreading covid these findings are supported by other studies (e.g., bonoti et al., 2022; joubert & wasserman, 2020) in which children illustrated an accurate understanding of the virus cell. this has been attributed to the abundance of information and pictures shared via the news, social media, and health and safety resources (bonoti et al., 2022). while access to information is helpful, it may also cause additional stress to children who are also processing the overwhelming amount of information or do not have an understanding of molecular cells. limitations despite the interesting findings of this study, there are a few limitations that should be noted. for example, because we imposed the research questions after the drawings were collected, we had no influence as to the exact methods used to initiate the drawings. had we been present for the drawing activity, and known this would lead to research, we would have asked the children additional questions and recorded longer responses to elicit additional context around their assigned meanings. additionally, we had little to no knowledge as to the drawing materials provided to children for their drawings. this became particularly relevant as we reviewed pictures drawn with one or limited colors and wondered if this held any significance. research (e.g., farolkhi & hashemi, 2011) suggests that the dark shading of face or body is correlated with both anxiety and depression, as is the choice of dark over light color choices. not knowing if the color choice was intentional or merely based on available materials meant that we could not draw such conclusions. future studies may also wish to more closely note the ages of the artists, as developmental maturity is expected around age 7. as children become more mature in their understanding of biology, it reasons that their conceptualizations of covid-19 may also be affected. some research (e.g., cornaggia, 2022) suggests that older children are more expressive in their drawing. more research is needed, however, as there are also studies (e.g., bonoti et al., 2022) that suggest age-related themes are not dominant, and that children of all ages produce images with common characteristics. implications and conclusions this study reaffirms the use of children’s drawings as a legitimate data source. we conducted the study to shed light on the perceptions children held about covid-19 in a way that went deeper than simply asking them to respond verbally to questions. in so doing, we were able to determine common themes that were important to students. we offer this method as a resourceful tool for teachers and families to gain insights into the perceptions of children, particularly when they may not have the vocabulary to properly convey their thinking. through drawing, children create an important entre to their way of thinking that can provide a trusted adult the opportunity to ask additional questions and clarifications. these windows into a child’s thinking provides adults valuable information as to how to better equip them to cope with stressful and anxiety-producing situations. in fact, kang et al. (2020) suggest art as a specific strategy for mitigating anxiety in children. for example, adults may be able to offer clear explanations about misconceptions, or simply provide additional emotional support, that otherwise may have been missed. kang et al. report that fear of the unknown is particularly worrisome for children, and thus it serves children well to create opportunities for conversation in age-appropriate terms and increases their trust in the caregivers. while the pandemic may be less threatening now than at the time of this study, there is still a preponderance of situations and events that elicit anxiety in children (and adults). we assert that teachers can use times throughout the day as appropriate, to offer drawing opportunities for students, in order to gauge student well-being. the findings of this study demonstrate the impact that covid-19 had on children, particularly 83 ‘everybody is sick and there won’t be a cure’: the power of privileging u.s. student voices / jones & mcnulty the negative associations with the lockdown, illness, and even death. while we may not know the impact this anxiety and negativity will have on children, it does send an important message that it is up to us, as adults, to continue to ask children how they are processing such information. this study amplifies the need to continue to monitor public messaging. many children repeated the messaging of wearing masks, getting vaccines, and maintaining social distance. such findings indicate that public messaging (and perhaps parental reinforcement of this messaging) was effective, and has impacted children, who felt they knew what strategies to adopt to keep safe. it is important to continue public messaging in ways that are appropriate for both children and adults, as it is apparent that children do in fact internalize this messaging. for researchers, the collection of children’s drawings centered on a global pandemic creates a unique opportunity for cross-cultural analysis of children’s lived experiences and perceptions. the covid-19 pandemic created universal experiences across the globe including, but not limited to, possible illness, social isolation, and psychological distress including fear, anxiety, and post-traumatic stress. examining children’s drawings of their experiences during the pandemic allows researchers to compare across cultures during a given time frame. given the handful of studies that examined children’s drawings from the pandemic, there appears to be representation across the globe. because of this study, we more fully appreciate the power of drawing and dictation to provide children voice as to how they are interpreting their worlds. discussions of drawings serve as an inviting way to involve students in sharing their thinking and is less intrusive than asking incessant questions in conversation. drawings and focused dictations, though certainly not a replacement for other therapeutical strategies, can help empower even young students and provide adults with better understandings of the supports children need as they make sense of their world. one key takeaway from this study is that children are often eager to share their perceptions and understandings through drawing, so as adults, we would be remiss not to use this as a tool for increasing our capability to assist them. references bhandari, r. (2021). a study exploring key areas of worry in children and their expression through drawings amid covid-19 outbreak in march 2020. international journal of child development and mental health, 9(1), 39-54. retrieved from https://he01.tci-thaijo.org/index. php/cdmh/article/view/247136 bland, d. (2012). analysing children’s drawings: applied imagination. international journal of research and method in education, 35(3), 235-242. https:// doi.org/10.1080/1743727x.2012.717432 blaisdell, c. (2012). inclusive or exclusive participation: paradigmatic tensions in the mosaic approach and implications for childhood research. childhoods today, 6(1). http://www. childhoodstoday.org/article.php?id=68 bonoti, f., christidou, v., & papadopoulou, p. (2022). children’s conceptions of coronavirus. public understanding of science, 31(1), 35-52. http:// doi.org/10.1177/09636625211049643 botsoglou, k., beazidou, e., kougioumtzidou, e., & vlachou, m. (2019) listening to children: using the ecers-r and mosaic approach to improve learning environments: a case study, early child development and care, 189:4, 635-649, doi: 10.1080/03004430.2017.1337006 bradding, a., & hortsman, m. (1999). using the write and draw technique with children. european journal of oncology nursing, 3(30), 170-175. https://doi.org/10.1016/s1462-3889(99)80801-1 braun v., clarke v. (2006). using thematic analysis in psychology. qualitative research in psychology, 3, 77–101. https://doi. org/10.1191/1478088706qp063oa burgess, s., rogers, m., jefferies, d. (2022). empathy and narrative: a discussion of contemporary stories from childcare and healthcare. international electronic journal of elementary education, 14(5), 631-646. chu, k.a., schwartz, c., towner, e., kasparian, n. a., & callaghan, b. (2021). parenting under pressure: a mixed-methods investigation of the impact of covid-19 on family life. journal of affective disorders reports, 5, https://doi.org/10.1016/j. jadr.2021.100161 clark, a. (2017). listening to young children. jessica kingsley publishers. clark, a., & moss, p. (2001). listening to children: the mosaic approach. london: national children’s bureau. cornaggia, a., bianco, f., gilli, g., marchetti, a., massaro, d., & castelli, i. (2021). children’s representations of the covid-19 lockdown and pandemic through drawings. frontiers in psychology. https://doi.org/10.3389/fpsyg.2022.960893 deguara, j. (2018). young children’s drawings: a methodological tool for data analysis. journal of early childhood research. https://doi. org/10.1177/1476718x18818203 december 2022, volume 15, issue 2, 75-85 84 dyson, a. h. (1986). transitions and tensions: interrelationships between the drawing, talking, and dictating of young children. research in the teaching of english, 20(4), 379–409. http:// www.jstor.org/stable/40171094 eskandari, m., motazedian, f., & kouchak khoshnevis, a. m. (2021). qualitative assessment; design of children's educational spaces with the mosaic approach (consistent with the taylor educational model). creative city design, 4(3), 143-155. farmer, j.l., spearman, m., qian, m., leonard, a. e., & rosenblith, s. (2018). using children’s drawings to examine student perspectives of classroom climate in a school-within-a-school elementary school. the elementary school journal, 118(3), 384-408. https://doi.org/10.1086/696194 farokhi, m., & hashemi, m. (2011). the analysis of children's drawings: social, emotional, physical, and psychological aspects. procedia social and behavioral sciences, 30, 2219-2224, https:// doi.org/10.1016/j.sbspro.2011.10.433 fernandez, b. (2020). supporting students following crises and natural disasters. in e. rossen & r. hull (eds.), supporting and educating traumatized children: a guide for school-based professionals (2nd edition). new york, ny: oxford university press. fowler, h. & jasper, s. (2022). coronavirus omicron updates: here’s what to know in north carolina on march 22. the news and observer, retrieved from https://www.newsobserver.com/news/ coronavirus/article259634169.html gonzález-calro, g., varea, v., & garcía-monge, a. (2022). children’s experiences of lockdown and social distancing in the covid-19 pandemic. journal of family issues, 1-24. https://doi. org/10.1177/0192513x221094038 greenfield, c. (2011). personal reflection on research process and tools: effectiveness, highlights and challenges in using the mosaic approach. australasian journal of early childhood, 36(3), 109–116. https://doi. org/10.1177/183693911103600314 günindi, y. (2015). preschool children’s perceptions of the value of affection as seen in their drawings. international electronic journal of elementary education, 7(3), 371-382. retrieved from https:// w w w. i e j e e.c o m / i n d ex . p h p/ i ej e e /a r t i c l e/ view/86/84 günindi, y. (2022). the effect of online education on children’s social skills during the covid-19 pandemic. international electronic journal of elementary education, 14(5), 657-665. retrieved from https://www.iejee.com/index.php/iejee/ article/view/1861 gupta, s. and jawanda, m.k. (2020), the impacts of covid-19 on children. acta paediatr, 109: 21812183. https://doi.org/10.1111/apa.15484 haffejee, s., & levine, d. t. (2020). 'when will i be free': lessons from covid-19 for child protection in south africa. child abuse & neglect, 110(pt 2), 104715. https://doi.org/10.1016/j. chiabu.2020.104715 hartel, j., noone, r., oh, c., power, s., danzanov, p., & kelly, b. (2018). the isquare protocol: combining research, art, and pedagogy through the drawand-write technique. qualitative research, 18(4), 433–450. https://doi.org/10.1177/1468794117722193 halpenny, a.m. (2021). capturing children's meanings in early childhood research and practice: a practical guide. new york, ny: routledge. hoofman, j., & secord, e. (2021). the effect of covid-19 on education. pediatric centers of north america, 68(5), 1071-1079. https://doi. org/10.1016/j.pcl.2021.05.009 horstman, m., aldiss, s., richardson, a., & gibson, f. (2008). methodological issues when using the draw and write technique with children aged 6 to 12 years. qualitative health research, 18(7), 1001– 1011. https://doi.org/10.1177/1049732308318230 huser, c. (2015). children’s voices on play in a mosaic approach study: children as conscious participants in a case study. boğaziçi üniversitesi eğitim dergisi, 26 (1), 35-48. retrieved from https://dergipark.org.tr/en/pub/buje/ issue/3829/51429 joubert, m., & wasserman, h. (2020) spikey blobs with evil grins: understanding portrayals of the coronavirus in south african newspaper cartoons in relation to the public communication of science. journal of science communication, 19(7), 1-26. https://doi.org/10.22323/2.19070208 kamei, a., & harriott, w. (2021). social emotional learning in virtual settings: intervention strategies. international electronic journal of elementary education, 13(3). retrieved from https://www. iejee.com/index.php/iejee/article/view/1487 kang, y. q., lim, t., ragen, e. s., tan, m. y., & aishworiya, r. (2020). managing children's anxiety during covid-19 pandemic: strategies for providers and caregivers. frontiers in psychiatry, 11, 552823. https://doi.org/10.3389/fpsyt.2020.552823 linder, l.a. bratton, h., nguyen, a. et al. (2017). comparison of good days and sick days of school-age children with cancer reflected through their drawings. quality of life research, 26(10), 2729-2738. https://doi.org/10.1007/s11136017-1621-6 85 ‘everybody is sick and there won’t be a cure’: the power of privileging u.s. student voices / jones & mcnulty lionard, m. (2006). children’s drawings as a methodological tool: reflections on the eleven plus system in northern ireland. irish journal of sociology, 15(2), 52-66. https://doi. org/10.1177/079160350601500204 machenjedze, n., malindi, m. j., and mbengo, f. (2019) the feasibility of the draw-and-write technique in exploring the resilience of children orphaned by aids. african journal of aids research, 18(1), 72-80. https://doi.org/10.2989/16085906.2018.155 6170 mandrapa, l. (2015) learn to decode children’s drawings. njvak djokovic foundation. martinerie, l., bernoux, d., giovannini-chami, l., fabre. a. (2021). children’s drawings of coronavirus. pediatrics; 148 (1), https://doi.org/10.1542/ peds.2020-047621 mitchell, l.m. (2006). child centered? thinking critically about children’s drawings as a visual research method. visual anthropology review, 22(1), 6073. https://doi.org/10.1525/var.2006.22.1.60 merriman, b. & guerin, s. (2006). using children’s drawings as data in child-centered research. the irish journal of psychology, 27(1-2), 48-57. https://doi.org/10.1080/03033910.2006.10446227 mondragon, n. i., munitis, e. a., berasategi, s.n., gorrotxategi, p., and santamaria, d. m. (2022). how are children coping with covid-19 health crisis? analyzing their representations of lockdown through drawings. childhood, 29(4), 1-16. https://doi.org/10.1177/09075682221101199 mouratidi, p-a., bonoti, f., & leondari, a. (2015). children’s perceptions of illness and health: an analysis of drawings. health education journal, 1-14. https://doi.org/:10.1177/0017896915599416 pečjak, s., pirc, t., podelsek, a., peklaj, c. (2021). some predictors of perceived support and proximity in students during covid-19 distance learning. international electronic journal of elementary education, 14(1), retrieved from https://www. iejee.com/index.php/iejee/article/view/1627 phelps, c., & sperry, l. l. (2020). children and the covid-19 pandemic. psychological trauma: theory, research, practice, and policy, 12(s1), s73–s75. https://doi.org/10.1037/tra0000861 rogers, m. & boyd, w. (2020). meddling with mosaic: reflections and adaptations. european early childhood education research journal, 28:5, 642-658, doi: 10.1080/1350293x.2020.1817236 rydström, l-l., ångström-brännström, c., blake, l., brayl, l., carter, b. et al. (2022). how children in sweden accessed and perceived information during the first phase of the covid-19 pandemic. scandinavian journal of public health, 50(1), 144-151. https://doi.org/10.1177/14034948211051884 samji, h., wu, j., ladak, a., vossen, c., stewart, e., dove, n., long, d., & snell, g. (2022). review: mental health impacts of the covid-1919 pandemic on children and youtha systemic review. child and adolescent mental health, 27(2), 173-189. https://doi.org/10.1111/camh.12501 segre, g., campi, r., scarpellini, f., clavenna, a., zanetti, m., cartabia, m., & bonati, m. (2021). interviewing children: the impact of the covid-19 quarantine on children’s perceived psychological distress and changes in routine. bmc pediatrics, 21(231), 1-11. doi: https://doi. org/10.1186/s12887-021-02704-1. unesco (2020). covid-19 school closures around the world will hit girls hardest. https://en.unesco. org/news/covid-19-school-closuresaroundworld-will-hit-girls-hardest. accessed august 11, 2022. walker, k. (2007). children and their purple crayons: understanding their worlds through their drawings. childhood education, winter, 84-102. https://link.gale.com/ a p p s / d o c /a 1 7 2 9 0 7 4 1 7/a o n e ? u = a n o n ~ 3e8f911&sid=googlescholar&xid=ae767086 zaloudikova, i. (2010). children’s conceptions of health, illness, death and the anatomy of the human body. in rehulka, e. (ed.) school and health 21: health education – contexts and inspiration. brno: masarykova univerzita, pp.123-140. zlateva, ani (2019). how to read children’s drawings. l-universita ta malta. international electronic journal of elementary education, september 2016, 9(1), 151-166. issn:1307-9298 copyright © iejee www.iejee.com elementary school students’ health-related self-beliefs* alicia l. fedewaa michael d. tolanda ellen l. ushera caihong r. lia a university of kentucky, usa received: 28 june 2016 / revised: 14 september 2016 / accepted: 26 september2016 abstract objective: increased action is needed to combat the growing epidemic of pediatric obesity. it is imperative that researchers investigate psychological and demographic variables that may be associated with pediatric obesity in order to formulate and implement more appropriate and effective interventions. the present study examined the univariate and multivariate relationships between child physical and psychological characteristics in a diverse sample of elementary students. methods: questionnaires were collected from 109 students (63 girls, 46 boys; mean age= 9.25 years) in grades 3-5 from two elementary schools in the southeastern united states. explanatory variables were gender, ethnicity, grade level, and body mass index; outcomes examined were self-reported life satisfaction, physical self-concept, social self-concept, general selfconcept, eating self-efficacy, and exercise self-efficacy scores. univariate and multivariate statistics were used. results: correlations showed children with higher physical self-concept, social selfconcept, general self-concept, and eating self-efficacy tend to have higher life satisfaction. regressions revealed that african american students had a higher physical self-concept than both white and hispanic students and older students had a higher perceived social self-concept than younger students. multivariate regression results showed that the explanatory influence of gender, ethnicity, and grade level varied across outcome variables. conclusions: the strongest explanatory variables of children’s perceived life satisfaction, self-concept and self-efficacy were children’s characteristics (age, gender, and race). interestingly, children’s psychological functioning was not found to be in direct relationship with their weight classification and children’s body mass index was not significantly related to most outcome variables. keywords: self-efficacy; life satisfaction; self-concept; physical health; well-being introduction recent data obtained by the national center for health statistics of the centers for disease control and prevention (cdc) indicate that approximately one third of children and adolescents age 2-19 years are either overweight (at or above the 85th percentile on the cdc’s bmi-for-age growth charts) or obese (at or above the 95th percentile; ogden, * corresponding author: alicia l. fedewa, university of kentucky 236 dickey hall lexington, ky 40506 phone: 859-257-9338 fax: 859-257-5662 email: alicia.fedewa@uky.edu http://www.iejee.com/ international electronic journal of elementaryeducationvol.9, issue1, 151-166, september, 2016 152 carroll, kit, & flegal, 2012). these statistics are alarming; not only are there serious health consequences for obese children and adolescents, but there are negative psychological and emotional consequences as well. obese children are often ridiculed by peers and perceived as weak or lacking in self-control. children as young as six have rated obese children as “less likable” than those who were not obese (smith, 1999). overweight children have reported a poorer quality of life, similar to that reported by young cancer patients receiving chemotherapy (thompson & shanley, 2004). as a result, obese children may experience lowered self-esteem and increased risk of depression in adulthood (sanchez-villegas et al., 2013; taylor, wilson, slater, & mohr, 2012). the negative effect of being overweight on children’s psychological state may result in body dissatisfaction or lead to maladaptive behaviors such as binge-eating or the misuse of laxatives and diet pills (bauer, patel, prokop, & austin, 2006; cole, waldrop, & d’auria, 2006). life satisfaction life satisfaction is a global evaluation of one’s life and/or specific life domains (e.g., family, friends, school; selgison, huebner, & valois, 2005). together, positive affect, negative affect, and life satisfaction comprise subjective well-being (huebner, suldo, & gilman, 2006). research on the life satisfaction of children and youth has lagged behind that of research with adults (huebner, seligson, valois, & suldo, 2006). few studies have examined life satisfaction among obese children. one study to date has assessed the relationship between body mass index and quality of life (haraldsted, christophersen, eide, nativg, & helseth, 2011), wherein an association was found between rated quality of life and perceived body image, but not body mass index. similar to researchers who have shown the negative psychological and emotional consequences experienced by children who are obese, seligson et al. (2005) reported that “individuals with decreased levels of life satisfaction are at a higher risk for a widened spectrum of both psychological and social problems such as depression, anxiety and poor social interactions” (p. 356). gilman and huebner (2003) found that depression shares a strong relationship with life satisfaction, with correlations ranging between .50-.60 across age ranges. in general, studies show children and adolescents report life satisfaction above the neutral range (huebner et al., 2006). however, based on the findings from the aforementioned studies, it is suspected that children and adolescents who are obese experience lower life satisfaction than non-obese children and adolescents. self-concept the terms self-concept and self-esteem are sometimes used interchangeably in the literature; however, whereas self-esteem refers to a global self-view, the current study focuses on specific dimensions of self-concept (e.g., social, physical) that develop as children age (marsh, xu, & martin, 2012). although the self-concept of a young child may be global, undifferentiated, and situation-specific, “with increasing age and experience (especially acquisition of verbal labels), self-concept becomes increasingly differentiated” (shavelson, huebner, & stanton, 1976, p. 414). the self-description questionnaire i (sdq1) adopted in this study is based on shavelson et al.’s (1976) multidimensional hierarchical model of self-concept, in which self-concept refers to individuals’ perceptions of themselves formed through their experiences with their environment that may be used to explain their behavior. self-concept is particularly influenced by environmental reinforcements and significant others (marsh et al., 2012). theorists have suggested that a negative self-image results from dissonance between how individuals view themselves and how they perceive others to view them (e.g., kimm, sweeny, & janosky, 1991). previous studies comparing levels of self-concept in obese and non-obese children have yielded mixed results. some studies have found significant differences between these two elementary school students’ health-related self-beliefs / fedewa, toland, usher & li 153 groups of children (griffiths, parsons, & hill, 2010; taylor, wilson, slater, & mohr, 2012); others have found no significant differences (falkner et al., 2001; kimm et al., 1991; wadden, foster, brownell, & finley, 1984). researchers hypothesize that these discrepant results may be due to the relative insensitivity of using measures of global self-concept (kimm et al., 1991). we therefore undertook the present study with an assessment of the differentiated aspects of self-concept that might be related to children’s physicality. gender, ethnicity, and severity of obesity have also been found to influence participants’ overall self-concept and individual subscale scores (kimm et al., 1991). it is of particular importance to examine potential ethnic differences in these self-concept variables given that prior research has found differences in multiethnic samples of obese children and adolescents in their emotional well-being, quality of life, and self-esteem (calzo, sonneville, haines, blood, field, & austin, 2012; fallon et al., 2005; strauss, 2000; viner, taylor, head, booy, & stansfel, 2006). findings from viner et al. (2006) suggest that the influence of obesity on psychological functioning is mediated by certain cultural factors. for example, fallon et al. (2005) compared the quality of life of overweight and nonoverweight black and white adolescents and found that, as adolescents became heavier, white adolescents described greater distress than black adolescents. strauss (2000) found that the consequence of obesity on self-esteem was significantly greater in white and hispanic girls compared to black girls. these findings suggest that ethnic differences in the evaluation of individuals’ weight and shape are influential factors in the consequence obesity has on weight control behaviors and psychological well-being and should be further explored (viner et al., 2006). research suggests that age and gender differences may also be critical variables to assess given possible differences in adolescent and female distress of being overweight or obese (bully & elosua, 2011; calzo, sonneville, haines, blood, field, & austin, 2012; mustillo, budd, & hendrix, 2013). self-efficacy in his social cognitive theory, bandura (2004) argued that biomedical interventions are not the only means by which individuals can improve their health; instead, individuals can exercise control over their health by changing their lifestyle habits. the primary mechanism by which individuals exercise control over their own behaviors is through their personal efficacy beliefs (bandura, 1997). unless people believe they can implement the necessary skills and strategies needed to achieve desired outcomes, they have little motivation to act. therefore, self-efficacy is a powerful predictor of individuals’ health behaviors (bandura, 1998). those who have stronger self-efficacy for implementing health-promoting behaviors are more likely to exhibit and maintain these behaviors, especially when faced with perceived challenges or impediments (bandura, 1998). such individuals will set higher goals for themselves and will remain more committed to achieving these goals (bandura, 2004). a lack of activity is believed to contribute to the development and/or maintenance of pediatric obesity; therefore, it is important that children have a strong self-efficacy for engaging in physical activity to either prevent or reduce obesity (annesi, 2006; trost, kera, ward, & pate, 2001). because a diet high in fat and kilocalories is related to pediatric obesity, it is also important for children to have a strong self-efficacy for regulating their dietary intake (annesi, 2006). trost et al. (2001) found significant differences in the reported physical activity self-efficacy between a group of ethnically diverse obese and non-obese sixth-grade students. specifically, obese children were less self-efficacious regarding their ability to overcome barriers to physical activity, asking their parents to provide opportunities for physical activity, and choosing physical instead of sedentary activities. international electronic journal of elementaryeducationvol.9, issue1, 151-166, september, 2016 154 to change health habits, motivational and self-regulatory skills are required (bandura, 2004). the beneficial effects of increased self-efficacy on health-promoting behaviors have been demonstrated with the treatment/regulation of numerous and various chronic illnesses and other health-related areas, including changing eating habits, reducing cholesterol intake, and adopting and adhering to physical exercise programs (bandura, 1998). another study used “the exercise barriers self-efficacy scales for children” to assess the effectiveness of the fit for life curriculum with children attending an afterschool program at the ymca (anessi, 2006). the 12-week curriculum involved cardiovascular activities, resistance training, distribution of general health and nutrition information, and a workbook to train participants in self-management and self-regulatory skills. results provided support for the curriculum, which was associated with significant increases in physical self-concept, exercise-related self-efficacy, and free-time physical activity. of particular significance, the increase in physical activity was greater for the treatment group that received more extensive training on self-management and selfregulatory skills; therefore, the program’s success was attributed to an increase in children’s self-efficacy (anessi, 2006). purpose the first aim of this study was to investigate the relationship between physical (body mass index) and psychological characteristics (physical, social, and general self-concept; selfefficacy for regulating food intake and exercise) in a sample of elementary students’. the second aim was to examine how scores on each psychological characteristic vary as a function of students' gender, ethnicity, grade level, and body mass index. the third aim was to compare the relative influence that gender, ethnicity, grade level, and body mass index have on each psychological characteristic. for instance, how does the influence of gender on life satisfaction differ from its influence on physical self-concept? a final aim was to explore whether scores on the combined set of self-concept or self-efficacy variables vary as a function of students' gender, ethnicity, grade level, and body mass index. in other words, might the self-belief variables collectively (as opposed to individually) vary as a function of student characteristics? and, if they vary, which selfperception variable(s) have influence on the differences for gender, ethnicity, grade level, and body mass index (i.e., do boys score lower or higher than girls on the set of variables?). method participants and procedure participants were 116 students (68 girls, 48 boys; meanage= 9.25 years) in grades 3-5 from two elementary schools in the southeastern united states. approximately 80% of participants received free or reduced-price lunch. students self-identified as african american (n= 41), white (n= 36), hispanic (n= 32), and other or mixed ethnicity (n= 7). due to the limited number of other or mixed ethnicity participants, we focused on the remaining 109 participants (see table 1). prior to data collection informed consent was obtained from the legal guardians of the participants. the questionnaire was distributed to participants by researchers in their classrooms at the beginning of the calendar year (i.e., january). data were collected in small groups of no more than six participants. participants completed the student survey as items were read aloud by the researcher. items were read aloud by a researcher. another researcher measured and recorded each student’s height and weight. elementary school students’ health-related self-beliefs / fedewa, toland, usher & li 155 table 1. sample description (n = 109) characteristic mean (sd) or n (%)* age 9.25 (1.03) gender girls 63 (57.8) boys 46 (42.2) ethnicity white 36 (33.0) african american 41 (37.6) hispanic 32 (29.4) height 55.5 (3.6) weight 87.8 (26.3) bmi 19.7 (4.1) grade level grade level 3 46 (42.2) grade level 4 32 (29.4) grade level 5 31 (28.4) school school 1 58 (53.2) school 2 51 (46.8) percentages may not add to 100 due to rounding. *numbers represent mean (standard deviation) for continuous variables and frequency (percentage) for categorical variables. measures demographics. students responded to survey questions by indicating their gender, birth date and age (later verified with school records), grade level, and race/ethnicity. life satisfaction. a modified version of the brief multidimensional students’ life satisfaction scale (bmslss; huebner, 1997) assessed overall life satisfaction. the 6-item scale addressed five specific domains: family, friends, self, school, and living environment and a global evaluation (e.g., “i am happy with my overall life”) (huebner et al., 2006). students typically answer each question by selecting one of the following options: terrible, unhappy, mostly dissatisfied, mixed (about equally satisfied and dissatisfied), mostly satisfied, or delighted (seligson, huebner, & valois, 2005). however, to facilitate student response and to make the completed questionnaire cohesive, the students responded using a 6-point likert scale that ranged from “definitely false” to “definitely true.” the bmslss has acceptable psychometric properties and has consistently yielded acceptable internal consistency estimates. huebner et al. (2006) reported an alpha coefficient of.76 for the 6-item bmslss when used with elementary school students. self-concept. a modified version of the self-description questionnaire i (sdq1; marsh, 1988) was used to assess self-concept. the original sdq i has 76 items and consists of 8 subscales (physical abilities, physical appearance, peer relations, parent relations, reading mathematics, general school, and general self) that measure children’s selfperceptions in various areas of self-concept (rubie-davies, 2006). the eight subscales can international electronic journal of elementaryeducationvol.9, issue1, 151-166, september, 2016 156 be analyzed separately or can be combined to form a measure of academic self-concept (the averaged scores on the reading, mathematics, and general school self-concept subscales) and nonacademic self-concept (the averaged scores on the physical abilities, physical appearance, peer relations, and parent relations subscales; smith & croom, 2001). for the current study, selected items from the physical appearance, peer relations, and general self subscales were included. physical abilities self-concept was not included because the items overlap with those included under the measure of self-efficacy for regulating exercise habits. furthermore, research indicates that children do not provide consistent responses to negatively worded items; therefore, negatively worded items were not included in the questionnaire (smith & croom, 2001). self-efficacy scale for regulating eating and exercise habits. a modified version of the selfefficacy scale for regulating eating and exercise habits (bandura, 2006) assessed students’ perceived capability to regulate eating and exercise habits. perceived efficacy is measured against a variation of challenges and impediments (e.g., perceived efficacy to implement healthy eating and exercise habits in different conditions: for exercise, when tired, during bad weather, and when upset; for eating: when at home, at a restaurant, or bored). the self-efficacy response scale is traditionally a 100-point scale; however, bandura (2006) suggested that pictorial descriptors be used with young children. to provide a visual representation of the response choices, the scale for the current questionnaire used progressively larger letters to represent the varying levels of true and false responses. participants responded using a 6-point likert-type format that ranged from 1 (definitely false) to 2 (definitely true). body mass index (bmi). body weight and standing height were measured with a calibrated portable scale and stadiometer. height was recorded in inches and weight was recorded in pounds (both to the nearest ¼). bmi [weight (kg) / height2 (m2)] is a simple and convenient method of measurement to classify children as overweight or obese (semiz, özgören, & sabir, 2007). bmi has been found to correlate with other fat thickness measurements; however, it is recommended for use with children when compared to skinfold thickness and waist/hip ratio (semiz, özgören, & sabir, 2007). for the current study, participants’ height and weight were measured and their bmi was then calculated. data analysis univariate multiple linear regression analyses were used to explain scores on each of six dependent variables (life satisfaction, physical self-concept, social self-concept, general self-concept, eating self-efficacy, and exercise self-efficacy) based on gender (referent: boy), ethnicity (referent: white), grade level (referent: grade 3), and bmi (mean centered). to address the third aim a multivariate multiple regression analysis was conducted to determine whether the influence any one predictor had across dependent variables varied. to address the final aim another multivariate multiple regression was conducted to explore the influence each explanatory variable had on each set of self-perception dependent variables. specifically, we first conducted omnibus multivariate wald tests for gender, ethnicity, grade level, and bmi for the set of self-concept variables (this analysis is akin to doing a mancova). if the omnibus multivariate wald test for a given predictor was significant, then a descriptive discriminant function analysis was conducted (as recommended by enders, 2003) to determine which perception variables contributed most to discriminating or separating groups. these final analyses were repeated for the set of self-efficacy variables. elementary school students’ health-related self-beliefs / fedewa, toland, usher & li 157 due to the polytomous nature of the item responses on each self-report scale, a rating scale model was fit to the item response data using marginal maximum likelihood estimation. then, expected a posteriori (eap; bock and mislevy, 1982) estimation was used to create student latent scores (m= 0, sd= 1). irtpro 2.1 (cai, dutoit, & thissen, 2011) was used to calibrate items and score students on each of the self-report scales. univariate and multivariate regressions were conducted in mplus 6.0 (muthén & muthén, 1998-2010) using maximum likelihood robust. spss version 21 was used to conduct descriptive discriminant function analyses. results univariate multiple regression table 2 (see appendix) provides the results for the univariate multiple regression models. only the models explaining physical self-concept and social self-concept scores were significant. for the model explaining physical self-concept, bmi and ethnicity were significant contributors to the model. as expected, bmi had a negative influence with physical self-concept, whereas african american students had lower physical self-concept scores than both white and hispanic students. for the model explaining social selfconcept, only grade level was a significant contributor to the model. that is, students in grade 5 had a lower perceived social self-concept than students in both grades 3 and 4. table 3. omnibus multivariate wald test for the equality of the influence of gender, ethnicity, grade level, and body mass index across dependent variables variable w df p gender 19.15 5 .0018** ethnicity 27.25 10 .0024** grade level 27.34 10 .0023** body mass index 6.48 5 .26 w, wald test of parameter constraints; df, degrees of freedom. *p < .05. **p < .01. ***p < .001. table 4. single degrees of freedom wald test for the coefficient difference of gender, ethnicity, and grade level across outcomes variable ls vs. psc ls vs. ssc psc vs. ssc psc vs. gsc psc vs. eat psc vs. exe ssc vs. gsc ssc vs. eat eat vs. exe gender 4.34* 13.72** * white vs. african american 7.81** 15.03* ** 12.57* ** 8.44** 4.61* 4.44* 3.91* african american vs. hispanic 8.68** 13.79* ** 12.27* ** 6.82** 6.25* 6.64* grade 3 vs. grade 5 7.65** grade 4 vs. grade 5 5.44* 8.37** 10.33* * 9.09** 4.16* ls, life satisfaction; psc, physical self-concept; ssc, social self-concept; gsc, general self-concept; eat, eating self-efficacy; exe, exercise self-efficacy. *p < .05. **p < .01. ***p < .001. international electronic journal of elementaryeducationvol.9, issue1, 151-166, september, 2016 158 multivariate multiple regression the global multivariate multiple regression model result was significant and indicated that our model fit the data, 2(36, n= 109)= 76.58, p < .001. multivariate results for equality of coefficients. the multivariate wald tests for the equality of an explanatory variable across each dependent variable showed the influence of gender, ethnicity, and grade level varied across dependent variables (see table 3). to track down the different coefficient effects across dependent variables (see table 2), single degrees of freedom wald tests were conducted for each explanatory variable (see table 4). only those tests that were significant are provided due to the large number of comparisons conducted. tables 2 and 4 show that the influence of gender on eating selfefficacy was opposite of its influence on social self-concept and exercise self-efficacy. this suggests that girls had lower eating self-efficacy scores compared to boys, but this pattern was reversed for social self-concept and exercise self-efficacy. when examining the influence of ethnicity across dependent variables, we find white vs. african american on physical self-concept (b= 0.70) was greater than its influence on life satisfaction (b= 0.12), general self-concept (b= 0.01), and exercise self-efficacy (b= 0.14). the influence of white vs. african american on social self-concept (b= 0.36) was greater than its influence on general self-concept. also, the influence of white vs. african american on eating self-efficacy (b= -0.19) was opposite its influence on physical self-concept, social self-concept, and exercise self-efficacy. this latter comparison indicates that african american students had lower eating self-efficacy scores compared to white students, but this was reversed for physical self-concept, social self-concept, and exercise self-efficacy. additional comparisons for ethnicity on dependent variables showed that the influence of african american vs. hispanic on physical self-concept (b= -0.56) was greater than its influence on life satisfaction (b= -0.03), general self-concept (b= 0.07), eating self-efficacy (b= 0.23), and exercise self-efficacy (b= -0.06). the influence of african american vs. hispanic on social self-concept (b= -0.33) was less than its influence on general selfconcept (b= 0.07), eating self-efficacy (b= 0.23), and exercise self-efficacy (b= -0.06). the influence of african american vs. hispanic on physical self-concept and social self-concept was opposite the influence of african american vs. hispanic on general self-concept and eating self-efficacy, indicating that hispanic students had lower physical self-concept and social self-concept scores compared to african american students. this pattern was reversed for general self-concept and eating self-efficacy. as for grade level, the difference between grade 3 vs. grade 5 on social self-concept (b= 0.45) was greater than the difference on general self-concept (b= 0.03). the difference between grade 4 vs. grade 5 on social self-concept (b= 0.49) was greater than what was observed on life satisfaction (b= 0.70), physical self-concept (b= 0.01), general self-concept (b= 0.06), and eating self-efficacy (b= -0.16). the difference between grade 4 vs. grade 5 on exercise self-efficacy (b= 0.14) was greater than the difference on eating self-efficacy. moreover, the difference between grade 4 vs. grade 5 on eating self-efficacy was opposite the difference of grade 3 vs. grade 5 on social self-concept and exercise self-efficacy. this finding suggests that students in grade 5 had lower eating self-efficacy scores compared to students in grade 4, but this was reversed for social self-concept and exercise self-efficacy scores. in general, table 4 shows the pattern of coefficient differences we observed for white vs. african american and hispanic vs. african american were similar. also, the coefficient differences across dependent variables were only notable for grade 4 vs. grade 5. elementary school students’ health-related self-beliefs / fedewa, toland, usher & li 159 multivariate results on the combined set of self-concept variables. the second multivariate multiple regression analysis was conducted to explore the influence or relationship that each explanatory variable had on all self-perception variables simultaneously when considered as a set. table 5 summarizes the multivariate wald tests for each explanatory variable on the set of self-concept variables. results showed that ethnicity and grade level had a significant explanatory influence on the set of self-concept variables, whereas gender and bmi did not. a descriptive discriminant data analysis was next conducted to determine where ethnic and grade-level groups differed on the composite self-concept variable and which self-concept variable(s) contributed most to the differences for ethnicity and grade. with respect to ethnicity, the standardized discriminant function coefficient showed physical self-concept (β= 1.01) and general self-concept (β= 0.54) contributed most to the composite function, the mean composite self-concept score was 0.61 for african american children, -0.31 for hispanic children and -0.42 for white children. these results suggest that african american children reported higher overall self-concept scores than did hispanic and white children. in regard to grade level, the standardized discriminant function coefficients showed that social self-concept contributed most to the composite self-concept variable (β= 1.28), followed by physical self-concept (β= -0.64) and general self-concept (β= -0.35). the mean composite self-concept score was -0.11 for students in grade 3, -0.30 for students in grade 4, and 0.48 for students in grade 5. these results suggest that students in grade 5 have higher overall self-concept than do students in grades 3 and 4. multivariate results on the combined set of self-efficacy variables. multivariate regression and discriminant analysis were also conducted for the set of self-efficacy variables. table 6 shows only gender had a significant explanatory influence on the collective self-efficacy variables. descriptive discriminant data analysis was conducted to determine where gender groups differed on the composite self-efficacy variables. the standardized discriminant function coefficients showed that exercise self-efficacy (β = 1 .09) and eating self-efficacy (β= -1.06) both contributed most to the linear composite. the mean composite self-efficacy variable was -0.27 for girls and 0.37 for boys. these results suggest that girls have lower eating self-efficacy and higher exercise self-efficacy relative to boys when the dependent variables are considered simultaneously. table 5. wald (w) test for the multivariate effect of gender, ethnicity, grade, and body mass index on physical self-concept, social self-concept, and general self-concept simultaneously variable w df p gender 2.490 3 .4771 ethnicity 23.386 6 .0007 grade 18.087 6 .0060 body mass index 3.997 3 .2618 international electronic journal of elementaryeducationvol.9, issue1, 151-166, september, 2016 160 table 6. wald (w) test for the multivariate effect of gender, ethnicity, grade, and body mass index on eating self-efficacy and exercise self-efficacy simultaneously variable w df p gender 14.369 2 .0008 ethnicity 4.545 4 .3373 grade 6.308 4 .1773 body mass index 0.382 2 .8257 discussion an extensive literature base points to the fact that pediatric obesity impacts children’s self-concept and quality of life (griffiths, parsons, & hill, 2010). however, many studies fail to take into account the complexity of additional child characteristics, including race/ethnicity, gender, and age, when considering this impact. the present study demonstrates that these variables are critical to understanding the relationship between children’s physical health status and psychological characteristics such as life satisfaction, self-concept, and self-efficacy. life satisfaction in general, findings revealed that the higher a child’s life satisfaction, the higher the child’s reported physical self-concept, social self-concept, general self-concept, and eating selfefficacy. students’ self-reported life satisfaction was not dependent on gender, race/ethnicity, grade level, and bmi, which is consistent with prior research that demonstrates relationships between perceived body image and self-concept, but not actual bmi (haraldsted, christophersen, eide, nativg, & helseth, 2011). with respect to variations across ethnicity, we found that the magnitude of difference between white and african american students on life satisfaction was smaller than the difference on physical self-concept. although prior literature reports some mixed findings with respect to the influence of ethnicity on youth life satisfaction (see proctor, linley, & maltby, 2009 for a review), our results support the majority of studies, which report a weak connection between ethnicity and life satisfaction (funk, huebner & valois, 2006; gilman & huebner, 2000). a similar relationship was found with age. that is, students’ level of life satisfaction did not vary significantly across grade levels. these findings lend credence to the body of literature indicating that most youth report a positive level of life satisfaction that remains relatively stable over time and for students of different demographic groups (proctor, linley, & maltby, 2009). self-concept our findings showed that children with lower bmi reported higher levels of physical selfconcept, but only after accounting for gender, race, and grade level; the simple correlation between bmi and physical self-concept was not significant. this is an important finding because it suggests that the relationship between a child’s physical health status (bmi) and physical self-concept varies depending on other characteristics of the child that need to be taken into account. when considering all self-concept variables as a combined linear set, african american children had higher self-concepts (made-up mostly of physical and social self-concept) than both hispanic and white children. although little research has investigated differences in physical and social self-concepts across children of varying racial backgrounds, some prior studies have shown that significant mental health differences exist between female african american, hispanic, and white individuals who elementary school students’ health-related self-beliefs / fedewa, toland, usher & li 161 are obese (anderson, cohen, & naumova, 2007; belue, francis, & colaco, 2009). depressed mood and higher levels of mental health problems have been reported for white adolescents compared to african american or hispanic adolescents. our research shows that perceived physical and self-concept are important constructs to consider when evaluating the mental health needs of children from diverse racial backgrounds, as higher levels of bmi may carry unique implications on one’s physical and social self-concept as a set according to one’s cultural or ethnic membership. our analyses indicated that older students had a higher social self-concept than younger students regardless of bmi; this was observed both univariately and multivariately. although this finding could mean that children’s self-concept may serve a buffering effect as they age, it is difficult to extrapolate the significance of these findings given that so few studies have examined the associations of bmi, age, gender, and self-concept (griffiths, parsons, & hill, 2010). further research investigating children’s overweight or obesity status over time is needed to clarify the moderating effect of these constructs. self-efficacy on the whole, children who reported a stronger sense of self-efficacy for regulating their exercise also reported higher physical, social, and general self-concept. students’ perceptions about their capability to regulate what they eat and how often they exercise did not vary by gender, race/ethnicity, grade level, or bmi, but multivariate results that compared the gender coefficients on eating and exercise self-efficacy were in opposite directions. we see this as good news, particularly the finding that, regardless of their bmi level, children reported similar levels of eating and exercise self-efficacy. when we considered eating and exercise self-efficacy among the set of other dependent variables we found that girls tended to display higher eating self-efficacy and lower exercise self-efficacy than did boys. although african american children reported higher levels of physical and social self-concept, african americans had lowered eating and exercise self-efficacy than white students. students in grade 5 reported lower eating selfefficacy scores than younger students but had higher exercise self-efficacy. strengths and limitations the diversity represented in the present sample allowed for comparisons across children of varying age, gender, and ethnic/racial backgrounds. few studies have been able to account for the complexity inherent in children’s physical and psychological development, particularly with a focus on children’s bmi. however, there are a number of limitations to this study. first, generalizations of the findings are limited to the elementary students recruited in this study. all students were recruited from two elementary schools in a southeastern state and therefore the findings may not be generalizable to national populations. second, there was a limited sample of students recruited to the study, with a risk of low statistical power for the multivariate analyses. thus, a larger sample size would help to replicate these findings, reduce any possible missed effects, and allow for mediation and moderation tests for the study variables. conclusion research has shown that children who are classified as overweight or obese may be at increased risk for lower levels of self-concept, life satisfaction, and self-efficacy. however, what the present study has shown is that children’s psychological functioning is not in direct relationship with their weight classification. instead, child characteristics (age, gender, and ethnicity) were more important explanatory variables in children’s perceived life satisfaction, self-concept, and self-efficacy, emphasizing the necessity in taking these constructs into account for future studies. international electronic journal of elementaryeducationvol.9, issue1, 151-166, september, 2016 162 • • • references american dietetic association (2006). position of the american dietetic association: individual-, family-, school-, and community-based interventions for pediatric overweight. journal of the american dietetic association, 106, 925-945. anderson, s.e., cohen p., naumova, e.n. et al. (2007). adolescent obesity and risk for subsequent major depressive disorder and anxiety disorder: prospective evidence. psychosom med., 69(8), 740 – 7. annesi, j. j. (2006). relations of physical self-concept and self-efficacy with frequency of voluntary physical activity in preadolescents: implications for after-school care programming. journal of psychosomatic research, 61, 515-520. bandura, a. (2004). health promotion by social cognitive means. health education and behavior, 31, 143-164. bandura, a. (1997). self-efficacy: the exercise of control. new york: freeman. bandura, a. (1998). health promotion from the perspective of social cognitive theory. psychology and health, 13, 623-649. bandura, a. (2006). guide for constructing self-efficacy scales. self-efficacy beliefs of adolescents (pp. 307-337). information age publishing. bauer, k.w., patel, a., prokop, l.a., & austin, s.b. (2006). swimming upstream: faculty and staff members from urban middle schools in low-income communities describe their experience implementing nutrition and physical activity initiatives. preventing chronic disease public health research, practice, 3(2), 1-8. belue r., francis l.a., & colaco b. (2009). mental health problems and overweight in a nationally representative sample of adolescents: effects of race and ethnicity. pediatrics, 123(2), 697 – 702. bock, r.d. and mislevy, r.j. (1982) ‘adaptive eap estimation of ability in a microcomputer environment’. applied psychological measurement, 6(4), 431–444. doi: 10.1177/ 014662168200600405 cai, l., du toit, s. h. c., & thissen, d. (2011). irtpro: flexible professional item response theory modeling for patient reported outcomes (version 2.1) [computer software]. chicago, il: scientific software international. calzo, j. p., sonneville, k. r., haines, j., blood, e. a., field, a. e., & austin, s. b. (2012). the development of associations among body mass index, body dissatisfaction, and weight and shape concern in adolescent boys and girls. journal of adolescent health, 51(5), 517-523. doi: 10.1016/j.jadohealth.2012.02.021 cole, k., waldrop, j., & d’auria, j. (2006). an integrative research review: effective school-based overweight interventions. journal of specialists in pediatric nursing, 11, 166-177. de ayala, r. j. (2013). factor analysis with categorical indicators: item response theory. in y. petcscher, c. schatschneider, & d. l. compton (eds.), applied quantitative analysis in education and the social sciences (pp. 208-242). new york, ny: routledge. enders, c. k. (2003). performing multivariate group comparisons following a statistically significant manova. measurement and evaluation in counseling and development, 36, 40–56. available from http://mec.sagepub.com/ elementary school students’ health-related self-beliefs / fedewa, toland, usher & li 163 falkner, n. h., neumark-sztainer, d., story, m., jeffery, r. w., beuhring, t. and resnick, m. d. (2001). social, educational, and psychological correlates of weight status in adolescents. obesity research, 9, 32–42. doi: 10.1038/oby.2001.5 fallon, e.m, tanofsky-kraff, m., norman, a.c., mcduffie, j.r., taylor, e.d., cohen, m.l., et al. (2005). health-related quality of life in overweight and nonoverweight black and white adolescents, journal of pediatrics, 147, 443-450. funk, b. a., huebner, e. s., & valois, r. f. (2006). reliability and validity of a brief life satisfaction scale with a high school sample. journal of happiness studies, 7, 41–54. gilman, r., & huebner, e. s. (2000). review of life satisfaction measures for adolescents. behaviour change, 17, 178–195. gilman, r. & huebner, s. (2003). a review of life satisfaction research with children and adolescents. school psychology quarterly, 18, 192-205. griffiths, l. j., parsons, t. j., & hill, a. j. (2010). self-esteem and quality of life in obese children and adolescents: a systematic review. international journal of pediatric obesity, 5(4), 282-304. doi:10.3109/17477160903473697 haraldstad, k., christophersen, k.a., eide, h., nativg, g.k., & helseth, s. (2011). predictors of healthrelated quality of life in a sample of children and adolescents: a school survey. journal of clinical nursing, 20(21-22), 3048-3056. doi: 10.1111/j.1365-2702.2010.03693.x huebner, e. s. seligson, j. l., valois, r. f., & suldo, s. m. (2006). a review of the brief multidimensional students’ life satisfaction scale. social indicators research, 79, 477-484. huebener, e. s., suldo, s. m., & gilman, r. (2006). life satisfaction. gg. bear, & k. m minke, eds. children’s needs: development, prevention, and intervention (pp.357-365). bethesda, md: nasp publications. kimm, s. y. s., sweeney, c. g., janosky, j. e. (1991). self-concept measures and childhood obesity: a descriptive analysis. developmental and behavioral pediatrics, 12, 19-24. marsh, h. w., xu, m., & martin, a. j. (2012). self-concept: a synergy of theory, method, and application. in k. r. harris, s. graham, & t. c. urdan (eds.) apa educational psychology handbook, vol. 1: theories, constructs, and critical issues (pp. 427-458). washington, dc: american psychological association. mustillo, s. a., budd, k., & hendrix, k. (2013). obesity, labeling, and psychological distress in latechildhood and adolescent black and white girls: the distal effects of stigma. social psychology quarterly, 76(3), 268-289. doi:10.1177/0190272513495883 muthén, l. k., & muthén, b. o.(1998–2010). mplus user’s guide (6th ed.). losangeles, ca: authors. ogden, c.l., carroll, m.d., kit b.k., & flegal k.m. (2012). prevalence of obesity and trends in body mass index among us children and adolescents, 1999-2010. journal of the american medical association, 307(5), 483-490. rubies, davies, c. m. (2006). teacher expectations and student self-perceptions: exploring relationships. psychology in the schools, 43, 537-552. sanchez-villegas, a., field, a. e., o'reilly, e. j., fava, m., gortmaker, s., kawachi, i., & ascherio, a. (2013). perceived and actual obesity in childhood and adolescence and risk of adult depression. journal of epidemiology & community health, 67(1), 81-86. doi:10.1136/jech2012-201435 shavelson, r. j., hubner, j. j., & stanton, g. c. (1976). self-concept: validation of construct interpretations. review of educational research, 46, 407-441. international electronic journal of elementaryeducationvol.9, issue1, 151-166, september, 2016 164 smith, k. e. & croom, l. (2000). multidimensional self-concepts of children and teacher beliefs about developmentally appropriate practices. journal of educational research, 5, 312-321. smith, j. c. (1999). childhood obesity. jackson, mississippi; the university press of mississippi. snethen, j. a., broome, m. e., & cashin, s. e. (2006). effective weight loss for overweight children: a meta-analysis of intervention studies. journal of pediatric nursing, 21, 45-56. strauss, r. s. (2000). childhood obesity and self-esteem. pediatrics, 105(1), 1-5. taylor, a., wilson, c., slater, a., & mohr, p. (2012). self-esteem and body dissatisfaction in young children: associations with weight and perceived parenting style. clinical psychologist, 16(1), 25-35. thompson, c., & shanley, e. (2004). overcoming childhood obesity. boulder, co: bull publishing company. trost, s. g., kerr, l. m., & pate, r. r. (2001). physical activity and determinants of physical activity in obese and non-obese children. international journal of obesity, 25, 822-829. viner, r.m, haines, m.m., taylor, s.j.c, head, j., booy, r., & stansfel, s. (2006). body mass, weight control behaviours, weight perception and emotional well-being in a multiethnic sample of early adolescents. international journal of obesity, 30, 1514-1521. wadden, t.a., foster, g.d., brownell, k.d., & finley, e. (1984). self-concept in obese and normalweight children. journal of counseling and clinical psychology, 52, 1104-1105. elementary school students’ health-related self-beliefs / fedewa, toland, usher & li 165 appendix: table 2. univariate multiple linear regression results of life satisfaction, physical self-concept, social self-concept, general selfconcept, exercise self-efficacy, and eating self-efficacy scores from gender, ethnicity, grade level, and body mass index (n = 109) life satisfaction physical selfconcept social selfconcept general selfconcept eating selfefficacy exercise selfefficacy independent variable b se β b se β b se β b se β b se β b se β constant -0.14 .18 -0.20 -0.44 0.19 -0.58 -0.34 0.21 -0.42 -0.01 0.17 -0.01 0.14 0.24 0.16 -0.32 0.19 -0.43 girl vs. boy 0.00 .14 0.00 0.11 0.14 0.07 0.17 0.15 0.11 -0.03 0.13 -0.03 -0.25 0.17 -0.15 0.28 0.14 0.19 ethnicity† .51 19.46*** 5.02 0.31 1.77 0.67 white vs. african american 0.12 .17 0.08 0.70*** 0.17 0.45 0.36 0.19 0.22 < 0.01 0.15 < 0.01 -0.19 0.21 -0.11 0.14 0.17 0.09 white vs. hispanic 0.08 .18 0.05 0.14 0.16 0.09 0.03 0.18 0.02 0.07 0.15 0.05 0.04 0.20 0.02 0.09 0.17 0.05 african american vs. hispanicǂ -0.03 .16 -0.02 -0.56*** 0.16 -0.33 -0.33* 0.16 -0.19 0.07 0.15 0.05 0.23 0.18 0.12 -0.06 0.16 -0.04 grade level† .65 1.05 12.94** 0.25 0.95 2.56 3rd vs. 4th grade 0.07 .16 0.05 0.15 0.17 0.09 -0.03 0.19 -0.02 -0.03 0.16 -0.02 0.14 0.20 0.07 0.14 0.17 0.09 3rd vs. 5th grade 0.14 .17 0.09 0.15 0.17 0.09 0.45*** 0.17 0.26 0.03 0.13 0.03 -0.02 0.20 -0.01 0.28 0.18 0.18 4th vs. 5th gradeǂ 0.06 .16 0.04 0.01 0.16 0.00 0.49*** 0.15 0.28 0.06 0.14 0.05 -0.16 0.17 -0.08 0.14 0.16 0.09 body mass index -0.03 .02 -0.18 -0.04* 0.02 -0.20 -0.02 0.02 -0.09 -0.02 0.01 -0.10 0.02 0.02 0.07 0.00 0.02 -0.01 r-squared .03 .20*** .11* .01 .04 .05 b, unstandardized coefficient; se, standard error; β, standardized coefficient. *p < .05. **p < .01. ***p < .001. †omnibus wald test of parameter constraints with 2 degrees of freedom. ǂ to obtain the third possible group comparison between african american vs. hispanic and grade 4 vs. grade 5 a second univariate regression model was conducted using african american and grade 4 as the referent group for ethnicity and grade level, respectively. international electronic journal of elementaryeducationvol.9, issue1, 151-166, september, 2016 166 www.iejee.com this page is intentionally left blank http://www.iejee.com/ microsoft word iejee_5_1_93_106 international electronic journal of elementary education, 2012, 5(1), 93-108. issn:1307-9298 copyright © iejee www.iejee.com cognitive factors that influence children’s learning from a multimedia science lesson florencia k. anggoro ∗∗∗∗ college of the holy cross, worcester, ma, usa nancy l. stein national opinion research center and the university of chicago, chicago, il, usa benjamin d. jee college of the holy cross, worcester, ma, usa received: 7 september 2012 / revised: 3 november 2012/ accepted: 12 november 2012 abstract the present study examined the cognitive factors that influence children’s physical science learning from a multimedia instruction. using a causally coherent text and visual models, we taught 4 th and 7 th grade children about the observable and molecular properties of the three states of water. we manipulated whether the text was read by a tutor (which supports simultaneous encoding of the verbal and visual information, i.e., temporal contiguity) or whether children read the text on their own (which supports self pacing and interpretation of the information). children in each condition received either static or dynamic graphics. results showed that, regardless of the type of graphics, children demonstrated the greatest learning gains when the text was read to them by a tutor. this effect was more pronounced for the younger children. thus, conditions that promote integration of verbal and visual information may provide the greatest support to children’s learning from a causally coherent multimedia science lesson. keywords: science learning, multimedia instruction, causal coherence, elementary education. introduction by the 4 th grade, national and state standards in the united states require children to learn about the water cycle and states of matter (national research council (u.s.), 1996; project 2061 (american association for the advancement of science), 1993). one central educational concern is that children are often presented with materials that are ∗ florencia k. anggoro, department of psychology, college of the holy cross. 1 college street. worcester, ma 01602. usa. phone: 508-793-2748. e-mail: fanggoro@holycross.edu international electronic journal of elementary education vol.5, issue 1, 93-106, 2012 94 incomplete, inaccurate, and otherwise ineffective (bar & galili, 1994). existing materials designed to teach physical science rarely contain all relevant concepts necessary for an accurate understanding of states of matter, and even when necessary concepts are included, the materials often lack clarity and coherence (duschl, et al., 2007). this can be devastating to novice students, who are especially dependent on coherence and explicitness during learning (mcnamara, kintsch, songer, & kintsch, 1996; shwartz, weizman, fortus, krajcik, & reiser, 2008; stein, hernandez, & anggoro, 2010; stein & trabasso, 1982; trabasso & bouchard, 2000; trabasso, secco, & van den broek, 1984). another concern, besides the materials themselves, is that learning is greatly affected by a student’s capacity to process the information that they receive. even if the learning materials are coherent and contain all of the relevant concepts, learning may be inhibited if the demands on a student’s cognitive resources are too great. the present study focused on this latter issue and examined the conditions that support and hinder children’s early learning of physical science. using materials developed from a theory of complex learning (stein et al., 2010), we examined how different processing demands affected 4 th and 7 th -grade children’s learning about the observable and molecular properties of the three states of water. a theory of complex learning in previous research, stein and colleagues used a theory of complex learning to create learning modules designed to teach elementary-school children about the observable and molecular properties of the three states of water (stein et al., 2010; stein, hernandez, anggoro, & hedberg, under review). according to this theory, knowledge acquisition in the sciences requires three types of learning: concept learning, causal explanation-based learning, and argument learning. in concept learning (klausmeier, 1992; mandler, 2008; winston, 1986), explicitness is necessary, especially when learners are novices with little or no prior knowledge of the concepts to be learned. the learning materials must describe all dimensions of the concept as well as the dimensions that are not part of the concept (especially when similar concepts exist) (klausmeier, 1992; winston, 1986). an explicit compare/contrast procedure must be used to evaluate similar concepts on critical dimensions, especially when error analyses show a high rate of confusion among certain features in two concepts (klausmeier, 1992). the reason for such explicitness is to achieve an accurate representation of the chosen concepts, and to avoid overor under-inclusion of members of a category due to faulty knowledge. science learning also involves causal explanation-based learning (stein & levine, 1989; stein & trabasso, 1982; trabasso & stein, 1997). for example, understanding states of water requires knowledge of the mechanism that causes water to retain its shape and volume in a solid state. learning about these causal mechanisms provides learners with transferable knowledge that allows them to explain related phenomena, such as how liquid water has a flexible shape but invariant volume. failing to provide causal explanations, however, results in superficial understanding (stein & levine, 1989; stein & trabasso, 1982; trabasso & stein, 1997), especially in novices who have little or no knowledge of the domain. cognitive factors that influence children’s learning from a multimedia science lesson / anggoro, stein & jee 95 finally, knowledge acquisition in science involves argument learning, which emphasizes the correction of learners’ misconceptions about newly learned concepts. many misconceptions occur because of the presence of an incorrect causal explanatory structure that underlies the misconception (e.g., slotta & chi, 2006, vosniadou & brewer, 1992). inaccurate beliefs in the domain must be changed and updated. this can be accomplished by providing evidence in favor of the correct concept and showing learners why their incorrect beliefs need to be changed. thus, in correcting student errors, an entirely new explanatory structure often needs to be acquired (stein & miller, 1993; thagard, 2000). the benefits and demands of multimedia instruction these elements of complex learning—concept learning, causal explanation-based learning, and argument learning—cannot be supported through text-based instruction alone. explicit visual models are needed to teach learners about complex spatial and causal properties and processes, such as the organization, speed, and movement of molecules, that are difficult to convey in words (e.g., larkin & simon, 1987). when used properly, visual models have been found to accelerate learning for both children and adults (gobert & buckley, 2000; goldberg & bendall, 1995), and can benefit learners at different levels of expertise (goldberg & bendall, 1995; jose & williamson, 2005; mayer, hegarty, mayer, & campbell, 2005; tversky, et al., 2008). recent work on children’s learning about states of water has found that the absence of visual models that exemplify core concepts and their relationships reduces comprehension by about 20%, even with a causally-coherent text (stein et al., under review). forming a coherent conceptual representation from visual and verbal information places high demands on the learner’s limited cognitive resources. as mayer and moreno (2003) discuss, the learner must organize the presented verbal information into a verbal model, the presented images into a visual model, and integrate these two representations into a coherent whole. there are several ways in which students can become overwhelmed during this process (see mayer & moreno, 2003, for an extended discussion). for one, each of the processing channels (verbal and visual) can become overloaded. thus, understanding can be derailed early on by the complexity of processing novel scientific text and images. another potential source of cognitive overload comes further downstream. if the student is unable to simultaneously hold the verbal and visual representations in working memory, then they will be unable to integrate them. maintaining and combining representations in each channel is therefore critical as well. mayer and colleagues have investigated ways to improve multimedia learning by targeting the different sources of cognitive load. to reduce the burden on visual and verbal processing, mayer and chandler (2001) broke a science lesson into smaller units and gave the learner control over the pacing of the lesson. compared to students who received the same information in one continuous stream, the students who could selfpace showed better learning and transfer of knowledge. thus, self-pacing could reduce the burden on a student’s limited cognitive resources and enhance their ability to form verbal and visual models from the lesson. international electronic journal of elementary education vol.5, issue 1, 93-106, 2012 96 to reduce the burden on holding and integrating the visual and verbal models, mayer and moreno (2003) suggested that text and images should be presented simultaneously. mayer and anderson (1991), for example, found that students evidenced better transfer of learning when they received a lesson in which narration accompanied—as opposed to followed—an animation. when text and images are presented simultaneously, the student may be less likely to lose the visual or verbal representations that must be integrated to form a coherent understanding from a multimedia science lesson. self-pacing and temporal contiguity have been shown to reduce different sources of cognitive load in multimedia learning, yet common forms of instruction often involve a tradeoff between these two factors. learning from an illustrated textbook, for example, is ubiquitous at all levels of education. this form of instruction may support self-pacing, since the student has control over how fast they read and progress. yet, reading from a textbook is low in temporal contiguity, since the text and images are encountered separately. another common form of instruction involves a tutor or instructor reading to the child. if a tutor takes the same learning materials (text and images) and reads aloud, then temporal contiguity is increased and it may be easier for the student to hold and integrate the visual and verbal components. yet this could reduce or eliminate the benefits of self-pacing, since the tutor would hold some or all of the control over the pace of reading. purpose and overview of research given the potential tradeoffs inherent to different ways of presenting the same multimedia science lesson, we sought to test which task, self-reading vs. tutor-reading, provided the most benefit to learners at different grade levels, 4 th and 7 th grade. we adopted the causally-coherent text from stein et al. (under review), which was developed using the principles of concept learning, causal coherence, and argumentation discussed earlier. we also used the same visual models as stein et al., which served to visually illustrate characteristic molecular properties of the three states of water that were verbally described in the text. because these learning materials are highly explicit and causally coherent, the burden of interpreting the text and images may be relatively low compared to a typical lesson on the same topics. nevertheless, the content of the lesson may be novel and challenging for children. if interpreting the content of the text and images is the primary challenge that learners face, then a selfpaced lesson may be more effective than tutor-paced lesson. however, if the main challenge of the lesson is holding and integrating information across verbal and visual modalities, then tutor-reading could be most effective. indeed, stein et al. (under review), which used tutor-read instruction exclusively, found evidence of impressive learning gains in this condition. the effects of the different conditions could also depend on age. older students may have greater metacognitive awareness (e.g., flavell, 2000; metcalfe & shimamura, 1994; schneider, 2008) in addition to greater reading skills and cognitive capacities. thus, the 7 th graders may be more resilient to the demands of processing the visual and verbal components of the lesson and integrating them into a coherent representation. if so, cognitive factors that influence children’s learning from a multimedia science lesson / anggoro, stein & jee 97 then the effects of reading condition should be especially pronounced for the younger children, who may be most reliant on self-pacing or temporal contiguity. in addition to the self-read condition and tutor-read conditions at each grade level, we also manipulated the nature of the visual models (static vs. dynamic) contained in the lesson as in stein et al. (under review). it is possible that the effects of condition will be especially pronounced for one type of visual model, for example, the tutor-read condition may be especially effective when the visual models are dynamic, because the student is better able to attend to changes in the visual models over time. this may be less important in a static image. thus, we had four experimental conditions: tutorread/static, tutor-read/dynamic, self-read/static, and self-read/dynamic. our control group included children who did not receive our instruction but instead received regular, “business-as-usual” classroom instruction. method participants participants were 158 fourth-grade children (m = 9 years, 11 months; range = 9 years, 0 months to 10 years, 8 months; 87 boys, 71 girls) and 172 seventh-grade children (m = 13 years, 2 months; range = 11 years, 9 months to 14 years, 8 months; 90 boys, 82 girls) recruited from four chicago public schools. participating schools were a classical magnet school, a math-science magnet school, an arts magnet school, and a neighborhood school. to enroll in magnet schools, children had to satisfy certain requirements specific to each school (e.g., standardized test scores in reading and math, or interest in an academic domain). to enroll in a neighborhood school, children qualified based on the geographical location of their parents’ home address. the racial composition of the sample was 43% african-american, 21% hispanic, 18% white, 10% asian/pacific islander, and 8% multi-racial. this distribution roughly paralleled the overall distribution of ethnicity in the chicago public elementary schools, as we purposely intended. materials we adopted two modules from the learning sequence developed by stein et al. (under review). the first module introduced and defined matter, the three states of water, and the shape and volume of solid and liquid. this module focused on whether or not the observable properties (i.e., shape and volume) of solid and liquid water change when water is transferred from one container to another. it also explained that gas (i.e., water vapor) is invisible to the human eye, and that to learn about gas requires an understanding of molecules. the second module focused on the organization, speed, and movement of molecules that define each state of water, and then compared and contrasted these properties in each of the three states. thus, the goal of the two modules was for children to understand that matter has properties that cannot seen by the human eye, that these properties can be visually modeled, and that the three states of water differ from one another in terms of the organization, speed, and movement of molecules. the shape and volume of water vapor were discussed after children learned about the molecular properties of the three states of water. visual depictions of water international electronic journal of elementary education vol.5, issue 1, 93-106, 2012 98 vapor molecules allowed children to “see” how the invisible properties of molecules in a gaseous state enable gases to take on the shape or volume of any container. descriptions of the observable and molecular properties of each state of water were embedded in a causally-coherent sequence such that shape was discussed first, volume next, and the organization, speed, and movement of molecules third. during the presentation of the organization, speed, and movement of water vapor molecules, the changeability of shape and volume were discussed. direct comparisons were then made between each of the three states, in terms of shape and volume, and the organization, speed, and movement of molecules. descriptions for the shape and volume in each state are presented in table 1. descriptions for the molecular properties of the three states are presented in table 2. table 1. observable properties of solid and liquid water shape volume solid constant constant liquid changeable constant table 2. molecular properties of the three states of water organization speed movement solid locked in place vibrate and jiggle in place don’t move out of lattice structure liquid close and “cling” to other water molecules moderate speed slip and slide around and over other molecules gas fill the entire container very fast speed “fly” around in all directions the causal coherence of the text becomes important in describing and illustrating how heat energy regulates the speed and movement of molecules, which in turn determines the state of matter. the speed and movement of molecules increases in proportion to the amount of heat energy absorbed by the molecules. after each state was defined in terms of both observable and molecular properties, the three states were contrasted. a solid was presented first, with a description of the speed and movement of molecules. a liquid was presented next, with an explanation of how the speed and movement of molecules increase and why shape is flexible in liquids versus solids. the gaseous state was presented last, with a discussion of how an even bigger increase in energy leads to molecules breaking away from one another, moving rapidly in a random fashion, and taking up all of the volume of a closed container or escaping into the air if the container is opened. we also adopted the visual models developed by stein et al. (under review). the static graphics, in the form of jpeg files, presented either as single illustrations (e.g., the lattice structure of solid water ice molecules), or as a series of three snapshots, representing the beginning, middle, and end of an event (e.g., water as a gas being transferred from one container to another). whenever possible, a series of three static pictures was used so that comparable content was presented in both the static and dynamic graphics conditions. thus, even though children in the static graphics cognitive factors that influence children’s learning from a multimedia science lesson / anggoro, stein & jee 99 condition never saw speed or movement conveyed dynamically, they did see three snapshots depicting the beginning, middle, and end points of each event sequence. the dynamic graphics, in the form of quicktime movies, presented actual motion (e.g., liquid water molecules moving over and under one another, water vapor molecules rapidly moving in a container). the learning modules and assessments were presented on individual macbook pro laptops. a data management program, “filemaker pro 8” was used to present the text and graphics, to collect pre and posttest assessment data, as well as to code all of the assessment responses. each study session was audio-recorded on the laptop and on an ipod as a back-up recorder. design and procedure children’s receptive vocabulary and verbal ability were assessed using the peabody picture vocabulary test, third edition (ppvt-iii). children’s performance on the ppvt was computed in terms of standard scores (4 th grade m = 103.54, sd = 14.77; 7 th grade m = 101.22, sd = 15.30) and percentile rank (4 th grade m = 58 th , sd = 29.20; 7 th grade m = 53 rd , sd = 30.13). these scores showed no difference between girls and boys in either grade. in each grade, children were assigned to one of the five conditions using a stratified randomization procedure. stratification assignments were based on children’s ppvt scores to ensure that vocabulary scores were normally distributed and equivalent across the five conditions at each grade level. the four experimental conditions were (1) a tutor-read/static graphics condition, where the text was read aloud to the child in conjunction with the presentation of static graphics, (2) a tutor-read/dynamic graphics condition, where the text was read aloud to the child with the presentation of dynamic graphics, (3) a self-read/static graphics condition, where the child read the text aloud in conjunction with the presentation of static graphics, and (4) a self-read/dynamic graphics condition, where the child read the text aloud with the presentation of dynamic graphics. children in the control group received only pre and posttests, with the same period of time in between the tests as in the experimental conditions. in each of the experimental conditions, children participated individually, with a trained tutor guiding each child through the learning modules and assessments. all text and accompanying graphics were presented on the computer screen, with the text on the left hand side of a computer screen and the accompanying graphics on the right hand side of the screen. in the tutor-read conditions, the experimenter read the text aloud to the child. the child was encouraged to read along silently, but was not required to do so. in the self-read conditions, the child read the text aloud to the experimenter. children in all experimental conditions were also asked to attend to the embedded graphics. children in the four instructional conditions participated in five sessions over an eight to ten-week period of time. in session 1, the ppvt was administered and demographic data were collected. session 2 consisted of a pretest that assessed children’s knowledge of the states of water. session 3 consisted of the presentation of the first states of international electronic journal of elementary education vol.5, issue 1, 93-106, 2012 100 water module (on solids and liquids), immediately followed by an assessment of children’s comprehension of the module. session 4 consisted of the presentation of the second states of water module (on gases and comparison across the three states), and a knowledge assessment immediately following the module. session 5 consisted of the posttest on states of water knowledge. post-testing occurred approximated three to four weeks after the completion of session 4. children in the control condition completed all pretests (sessions 1 and 2) and the posttest (session 5). during the time between pre and posttests, children in the control condition participated in their regular classroom instruction. the pre and posttests included the same items, which were composed of: (1) true/false questions, (2) yes/no questions, (3) explanations for t/f and y/n answers, (4) short answer questions, and (5) open-ended questions. we began by asking children to name the three states of matter. we then asked 10 questions for each state. for the purposes of our analyses, the relevant questions were the following (using solid water as an example): 1. did the shape of the solid change when you transferred it from container 1 (short and skinny) to container 2 (tall and wide)? why or why not? 2. did the volume of the solid change when you transferred it from container 1 to container 2? why or why not? 3. true/false: the solid changes shape as it is transferred from container 1 to container 2. 4. true/false: the solid changes volume as it is transferred from container 1 to container 2. 5. true/false: there are more solid molecules in container 2 than there were in container 1. 6. do you know anything about the molecules that make up solid water ice? 7. what do you know about the molecules that make up solid water ice? scoring. filemaker pro automatically saved children’s pre and posttest responses as the answers were typed into the computer. the computer program automatically scored responses to the t/f and y/n questions. the remaining responses were scored manually (reliability among three coders was 96%). all questions concerning observable properties of water were t/f or y/n, whereas all questions concerning molecular properties of water were open-ended, as described below. for the observable properties (i.e., shape and volume) of solid and liquid water, we tabulated children’s responses to the t/f and y/n questions (questions 1-4 listed above). thus, there were a total of 8 questions, all requiring dichotomous responses. accuracy scores were computed as the proportion of correct responses out of 8. for the molecular properties of each state, children’s answers to the three open-ended questions for each state (see question 7 above for solid) were scored with respect to ideal correct responses and “gist” responses, as described below. cognitive factors that influence children’s learning from a multimedia science lesson / anggoro, stein & jee 101 ideal correct responses were explicitly stated in the text. for each state of water, three components constituted a complete, ideal answer. the components focused on the organization, speed, and movement of molecules in each state (see table 2). in addition to ideal responses, children also provided responses that were acceptable variations on the ideal correct responses (i.e., they maintain the “gist” of the components). for example, some children stated that solid water molecules are frozen in place rather than locked in place. these gist correct responses were coded as correct because they showed that children understood the conceptual content, even though they did not use the exact language provided in the text. all correct responses (i.e., ideal and gist), as well as examples of children’s actual responses, are listed in table 3. table 3. ideal and “gist” correct responses for the molecular properties of the three states of water solid water ideal correct responses examples molecules of solid water are locked in place the molecules of solid water […] locked in place […] molecules of solid water vibrate; jiggle back and forth molecules of solid water ice do not move, but they still vibrate molecules of solid water do not move over and around one another the molecules just vibrate instead of moving around gist correct responses examples molecules of solid water are frozen um, the molecules […] they’re frozen in place molecules of solid water move slower than molecules of liquid water the solid water ice molecules move slower than they would in water molecules of solid water form a lattice structure the molecules are […] in a lattice structure the molecular structure of solid water results in a fixed shape […] they stay in the same shape they were put in before they were frozen. liquid water ideal correct responses examples molecules of liquid water move at a moderate speed; faster than solid water molecules, but slower than water vapor the molecules of liquid water can move but not at a very fast speed […] molecules of liquid water move around, slip and slide over and under one another the molecules of liquid water […] slide under and over each other […] molecules of liquid water cling to one another the molecules of liquid water stick together. gist correct responses examples molecules of liquid water are loose, not locked in place the molecules of liquid water are not locked together-they are loosely packed molecules of liquid water move around more than molecules of solid water; no mention of slipping and sliding the molecules of liquid water […] move around because they aren’t solid so they don’t stay in place molecules of liquid water do not move fast enough to break away from one another move fast but not fast enough […] to break far away from each other international electronic journal of elementary education vol.5, issue 1, 93-106, 2012 102 molecules of liquid water are close together the molecules in liquid water […] stay next to each other. water vapor ideal correct responses examples molecules of water vapor move very fast […] moving really, really quickly molecules of water vapor are able to break away from one another […] they can break away from each other […] molecules of water vapor fill any space in which they are placed (e.g., container, room) […] spread out to make the same size as whatever it’s in table 3.(cont.) ideal and “gist” correct responses for the molecular properties of the three states of water gist correct responses examples molecules of water vapor move around freely in any direction, without reference to breaking away from one another water vapor, the molecules can move in any direction they want […] molecules of water vapor spread out all over, without reference to filling an entire space they spread out all over the place. molecules of water vapor are loose, with a possible reference to other states they’re really loose and not compact at all […] the lack of structure for water vapor molecules results in no fixed volume they have no fixed volume […] the lack of structure for water vapor molecules results in no fixed shape they have […] no fixed shape […] if a child generated at least one ideal correct response for a question, they were given one point. if they were unable to generate at least one ideal correct response for a question, they were given no points. this procedure was applied to each of the three molecular questions (i.e., for solid water, liquid water, and water vapor), and then the mean of the three scores was computed to obtain the proportion of ideal correct responses across all three states. results observable properties of solid and liquid water we expected children to have some prior knowledge about the observable properties of solid and liquid water, especially the older children. thus, pre-post gain scores on observable properties of solid and liquid should be relatively small compared to gains on learning about molecular properties of the three states. nevertheless, we compared learning gains on observable properties across the grade levels and conditions. the results were analyzed with a 2 (grade: 4 th vs. 7 th ) x 2 (reading condition: tutorread vs. self-read) x 2 (graphics condition: static vs. dynamic) between-groups analysis of covariance (ancova). the dependent variable was the pre-post gain score in the proportion of correct responses to y/n and t/f questions about the shape and volume of solid and liquid water. standardized scores on the ppvt were included as a covariate. the analysis revealed a marginally significant effect of grade, f(1, 279) = 3.69, mse = .07, p = .06, ηp 2 = .01, with 4 th grade participants showing greater gain scores (m = .23, cognitive factors that influence children’s learning from a multimedia science lesson / anggoro, stein & jee 103 sd = .25) than 7 th grade participants (m = .15, sd = .28). no other main effects or interactions approached significance, fs < 1.5, ps > .25. in addition to the comparisons between the experimental conditions, we analyzed performance relative to the control condition. the gain scores for the 7 th grade control group (m = .09, sd = .22) were marginally higher than those of the 4 th grade control group (m = -.02, sd = .18), t(39) = 1.78, p = .08. gain scores for the 4 th grade experimental conditions were significantly higher than those for the 4 th and 7 th grade control conditions, ts > 2.12, ps < .05, with the exception of the 4 th grade participants in the self-read/static graphics condition, whose gain scores (m = .18, sd = .29) were not significantly greater than the 7 th grade control participants’, t(39) = 1.18, p = .12. gain scores were generally lower in the 7 th grade experimental conditions, as revealed by the ancova. only the self-read/static graphics condition (m = .19, sd = .26) had higher gains than the 7 th grade control condition, t(53) = 1.72, p < .05. the other 7 th grade experimental conditions had higher gains than the 4 th grade control participants, ts > 2.70, ps < .05, but not the 7 th grade control participants, ts < 1.35, ps > .09. molecular properties of the three states of water the results were analyzed with a 2 (grade: 4 th vs. 7 th ) x 2 (reading condition: tutorread vs. self-read) x 2 (graphics condition: static vs. dynamic) between-groups analysis of covariance (ancova). the dependent variable was a pre-post gain score in the proportion of ideal correct responses to questions about the molecular properties of the three states of water. standardized scores on the ppvt were used as a covariate in the analysis. gain scores were arcsine transformed for the analysis to adjust for the unequal variances between the conditions. the descriptive statistics reported below, however, represent the original scale of measurement. the results are shown in figure 1. the ancova revealed a main effect of grade, f(1, 279) = 6.35, mse = .31, p < .05, ηp 2 = .02, with 7 th grade participants showing greater gain scores (m = .70, sd = .36) than 4 th grade participants (m = .64, sd = .39). there was also a main effect of reading condition f(1, 279) = 46.72, mse = .31, p < .05, ηp 2 = .14, such that participants in the tutor-read condition had greater gain scores (m = .77, sd = .31) than participants in the self-read condition (m = .46, sd = .41). there was, however, no effect of graphics condition, f(1, 279) = 0.54, mse = .31, p = .46, ηp 2 < .01. the analysis also revealed a marginally significant interaction between grade and reading condition, f(1, 279) = 3.15, mse = .31, p = .08, ηp 2 = .01. this trend is due to the fact that 4 th grade participants showed a larger difference in gain scores between the tutor-read condition (m = .77, sd = .30) and self-read condition (m = .36, sd = .41) than the 7 th graders (m = .70, sd = .36 for tutor-read; m = .55, sd = .40 for selfread). no other interactions approached significance, fs < 1, ps > .30. performance of the experimental conditions was also compared to the control conditions. the gain scores for the 4 th grade control group (m = .15, sd = .24) were equal to those of the 7 th grade control group (m = .16, sd = .38), t(39) = 0.10, p = .92. gain scores for the experimental conditions were significantly higher than those for each of the control conditions, ts > 2.20, ps < .05, with the exception of the 4 th grade international electronic journal of elementary education vol.5, issue 1, 93-106, 2012 104 participants in the self-read/static image condition, whose gain scores (m = .16, sd = .38) were only marginally greater than the 7 th grade control participants’, t(42) = 1.28, p = .10. figure 1. mean gain scores in the proportion of ideal correct responses to questions about the molecular properties of the three states of water discussion the present study examined the cognitive factors that influence children’s learning about the observable and molecular properties of the three states of water by manipulating the delivery of a causally-coherent lesson (tutor-read vs. self-read) and the nature of the images that the children received (static vs. dynamic). the results revealed several important findings. first, children at both grade levels had some prior knowledge about the observable properties of solid and liquid water, and 4 th graders showed the greatest improvement at posttest. second, even though children at both grade levels began with little to no knowledge of molecular properties of the three states water, they were able to learn about these properties through the causallycoherent lesson. third, even when vocabulary scores were statistically controlled, children in the tutor-read condition learned more than those in the self-read condition, and this difference was especially pronounced for 4 th graders. finally, at both grade levels children learned equally well regardless of the type of graphics (static vs. dynamic) they received. in the introduction, we characterized the two reading conditions in terms of their costs and benefits to different components of multimedia learning. self-reading provides the benefit of self-pacing the lesson, but does not support integrating of the verbal and visual information. tutor-reading forfeits control of the pace of the lesson, but the temporal contiguity of verbal and visual information processing supports integration of the two modalities. our finding of overall greater learning gains in the tutor-read condition suggests that integrating the verbal and visual information was the greatest challenge to the children. when a tutor reads to the learner, integration, and thus learning, was enhanced. cognitive factors that influence children’s learning from a multimedia science lesson / anggoro, stein & jee 105 it is interesting that the tutorvs. self-read effect was especially pronounced in the younger age group (though, as noted, this interaction was marginally significant). younger children have poorer reading skill and metacognitive awareness than older children, and these variables could have contributed to the 4 th graders’ exacerbated difficulties in the self-read conditions. although we did not collect data on children’s reading level and metacognitive ability, the tutorvs. self-read effect was found when children’s vocabulary—a strong predictor of reading ability and general cognitive development—was statistically controlled. in interpreting these results it is important to take into account potential limitations of the present study. the text that we used was unique because it was designed to be causally coherent. it is possible that children would have benefitted more from selfreading (and therefore self-pacing) if the text lacked this coherence (as is the case in most textbooks), because understanding the content would be more challenging. if the text lacked coherence, the learners would have been required to fill in gaps using their prior knowledge (mcnamara et al., 1996), and the more controlled pace of self-reading could have facilitated this process. another potential concern is that children in the tutor-read conditions may have been more engaged in the lesson than those who were self-reading. that is, it is possible that they paid closer attention to both the verbal and visual information and put more effort into integrating the two. however, a tutor was also present to oversee the children who were self-reading. it is equally plausible that children who were self-reading were more engaged because they had to read the text themselves, making the lesson more interactive for them. finally, our analyses used the children’s grade level/age as a proxy for cognitive control and capacity. ideally, we would have a measure for each of these variables to test their contributions to the learning outcomes and to rule out other ageand context-related differences, such as everyday experience with water, parental/caregiver expertise in science, and the level of scientific discourse in the children’s broader communities. these are important considerations for future research. • • • acknowledgements this work was supported in part by a grant from nsf 0529648. we are indebted to marc hernandez for his contributions. we would also like to thank the principals, teachers, and students involved in this study. florencia k. anggoro received her ph.d. in cognitive psychology from northwestern university. she is currently an assistant professor of psychology at the college of the holy cross. she studies conceptual development, particularly the role of language, culture, and formal learning experiences in shaping children’s concepts. her particular interest is in concept acquisition in stem domains. international electronic journal of elementary education vol.5, issue 1, 93-106, 2012 106 nancy l. stein received her ph.d. in psychology and education from stanford university. for over 35 years she has studied reading, models of comprehension, narrative and argument understanding, emotion, cognitive development, instruction, and learning science. her current research in early science learning combines her expertise in education, instruction, and learning science with her background in mathematics and chemistry. benjamin d. jee received his ph.d. in cognitive psychology from the university of illinois in chicago. he is currently a visiting assistant professor of education at the college of the holy cross. he studies knowledge acquisition and expertise, with particular interest in the visual learning mechanisms that promote stem learning. references bar, v., & galili, i. (1994). stages of children's views about evaporation. international journal of science education, 16(2), 157-174. duschl, r. a., schweingruber, h. a., shouse, a. w., national research council (u.s.). committee on science learning kindergarten through eighth grade. national research council (u.s.). board on science education & national research council (u.s.) (2007). taking science to school: learning and teaching science in grades k-8. washington, d.c.: national academies press. flavell, j. h. ( 2000 ). development of children’s knowledge about the mental world. international journal of behavioral development, 24, 15–23. gobert, j., & buckley, b. (2000). special issue editorial: introduction to model-based teaching and learning. international journal of science education, 22(9), 891-894. goldberg, f., & bendall, s. (1995). making the invisible visible: a teaching/learning environment that builds on a new view of the physics learner. american journal of physics 63(11), 978-991. jose, t. j. & williamson, v. m. (2005). molecular visualization in science education: an evaluation of the nsf funded workshop. journal of chemical education, 82(6), 937-943. klausmeier, h. j. (1992). concept learning and concept teaching. educational psychologist, 27(3), 267. larkin, j., & simon, h. (1987). why a diagram is (sometimes) worth ten thousand words. cognitive science, 11, 65-99. mandler, j. m. (2008). on the birth and growth of concepts. philosophical psychology, 21(2), 207230. mayer, r.e., & anderson, b. (1991). animations need narrations: an experimental test of a dualcoding hypothesis. journal of educational psychology, 3, 484-490. mayer, r.e., & chandler, p. (2001). when learning is just a click away: does simple user interaction foster deeper understanding of multimedia messages? journal of educational psychology, 93, 390-397. mayer, r. e., hegarty, m., mayer, s., & campbell, j. (2005). when static media promote active learning: annotated illustrations versus narrated animations in multimedia instruction. journal of experimental psychology: applied, 11(4), 256-265. mayer, r. e., & moreno, r. (2003). nine ways to reduce cognitive load in multimedia learning. educational psychologist, 38, 43-52. cognitive factors that influence children’s learning from a multimedia science lesson / anggoro, stein & jee 107 mcnamara, d. s., kintsch, e., songer, n. b., & kintsch, w. (1996). are good texts always better?interactions of text coherence, background knowledge, and levels of understanding in learning from text. cognition and instruction, 14(1), 1-43. metcalfe, j., & shimamura, a. p. (1994). metacognition: knowing about knowing. cambridge, ma: mit press. national research council (u.s.) (1996). national science education standards: observe, interact, change, learn. washington, dc: national academy press. project 2061 (american association for the advancement of science) (1993). benchmarks for science literacy. new york: oxford university press. romance, n. r., & vitale, m. r. (2001). implementing an in-depth expanded science model in elementary schools: multi-year findings, research issues, and policy implications. international journal of science education, 23, 373-404. romance, n. r., & vitale, m. r. (2010). effects of an integrated instructional model for accelerating student achievement in science and reading comprehension in grades 1-2. presented at the annual meeting of the american educational research association, denver, co. schneider, w. (2008). the development of metacognitive knowledge in children and adolescents: major trends and implications for education. mind, brain, and education, 2, 114– 121. shwartz, y., weizman, a., fortus, d., krajcik, j., & reiser, b. (2008). middle school science curriculum: coherence as a design principle. paper presented at the national association of research in science teaching. slotta, j. d., & chi, m. t. h. (2006). the impact of ontology training on conceptual change: helping students understand the challenging topics in science. cognition and instruction, 24, 261-289. stein, n. l., hernandez, m. w., & anggoro, f. k. (2010). a theory of coherence and complex learning in the physical sciences: what works (and what doesn't). in n. l. stein & s. w. raudenbush (eds.), developmental science goes to school. new york: taylor and francis, inc. stein, n. l., hernandez, m. w., anggoro, f. k., & hedberg, e. (under review). a developmental study of physical science learning: modeling invisible processes and starting early. stein, n. l., & levine, l. j. (1989). the causal organization of emotional knowledge: a developmental study. cognition & emotion, 3(4), 343-378. stein, n. l., & miller, c. a. (1993). a theory of argumentative understanding: relationships among position preference, judgments of goodness, memory, and reasoning. argumentation, 7, 183-204. stein, n. l., & trabasso, t. (1982). what's in a story: an approach to comprehension and instruction. in r. glaser (ed.), advances in instructional psychology, 2, 212-267. hillsdale, nj: lawrence erlbaum associates. thagard, p. (2000). coherence in thought and action. cambridge: the mit press. trabasso, t., & bouchard, e. (2002). teaching readers how to comprehend text strategically. in block, c.c. & m. pressley (eds.), comprehension instruction: research-based best practices (pp. 176-200). new york: the guilford press. international electronic journal of elementary education vol.5, issue 1, 93-106, 2012 108 trabasso, t., secco, t., & van den broek, p. w. (1984). causal cohesion and story coherence. in h. mandl, n. l. stein & t. trabasso (eds.), learning and comprehension of text (pp. 83-111). hillsdale, nj: lawrence erlbaum associates. trabasso, t., & stein, n. l. (1997). narrating, representing, and remembering event sequences. in p. w. van den broek, p. j. bauer & t. bourg (eds.), developmental spans in event comprehension and representation: bridging fictional and actual events (pp. 237-270). hillsdale, nj: lawrence erlbaum associates. tversky, b., heiser, j., lozano, s., mackenzie, r., & morrison, j. (2008). enriching animations. in r. lowe & w. schnotz (eds.), learning with animation. cambridge: cambridge university press. vosniadou, s. & brewer, w. f. (1992). mental models of the earth: a study of conceptual change in childhood. cognitive psychology, 24, 535-585. winston, p. (1986). learning by augmenting rules and accumulating censors. in r. s. michalski, carbonell, j. g., & mitchell, t. m. (ed.), machine learning: an artificial intelligence approach. new york: morgan kaufmann. microsoft word iejee_5_3_karatas international electronic journal of elementary education, 2013, 5(3), 249-268. the effect of learning environments based on problem solving on students’ achievements of problem solving ilhan karatas ∗∗∗∗ bülent ecevit university, turkey adnan baki karadeniz technical university, turkey received: 13 april 2013 / revised: 25 june 2013 / accepted: 3 july 2013 abstract problem solving is recognized as an important life skill involving a range of processes including analyzing, interpreting, reasoning, predicting, evaluating and reflecting. for that reason educating students as efficient problem solvers is an important role of mathematics education. problem solving skill is the centre of mathematics curriculum. students’ gaining of that skill in school mathematics is closely related with the learning environment to be formed and the roles given to the students. the aim of this study is to create a problem solving based learning environment to enhance the students’ problem solving skill. within this scope, students’ practiced activities and problems that provide them to proceed in polya (1945)’s problem solving phases and throughout the study, students’ success in problem solving have been evaluated. while experimental group students received problem solving based learning environment performed, control group students have continued their present program in this quise-experimental study. eleven problem solving activities were given to the students at the beginning, middle and end of the study and the students’ performances were analyzed based on problem solving phases. the findings illustrated that the experimental group students’ success in problem solving activities has increased while the control group students’ success has not changed significantly. keywords: mathematics education, problem solving, polya’s problem solving phases. introduction an in our everyday lives, we use problem-solving skills. also, most of us have to make daily plan, make decisions in our business and manage our budget. all of these events require logical thinking and also problem solving skill (weidemann, 1995). problem solving is ∗ ilhan karatas, bülent ecevit university, eregli faculty of education, department elementary education, karadeniz eregli, zonguldak, turkey. phone: +(90) 372 323 38 70. e-mail: ilhankaratas@beun.edu.tr the effect on learning environments based on problem solving / karatas & baki 250 recognized as an important life skill involving a range of processes including analyzing, interpreting, reasoning, predicting, evaluating and reflecting (anderson, 2009). for these reasons one of the aims of mathematics teaching is to educate students as efficient problem solvers (baki, 2008). therefore, problem solving is considered as a central to school mathematics. it is highlighted in reform documents by national council of teachers of mathematics (nctm, 1989, 2000) as a key factor of change in mathematics education. nctm (2000) states that students should be given chance to apply and adapt a variety of appropriate strategies to solve problems; and monitor and reflect on the process of mathematical problem solving in instructional programs during the problem solving process. similarly, kilpatrick, swafford, and findell (2001) argue that problem solving provides an important context for students to learn numbers and other mathematical terms and problem-solving ability is enhanced when students have opportunities to solve problems themselves and to see problems being solved. thus, problem solving is important as a way of doing, learning and teaching mathematics. therefore, preparing mathematics curriculum in the centre of problem solving appears to be important. problem solving in the reform movement over the past decades, there have been many changes in mathematics teaching. there are some foundations in that period of change such as the national council of teachers of mathematics (nctm, 2000), the national research council (nrc, 1989) as well as the third international mathematics and science study (timss) and the ontario mathematics curriculum (ontario ministry of education and training, 1997). these foundations put an emphasis on problem solving in mathematics learning. in compliance with nctm, problem solving is an integral part of all mathematics learning. so, problem solving should not be an isolated part of the curriculum” (nctm, 2000). at the same time, the teacher has an important role in the development of students’ problem solving skill and the teachers must choose problems that engage students (nctm, 2000). similarly, nrc states in its report which was published in 2001 that problem solving ability is enhanced when students have opportunities to solve problems themselves and see problems being solved. problem solving also provides opportunities with teachers to assess students’ performance (kilpatric, et al., 2001). timss, providing trend data on students’ mathematics and science achievement from an international perspective, gives mathematics educators many educational implications. timss data show higher mathematics achievement when teacher emphasize reasoning and problem solving activities (mullis, et al., 2000). according to timss, the students in japan are more successful than the students in us and canada. the factor behind that difference is that while 49% of the teachers in japan emphasize reasoning and problem solving, this rate for the teachers in us and canada is 18% and 13% successively (mullis et al., 2000). therefore, a correlation between problem-solving and students' achievement in mathematics is seen clearly. based on nctm standards, problem solving has been emphasized in the curriculum that has been prepared in ontario state in canada. that curriculum describes problem solving as a skill that should be along with mathematics teaching. however, in the curriculum students should use problem solving methods not only in problem solving task in mathematics, but in other appropriate circumstances. they should also use problem solving methods extensively as a means of developing the full range of mathematical skills and knowledge in all strands (ontario ministry of education and training, 1997). in the light of changes that have occurred in mathematics teaching, in turkey mathematics curriculum of primary and secondary schools has been renewed in 2005 with the reforms in the effect on learning environments based on problem solving / karatas & baki 251 education. the mathematics curriculum bases on the principle that every child can learn mathematics and lays stress on basic mathematical skills such as problem solving, communication and reasoning. as one of the most important goals, the national curriculum by turkish ministry of education (meb) defines problem solving as not a subject matter to be taught but a process helping students to gain essential skills to solve problems. how the students solve the problem, which data contribute to that solving, how they represent that problem (table, figure, concrete object etc.), how the strategy that they chose and representation manner make the solution easier, and how the students explain the solution to their peers should be emphasized in that curriculum (meb, 2006). role of problem solving in mathematics education problem solving has an important role in mathematics teaching and it is also been the centre of mathematics programs (nctm, 1989; nctm, 2000, meb, 2006; howland, 2001). thus improving the students’ problem solving skills have been emphasized in the program of mathematical studies. problem solving enables students to do mathematics and to comprehend mathematics meaningfully (van de walle, 2001). when it is taken into account that permanent learning takes places at social surroundings, artzt and armour-thomas (1992) state that problem solving settings that based on class discussion gives students a chance to analyze their thought, students can share and compare their thought with their peers in that setting and that setting also makes discussion of different ideas possible. it is pointed out that problem solving has some advantages such as developing students’ responsibility, directing them to searching, raise their interest for learning, providing students with permanent learning, increasing students’ motivation etc. (fisher, 1990). therefore, process of learning should help students to develop a sense of being responsible for their learning. an effective learning process also helps students to grow their interest in learning and promote students to share ideas to each other and finally make learning as stable as possible teachers have many opportunities to build knowledge about teaching problem solving and using problems as a focus of learning in mathematics (cai, 2003). when used as methods for instructional method, it allows students their own understanding and takes some ownership for their learning. additionally, students perceive an active role in problem solving activities by which their thoughts and ideas become a focus of learning activities (annable, 2006). in addition, schoenfeld (1992) advocated that problem solving based learning environments enables students to have deep mathematics knowledge and gives them the opportunity of pursue their own mathematics learning enthusiasm. hiebert and wearne (2003) point out that the process of problem solving improves and enrich students’ mathematical perception. annable (2006) has taught mathematics to6th grade students on the basis of problem solving so as to enhance their problem solving and critical thinking skills. he also revealed that when problem solving strategies are stressed in the learning environment, and the students discuss the problems with their peers, students’ skill of problem solving advances. similarly, perveen (2010) has carried out a study on the effect of problem solving on the success of 10th graders. for that experimental study based on polya (1945) (heuristic phases of the problem-solving approach) was performed and the students in experimental group was taught by problem solving approach, thereafter the study revealed that the academic achievement of the students in experimental group is much better than the students in control group. schoenfeld (1989) also performed a study that based on development of high school students’ metacognitive skills. according to the result of that study, class discussion oriented teaching gives students opportunity to express their ideas and share those ideas with their peers. therefore, it is necessary to design learning environment which is suitable the effect on learning environments based on problem solving / karatas & baki 252 for the development of students’ problem solving skill. moreover, it is relatively important for students to share their thoughts with their peers in problem solving environment. purpose of the study the purpose of this study is to develop students’ problem solving skill by designing learning environment that based upon problem solving. within this scope, learning environments where the students have the chance to pace polya (1945)’s problem solving phases have been designed and students’ problem solving and their progress have been evaluated. method participants this study’s sample consists of 53 7th grade students. 27 of them are experiment group and 26 of them are control group. these students’ 6th grade mathematics final exam results were compared and there was no significant difference (t (51) = 1.298, p>.5) between the groups. thus, the experimental and control group students' math competencies were observed to be similar before the study. students in experimental and control groups were taught by the same teacher. study context turkish education is compulsory for every turkish citizen from the age of six to the age of eighteen, regardless of gender and socio-economic status, and is free of charge in state schools across the country. the education system comprised four years of primary school, four years of middle schools, and four years of secondary schools. the secondary school where the study carried out is located in an area where middle income families live. furthermore, mathematics teacher who takes part in this study has a master and degree in mathematics teaching and currently she was pursuing her doctorate degree by the time of the study. the teacher also participated in professional development programs and has qualifications that can make him an expert in his area. the teacher graduated from mathematics teaching programme in 1999 and has been working as a secondary school teacher since that date. thus, the teacher can practice mathematics curriculum efficiently in that process. the teacher in that study is regarded to be close to current development in mathematics teaching area and can adapt them easily as he continues to doctorate programme. for all of these reasons, teacher’s readiness for innovation is really important for the study to be performed effectively. the teacher gives importance to student centered education and gives place to students’ thought in the class. procedure experimental group. as the learning environment is very important for the development of students’ problem solving skills, how the learning environment should be was decided first. it was necessary to determine problems that the students would discuss and solve in that learning environment. the teacher who would teach control and experimental groups was chosen and the gains of the curriculum for 7th graders were defined and how they should be performed in the class were decided. how the problems should be dealt within the classroom was discussed with the teacher who would teach these participating groups. the worksheets that would to be given the students and the phases of problem solving were arranged and the instructions in those worksheets were outlined. five pilot studies were done on teacher’s gaining experience, context of worksheets, deciding on the problems that would be used in the learning environment and researcher’s gaining experience. in pilot study, researcher made observation and took observation notes for teacher to use the problems effectively in the classroom. in parallel with observational data of researcher and the effect on learning environments based on problem solving / karatas & baki 253 opinion of participating teacher, final shape of the problems that would be used in the learning environment was structured. nine problems used in the learning environment were determined with the teacher. the problems were prepared considering their being applicable to multiple ways of solutions and results, directing students to discussion, involving problem solving skills and being suitable for concept learning. a sample problem used in the study was given in figure 1. miss yasemin goes to the grocery store to supply her three months rice need. she wants to buy 16 kg rice. however, rice is sold in bags of different sizes, as shown in the figure below. decide on which bags miss yasemin should buy to make the most effective shopping. figure 1. a sample problem (tl represents turkish liras). in the second part of the study, nine problems were carried out on experimental group students in parallel with polya's (1945) heuristic phases of the problem-solving approach to improve students’ problem solving skills. the mathematics teacher in the experiment group implemented the connected mathematics project’s (cmp) instructional model: “launch, explore and summarize” in their teaching. this model of instruction involves three main phases. in the launch phase, the teacher explained the problem to whole class and tried to make students understand and raise their interests for the problem situation. in the explore phase, students search and try to find possible solutions for the problem situations either individually or in groups during that process. in that phase, while students were dealing with solutions of the problems, the teacher were observing them and giving them tips in some situations. the teacher tried to form the basis for the class discussion by analyzing the solutions and process. in the summary phase, students discuss their solutions and share their strategies they used to reach a solution. students decide on the most appropriate way of solution and solve the problems with the help of their teacher. they will also appreciate other approaches proposed by their peers to the problem, and can see ways to enhance their own strategies. the teacher also offers guidance and suggestions for a deeper understanding of the concepts and more effective and efficient problem solving strategies (reys, reys, lappan, holliday, & wasman, 2003). control group. the existing mathematics curriculum was applied to the control group. that curriculum aims to develop students’ problem-solving, reasoning, communication skills by means of activity-based approach (meb, 2006). teaching was preceded in accordance with the examples in teacher’s book. both experimental and control groups were taught by the same teacher. teaching process was carried out by providing the terms and definitions in mathematics teaching curriculum. data collection instrument in order to evaluate students’ problem solving performance 11 problems that involve the subjects in 7th grade curriculum were used as a means of collecting data. the problems which the effect on learning environments based on problem solving / karatas & baki 254 would give use chance to evaluate their problem solving skills and that process were carried out in line with the teacher’s view. the problems were separated into three different groups and were distributed to experimental and control groups and students’ solving were taken under review. the problems that were prepared with participating teacher have been considered to contain a different solution methods and problem-solving skills. both experimental and control groups practiced three of the problems in initial phase, four of them in middle phase and the other four problems in final phase. these problems were applied to the students and were assessed. among the problems used as a means of data collecting tool chocolate problem is shown in table 1. table 1. sample problems in data collection instrument chocolate problem different sizes of chocolate sold in the supermarket are presented below. prices in the table are determined by the number of packet type and chocolate. think about which package you choose as a customer? please do and explain the problem. package type numberof chocolate price small package 2 items 36 kuruş middle package 5 items 1 tl family size package 12 items 2.2 tl king size package 18 items 3 tl data analysis solutions that the students found out for 11 problems consisted of the data. therefore, there have been 11 different problem worksheets for both control and experimental groups at the end of the study. the students’ problem worksheets were scored with the help of a scale prepared by northwest regional educational laboratory mathematics and science education center (nwrel) and used to evaluate other problem solving studies (url, 2006). according to the scale, problem solving phases were given digital numbers as one, two, three, and four points (understanding the problem, developing a plan, carrying out the plan, and looking back). in the problem phase of understanding the problem, if the students understand the problem completely, they are valued with four points. if they understand a little, they are valued with three points. if they don’t understand, they are valued with two points. if they don’t pay any effort to understand the problem, they are valued with one point. in the phase of developing a plan, if the students choose a suitable strategy to lead them the solution, they are given four points. if they choose only a piece of the strategy that helps them solve the problem, they are given three points. if they choose an unsuitable strategy, they are given two points. if they don’t choose any strategies, they are given one point. in the phase of carrying out the plan, if the students reach the correct solution, they are given four points. if they find some parts of the solution correctly, they are given three points. if they find out an incorrect solution, they are given two points. if they don’t reach any solutions, they are given one point. in the phase of looking back, if the students confirm the results logically, they get four the effect on learning environments based on problem solving / karatas & baki 255 points. if the students confirm the result partially, they get three points. if they don’t know how to confirm the result, they get two points. if they don’t confirm the result, they get one point. consequently, the total score that the students get from 11 problems and the scores from each phases of study were used to assess each student’s problem solving skills. to determine the students’ problem solving success, the points that the students got from the problems applied to them in three different times were calculated. the average of each student’s points that they got from the first three of the problems were calculated as the points that they got from the first phase. the average of fourth, fifth and sixth problems’ points were calculated as second phase points and the average of the points taken from the last four of the problems were regarded as third phase points. so, the points which each student got from three stages were obtained. to compare the results of control and experimental groups, t test and analysis of covariance (ancova) test were applied. results in this section, the development of problem-solving success of the experimental and control group students who participated in the study was examined. students’ results from total of 11 problems used as a data collection tool were calculated separately for each problem. problems were applied to the students as separate groups like three, four and problem four. scores of students that they got from the each group of problem were calculated and students’ problem solving scores were obtained. problem-solving scores of students in the experimental and control groups were compared using independent t test and ancova. experimental and control groups students’ scores of first and second application problems were compared using the t test for independent groups obtained data were summarized in table 2. table 2. the results of paired sample t test between experimental and control groups of students’ total scores in first and second application. variables groups n x sd df t p first application experimental 26 8.942 2.421 51 0.836 .407 control 27 8.481 1.501 second application experimental 26 9.346 1.547 51 3.011 .004 control 27 8.157 1.321 according to the data in the table; while there was no significant difference between experimental and control groups students’ scores that they got from the problems in the first practice (t(51)=.836, p>.05), in the second practice differences were found between the scores of the student (t(51)=3.011, p>.05). for that reason, an explicit difference between experimental and control groups students emerged towards the end of the study. ancova test was carried out in order to evaluate experimental and control group’s students’ success of problem-solving at the beginning and end of the study. ancova results that the students obtained from the first and third performance are shown in table 3. the effect on learning environments based on problem solving / karatas & baki 256 table 3. descriptive analysis of experimental and control groups of students’ scores in first and third application. groups n aplicationfirst x napplicatio third x napplicatio thirdsd napplicatio hirdcorrected x experimental 26 8.942 10.012 0.989 9.903 control 27 8.481 8.197 1.114 8.303 according to the data in the table; for the first practice applied in the beginning of the study, experimental group students’ score average is x = 8.942; on the other hand control group students’ score average is x = 8.482. however, for the third performance applied at the end of the study experimental groups students’ score average is x = 10.012 and the control group students’ score average is x = 8.197. these results show that while experimental group students’ problem solving success has increased, control group students’ problem solving success has decreased. in order to evaluate the groups’ success of problem solving in first and third practice, ancova test was applied and the results are given in the table 4. table 4. ancova results of experimental and control groups’ scores in the third application which was organized according to the first application. source of variance sum of squares df mean square f sig.(p) first application 22.074 1 22.074 31.817 .000 group 33.021 1 33.021 47.597 .000 error 34.688 50 .694 total 4477.819 53 according to the data given in the table it seen that adjusted difference between problemsolving scores of problem-solving groups is statistically significant (f (1-50) =47.597, p<.05). it shows that problem solving based method of instruction improves the students’ problem solving success positively. as shown in table 5, multiple and repetitive tests were carried out in order to observe the change depending on the effects of problem solving based method of instruction on problem solving success. table 5. the results of repeated measures analysis about the success of problem solving impact wilks’ λ f sd p 2 ω strength time .91 2.42 2 .10 .09 .47 time*experiment .84 4.82 2 .01 .16 .77 according to the data in the table; it is seen that there is not any significant difference on interference program for time, at the level of wilks’ λ = .91, f = 2.42, p>.05.on the other hand, it is observed that for same variance time* experiment effect is significant at the level of wilks’ λ = .84, f = 4.82, p<.05. as regards these results, even if some or all of the students in the effect on learning environments based on problem solving / karatas & baki 257 control group were in experimental group and some or all of the students in experimental group were in control group, the improvement in problem solving success of the students who were taught in the environment depending on problem solving has progressed remarkably. therefore, if the study had continued in the same way, while there wouldn’t be any change on the problem solving success of the students in control group, the rise in the success of the students in experimental group would continue. in the process of students’ problem solving, students’ scores in each step were calculated and evaluated statistically in order to compare them according to polya’s problem solving phases. the scores that the students in control and experimental groups took in the process of understanding the problem were compared with the help of independent t test. obtained results were given in table 6. table 6. the results of paired sample t test between experimental and control groups of students’ scores in first and second application about the phase of understanding the problem variables groups n x sd df t p first application experimental 26 2.756 .467 51 .175 .862 control 27 2.775 .308 second application experimental 26 2.878 .209 51 1.925 .060 control 27 2.731 .329 regarding the results in the table; there isn’t any significant difference between control and experimental groups students’ scores that the students got for understanding the problem phase in the first application (t(51)= .175, p>.05). similarly, there isn’t any significant difference between control and experimental groups students’ scores that the students got for understanding the problem phase in second application (t(51)= 1.925, p>.05). however, when the scores that the students in control and experimental groups got in comprehension phase were examined, control group students’ average is x = 2.736 for the first application and the raise for the second application is x = 2.878, and in the control group x = 2.775 for the first application and x = 2.731 for the second application it noticed that even if it is just a bit, there is a drop in the scores. descriptive statistics of scores that the students got from the first and third application in understanding the problem phase are given in table 7. table 7. descriptive analysis of experimental and control groups of students’ scores in first and third application about understanding the problem groups n aplicationfirst x napplicatio third x napplicatio thirdsd napplicatio hirdcorrected x experimental 26 2.756 2.951 .15842 2.952 control 27 2.775 2.722 .25737 2.722 according to data in the table; while the control group students’ average scores taken from the problems in the first application is x = 2.756 for understanding the problem phase, the effect on learning environments based on problem solving / karatas & baki 258 control group students average score is x = 2.775.at the end of the study average scores of the problems in the third group for experimental groupis x = 2.951, on the other hand, control group students’ average is x = 2.722. all these findings show that experimental group students who were taught in the environment based on problem solving made a progress in understanding the problem phase. ancova results for the third application are given in table 8. table 8. ancova results of experimental and control groups’ scores in the third application which was organized according to the first application about understanding the problem source of variance sum of squares df mean square f sig.(p) first application .004 1 .004 .080 .778 group .702 1 .702 14.955 .000 error 2.346 50 .047 total 428.993 53 according to data given in table; it is clear that there is a significant difference between adjusted problem solving scores of the groups related with groups’ understanding the problem phase(f(1-50)=14.955, p<.05). this result indicates that the students’ success related to understanding the problem phase has improved positively throughout the study. the scores that the control and experimental group’s students got in the planning phase were compared using t test. obtained results were given in table 9. table 9. the results of paired sample t test between experimental and control groups of students’ scores in first and second application about the phase of developing a plan. variables groups n x sd df t p first application experimental 26 2.384 .664 51 .987 .329 control 27 2.224 .494 second application experimental 26 2.509 .465 51 2.864 .006 control 27 2.151 .445 according to data in table; it is seen that there is no difference between the experimental and control groups students’ average score in the first application of plan phase (t(51) = 0.987, p >.05). however, experimental and control groups students’ average scores appear to be different from each other(t (51) = 2.864, p <.05). when the experimental and control groups students’ scores in plan phase were analyzed, the raise in the scores of control groups students is clear ( x =2.384 for the first application and x = 2.676 for the third application). the effect on learning environments based on problem solving / karatas & baki 259 on the other hand, there is a decrease in control groups students’ scores ( x = 2.224 for the first application and x = 2.197 for the third application). descriptive statistics of experimental and control groups students’ scores from the first application carried out at the beginning and the third application carried out at the end of the study which is about plan phase can be seen in table 10. table 10. descriptive analysis of experimental and control groups of students’ scores in first and third application about developing a plan groups n aplicationfirst x napplicatio third x napplicatio thirdsd napplicatio hirdcorrected x experimental 26 2.384 2.676 .26908 2.653 control 27 2.224 2.197 .29612 2.220 according to data in the table; while control group students’ average score taken from the problems in plan phase is x = 2.384, experimental group students’ average score is x = 2.225. moreover, experimental group students’ average score from the problems in third application is x = 2.676; however, control group students’ average score is x = 2.197 at the end of the study. this point out that the students of experimental group in which problem based learning environment is conducted showed progress in relation to the planning phase while the control group did not alter in the same way. ancova results of groups’ third application are given below. table 11. ancova results of experimental and control groups’ scores in the third application which was organized according to the first application about developing a plan source of variance sum of squares df mean square f sig.(p) first application 1.237 1 1.237 21.682 .000 group 2.436 1 2.436 42.699 .000 error 2.853 50 .057 total 320.701 53 according to data in table; it is understood that the difference between the groups’ adjusted problem solving scores about developing a plan is significant (f(1-50) =42.699, p<.05). this shows that learning environment based on problem solving effects students’ success related with developing a plan positively. the scores that the experimental and control groups students got in the carrying out the plan phase has been compared by conducting t test. obtained results were presented in table 12. the effect on learning environments based on problem solving / karatas & baki 260 table 12. the results of paired sample t test between experimental and control groups of students’ scores in first and second application about the phase of carrying out the plan variables groups n x sd df t p first application experimental 26 2.224 .739 51 1.482 .145 control 27 1.948 .593 second application experimental 26 2.217 .573 51 3.010 .004 control 27 1.759 .536 according to data in the table; in the first application experimental and control groups students didn’t get significantly different scores in carrying out the plan phase (t(51)= 1.482, p>.05). on the other hand, in the second application significantly different scores were seen between experimental and control group students. (t(51)= 3.010, p<.05). the descriptive statistics of scores that the control and experimental group students got from the carrying out the plan phase’s problems performed in the beginning and end of the study can be seen in table 13. table13. descriptive analysis of experimental and control groups of students’ scores in first and third application about carrying out the plan groups n aplicationfirst x napplicatio third x napplicatio thirdsd napplicatio hirdcorrected x experimental 26 2.224 2.461 .410 2.433 control 27 1.948 1.910 .460 1.938 according to data in the table; in the first application experimental group students’ average score taken from the problems in the carrying out the plan phase is x = 2.224; however, control group students’ average score is x = 1.948. furthermore, at the end of the study experimental group students average score taken from the problems in the third application is x = 2.461 and control group students’ average score is x = 1.910.it is obvious that experimental group students who were taught in problem solving based learning environment has improved in problem application phase. in the third application conducted at the end of the study the relation between the groups about the planning phase has been described with ancova test and obtained results are in table 14. table 14. ancova results of experimental and control groups’ scores in the third application which was organized according to the first application about carrying out the plan. source of variance sum of squares df mean square f sig.(p) first application .846 1 .846 4.763 .034 group 3.086 1 3.086 17.380 .000 error 8.879 50 .178 total 265.812 53 the effect on learning environments based on problem solving / karatas & baki 261 according to data in the table; it is seen that the difference in the adjusted problem solving scores related with groups’ carrying out the plan phase is significant (f(1-50) =17.380, p<.05).these points out that learning environments have a positive influence on the success of students about carrying out the plan phase and experimental group students show significant difference when compared with control group. experimental and control groups students’ score that the students got from the looking back phase have been compared by applying independent t test. obtained results were presented in table 15. table 15. the results of paired sample t test between experimental and control groups of students’ scores in first and second application about the phase of looking back variables groups n x sd df t p first application experimental 26 1.717 0.475 51 1.120 .268 control 27 1.564 0.514 second application experimental 26 1.759 0.460 51 3.464 .001 control 27 1.376 0.336 according to data in the table; it is apparent that there is no difference between the experimental and control groups students’ scores for the first application of looking back phase (t(50) = 1.120, p>.05). in the second application the difference between experimental and control groups students’ score is clear (t (51) = 3.464, p<.05). descriptive statistics of experimental and control groups students’ looking back phase scores that they got from the problems applied at the beginning and end of the study can be seen in table 16. table 16. descriptive analysis of experimental and control groups of students’ scores in first and third application about looking back groups n aplicationfirst x napplicatio third x napplicatio thirdsd napplicatio hirdcorrected x experimental 26 1.717 1.826 .322 1.792 control 27 1.564 1.342 .387 1.376 according to data in the table; when the scores for looking back phase have been analyzed throughout the study, it can be seen that there is not an observable difference between experimental and control groups. experimental group students’ average score from the first application of the problems is x = 1.717 whereas control group students’ average score is x = 1.564. moreover, at the end of the study experimental group students’ average score from the problems of third application is x = 1.826 and control group students’ score is the effect on learning environments based on problem solving / karatas & baki 262 x = 1.342. the relation between the groups about the looking back phase is defined with ancova test and is presented in table 17. table 17. ancova results of experimental and control groups’ scores in the third application which was organized according to the first application about looking back source ofvariance sum of squares df mean square f sig.(p) first application 1.920 1 1.920 20.952 .000 group 2.221 1 2.221 24.233 .000 error 4.583 50 .092 total 141.951 53 according to data in the tablet here is a significant difference between the groups’ adjusted problem solving scores of looking back phase (f (1-50) =24.233, p<.05). this makes it obvious that problem solving based learning environment improves the students’ success about looking back phase. discussions and conclusion this study focused on helping students to develop their problem solving skills and achivement in mathematics through a learning activity designed by polya's (1945) heuristic phases of the problem-solving approach. the study revealed that while the experimental group students’ achievements of problem solving increased, control group students’ achievement on problem solving have not changed significantly. this difference might be attributed to the learning environment applied to the experimental group students. polya’s problem solving phases and problem solving strategies were discussed in that learning environment based on problem solving. in this learning environment it is aimed to provide students with a heuristic problem solving experience. barrett and compton (2003) emphasized that an effective problem solving experience helps students expand their thinking, encourages persistence through difficulties, and empowers them to navigate their own learning. for that reason problem solving experiences that will be provided for the students may likely cause to the development of students’ problem solving success, thus their skills will improve. it is observed that both the experimental and control groups students’ average scores in problem solving success test were similar at the beginning of the study, the experimental group students’ average scores in problem solving test applied in the middle and end of the study have increased while the data did not show a similar change for the control group students. when we look at the findings of the relevant studies aiming to develop students’ problem solving skills, we see a similar pattern. there is a consensus among these studies that problem solving strategies hold a great promise to enhance students’ problem solving skills in mathematics learning (keller, 1990; lee, 1982; yazgan & bintaş, 2005; verschaffel, de corte, lasure, vaerenbergh, bogaerts, & ratinckx, 1999; garnette, 1990; altun, memnun & yazgan, 2007) improves and their problem solving success increases. this study examined the students’ development in polya’s (1945) problem solving phases and concluded that in the understanding the problem phase, the experimental group students’ success in the problems applied to them rise significantly; however, there is no the effect on learning environments based on problem solving / karatas & baki 263 difference in the success of control group students in that phase. in the understanding the problem phase the experimental group students used complex skills of dealing with shapes, tables, diagrams to solve their problems. on the other hand, control group students have only written the data and asked stable questions for the problems. that inclination of experimental group students results from the emphasis that is for the importance of understanding the problem phase in the problem solving process in the learning environment and students’ discussion of different strategies for analyzing the problems. in the study of rose (1991), it is understood that the students are not usually aware of the knowledge which helps them for the process of problem solving in understanding the problem phase. in this respect, the things that can contribute to the solution process have been discussed in the learning environment where the experimental group students were successfully employed. therefore, the students’ success in the understanding the problem phase have advanced by this research. there was no statistically important difference between the experimental and control groups students’ achievements in the problem test applied at the beginning of the study, on the other hand, in the problems applied in the middle and end of the study there was a significant difference toward the experimental group students. at this stage, the students have difficulty in the selection of strategies which help them for the solution. this result has coincided with cmajdalka’s result (1999). it can be said that in the learning environments there are various strategies to reach the problem’s result and due to the discussion of specific strategies that the students use in the classroom, there is a change on behalf of experimental group students. similarly, in phase of carrying out the plan the experimental group students’ success in the problems administrated throughout the study has progressed. at this stage, both the control group and the experimental group students have made errors in the process of solution. the students have shared the solution process and activities with their peers and the calculation errors made in the process of carrying out the plan have been emphasized in the learning environments. thus, it is realized that towards the end of the study there were reductions in the experimental group students’ errors. however, it is concluded that the students are insufficient in looking back phase. in looking back phase, when the experimental and control group students’ average scores that they took from the first, second and third groups of problems were analyzed, even if it is not statistically significant, while there is an increase in the average scores of the experimental group students, the control groups’ scores on this phase showed a decrease. this result can be attributed to the students’ ineffective use of looking back phase and their getting further away from that tendency. although various solution process and strategies were discussed throughout the looking back phase in the learning environment, it is seen that the student have not used looking back phase effectively. especially, the students reaching the conclusion by setting up equations decided accuracy of the result by putting the found value in the equation in looking back phase. this situation is observed to be more common among the control group students, while it was found less among the students in the experimental group. that can result from the situation that throughout the study the experimental group students give importance to problem solving skills by using various ways for looking back phase in the learning environment. in mubark and zaman’s study (2012) it is found out that the students had difficulty in looking back phase. the result obtained from this study matches up with mubark and zaman’s results (2012). this study includes problem solving activities that are applied to problem solving based learning environment and instructions that help students to successfully complete problem solving phases and also while the results were discussed, discussion regarding the problem the effect on learning environments based on problem solving / karatas & baki 264 solving phases was condensed. for this reason, at the end of the study it is emerged that the experimental group students were appeared to be more successful than control group students in preceding the problem solving phases throughout the application phase. the results in this study match up with the studies in literature (nancarrow, 2004; seaman, 1995; stacey, 1992; pouradavood, 2003; pugalee, 2001; diezmann, watters & english, 2001). educational implications in the process of problem solving when the problem solving phases which polya (1945) suggested are carried out successfully and efficiently, the students’ problem solving skills and achievements improve significantly. therefore, in mathematics education students should be provided with the activities to proceed in problem solving phases in the learning environments that are enriched with problem solving activities. the importance of taking systematic phases in problem solving process should be emphasized for the students. besides, in this study students’ various problem solving processes were discussed in the classroom and evaluations related with the proposed solutions were made. thus, the problems that will be solved using different problem solving strategies should be discussed in the learning environment. the students should be given chance to evaluate their peers’ proposed solutions in classroom discussions. . . . acknowledgements the authors would like to thank ercan atasoy and his students for participating in this research. ilhan karatas received his ph.d. in mathematics education, from karadeniz technical university in turkey in 2008. he is an assistant professor at the department of elementary mathematics education at the bulent ecevit university. his research interests focus on problem solving, computer-based mathematics education and teacher education. adnan baki is a professor at the department of secondary science and mathematics education in karadeniz technical university, turkey. he was awarded the degree of md in mathematics education in 1989 from the university of new brunswick in canada. he received a phd from the institute of education in university of london in 1994. he publishes papers on teaching and learning mathematics. at present he has a strong interest in the field of teaching and learning mathematics with computers. references baki, a. (2008). kuram ve uygulamada matematik eğitimi [mathematics education in theory and practice], ankara: harf yayınları. altun, m., memnun, d. s. & yazgan, y. (2007). the primary school teacher trainees’ skills and opinions on solving non-routine mathematical problems. elementary education online, 6(1), 127-143. anderson, j. (2009). mathematics curriculum development and the role of problem solving. in k. school (ed) proceedings of 2009 australian curriculum studies association national biennial conference. curriculum: a national conversation (pp. 1-8). (2-4 oct 2009) http://www.acsa.edu.au/pages/page484.asp. the effect on learning environments based on problem solving / karatas & baki 265 annable, c. j. (2006). developing critical thinking skills and mathematical problem solving ability in grade six students (master's thesis). available from proquest dissertations and theses database. (umi no. mr27029) artzt, a.f. & armour-thomas, e. (1992). development of a cognitive-metacognitive framework for protocol analysis of mathematical problem solving in small groups. cognition and instruction, 9 (2), 137-175. cai, j. (2003). what research tells us about teaching mathematics through problem solving? in f. k. lester (ed.), teaching mathematics through problem solving: prekindergarten – grade 6 (pp. 241– 253). reston, va: nctm. cmajdalka, s. l. (1999). problem solving difficulties in mathematics: examining five process skills in multidigit addition for hispanic limited english proficient students (doctoral dissertation). available from proquest dissertations and theses database. (umi no. 9957407) diezmann, c. m., watters, j. j., & english, l.d. (2001) implementing mathematical investigations with young children. in proceedings 24th annual conference of the mathematics education research group of australasia, (p 170-177), sydney. fisher, r. (1990). teaching children to think. stanley, thornes: cheltenham. garnett, k. f. (1990). developing heuristics in the mathematics problem solving processes of sixth-grade children: a nonconstructivist teaching experiment (master's thesis). available from proquest dissertations and theses database. (umi no. 9115878 hiebert, j., & wearne, d. (2003). developing understanding through problem solving. in h. l, schoen & r. i. charles (eds.), teaching mathematics through problem solving: grades 6-12 (pp, 3-13). reston, va: national council of teachers of mathematics. howland, m. (2001). sixth-grade students’ use of schema knowledge in word problem solving (master's thesis). available from proquest dissertations and theses database. (umi no. 1403973) keller, j. j. (1990). strategy games: developing positive attitudes and perseverance toward problem solving with fourth grades. retrieved from eric database. (ed323013). kilpatric, j., swafford, j. & findell, b. (eds.) (2001). adding it up: helping children learns mathematic. washington, dc: national academy press. lee, k., s. (1982). fourth graders’ heuristic problem-solving behavior. journal for research in mathematics education, 13 (2), 110-123. milli eğitim bakanlığı (ministry of education). (2006). i̇lköğretim okulu matematik dersi öğretim program [elementary mathematics curriculum]. ankara: meb yayınları mubark, m., & zaman, a. (2012). mathematical problem solving of gifted students in jordan: an exploratory study based on polya´s four-step model. journal of institutional research south east asia, 10 (1), 25-37. mullis, i.v.s., martin, m.o., gonzalez, e.j., gregory, k.d., garden, r.a., o’connor, k.m., chrostowski, s.j., & smith, t.a. (2000). timss 1999 international mathematics report: findings from iea’s repeat of the third international mathematics and science study at the eighth grade. chestnut hill, ma: boston college. nancarrow, m. (2004). exploration of metacognition and non-routine problem based mathematics instruction on undergraduate student problem solving success (doctoral dissertation). available from proquest dissertations and theses database. (umi no. 3137375) the effect on learning environments based on problem solving / karatas & baki 266 national council of teachers of mathematics. (2000). principles and standards for school mathematics. va: reston. national council of teachers of mathematics. (1989). curriculum and evaluation standards for school mathematics, reston: va national research council. (1989). everybody counts: a report to the nation on the future of mathematics education, washington, dc: national academy press. ontario ministry of education and training. (1997). the ontario curriculum grades 1-8: mathematics, toronto: queen’s printer for ontario. pervenn, k. (2010). effect of the problem-solving approach on academic achievement of students in mathematics at the secondary level. contemporary issues in education research, 3 (3), 9-13. polya, g. (1945). how to solve it: a new aspect of mathematical method. princeton university press: princeton, new jersey. pourdavood, r. r. (2003). problem solving dynamics-students nonroutine problem solving engagement: a case study of four ninthgrade mathematics students (doctoral dissertation). available from proquest dissertations and theses database. (umi no. 3107286) pugalee, d. k. (2001). writing, mathematics and metacognition: looking for connections through students’ work in mathematical problem solving. school science and mathematics, 101 (8), 236-245 reys, r., reys, b., lapan, r., holliday, g., & wasman, d. (2003). assessing the impact of standards-based middle grades mathematics curriculum materials on the student achievement. journal for research in mathematics education, 34(1), 74-95. rose, t. d. (1991). strategies and skills used by middle school students during the solving of non-routine mathematics problem (master's thesis). available from proquest dissertations and theses database. (umi no. 9133768) schoenfeld, a.h. (1989). explorations of students' mathematical beliefs and behavior. journal for research in mathematics education, 20, 338-355. schoenfeld, a.h. (1992). learning to think mathematically: problem solving, metacognition, and sense making in mathematics. in d.a. grouws (ed.), handbook of research on mathematics teaching (pp.334-370). new york: macmillan publishing. seaman, c. r. (1995). effects of understanding and heuristics on problem solving in mathematics (doctoral dissertation). available from proquest dissertations and theses database. (umi no. nn08476) stacey, k. (1992). mathematical problem solving in groups: are two heads better than one? journal of mathematical behavior, 11, 261-275. url (2006) http://www.nwrel.org/msec/mpm/scoregrid.php 10/11/2006 van de walle, j. a. (2001). elementary and middle school mathematics: teaching developmentally (4th ed.), new york: addison wesley longman. washington, dc: national academy press. verschaffel, l., de corte, e., lasure, s., vaerenbergh, g.v., bogaerts, h. & ratinckx, e. (1999). learning to solve mathematical application problems: a design experiment with fifth graders. mathematical thinking and learning, 1 (3), 195-229. weidemann, w. (1995). problem solving math class: word problems were never like this. middle school journal, 27 (1), 11-17. the effect on learning environments based on problem solving / karatas & baki 267 yazgan, y. & bintaş, j. (2005). fourth and fifth grade students’ levels of using problem solving strategies: a teaching experiment. hacettepe university journal of education, 28, 210-218. international electronic journal of elementary education, june 2016, 8(4), 643-658. issn:1307-9298 copyright © iejee www.iejee.com prospective middle school mathematics teachers’ knowledge of linear graphs in context of problem-posing tuğrul kara a recep tayyip erdogan university, turkey received: 17 january 2016 / revised: 14 march 2016 / accepted: 24 march 2016 abstract this study examined prospective middle school mathematics teachers’ problem-posing skills by investigating their ability to associate linear graphs with daily life situations. prospective teachers were given linear graphs and asked to pose problems that could potentially be represented by the graphs. their answers were analyzed in two stages. in the first stage, the problems were evaluated in terms of whether they represented daily life situations or not and in the second stage, the conceptual validity of the responses was examined. prospective teachers were found to experience difficulties in selecting stories that were appropriate for the structures of the linear graphs and in accurately conveying the data in the graphs through their stories. of the five types of errors identified in the problems posed, the failure to express linearity was the most common. in addition, statistical analyses showed that success in problem-posing declined as the complexity of the data in the graphs increased. keywords: problem-posing, linear graphs, prospective middle school mathematics teachers. introduction content knowledge is one type of knowledge that teachers of mathematics need to possess to ensure student achievement (aslan-tutak & adams, 2015; ball, thames & phelps, 2008; goos, 2013; mewborn, 2001). this is because mathematics content knowledge is a crucial factor influencing the quality of mathematics teaching (ball, lubienski & mewborn, 2001). according to national council of teachers of mathematics [nctm] (2000), teachers should have an in-depth understanding of the mathematical concepts they teach. similarly, ma (2010) stated that teachers should have a profound understanding of the mathematical concepts they use in their teaching. one of the assessment tools used to examine teachers’ mathematical knowledge and to identify their errors and conceptual misunderstandings is problem-posing (kılıç, 2013; rizvi, 2004; ticha & hospesova, 2009). stoyanova (1998) emphasizes the common agreement among researchers that the problems posed by students provide important clues about their mathematical skills. the current study therefore examined prospective middle school mathematics teachers’ mathematics knowledge, using linear graph problem-posing activities that emphasize the skill for translating between representations.  corresponding author: tuğrul kar, recep tayyip erdoğan university, department of elementary mathematics education, faculty of education, 53200, çayeli-rize, turkey. phone: +90 464 532 84 54. e-mail: karrtugrul@gmail.com http://www.iejee.com/ international electronic journal of elementary education vol.8, issue 4, 643-658, june 2016 644 students’ ability to understand and use representations is influenced by their teachers’ knowledge of representation (hjalmarson, 2007; stylianou, 2010). teachers must possess fluent knowledge about representation types and the transition among these types in order to establish a conceptual learning environment (ball, hill & bass, 2005; mcallister & beaver, 2012). problem-posing is an important assessment tool for determining the ability to transition among representation types. friedlander and tabach (2001) argue that problemposing is a frequently used tool when translating from different types of representation to daily life situations. walkington, sherman and howell (2014) found that personalized problems related to students’ out-of-school interests are more effective at improving achievement with regard to linear functions. moreover, these researchers argue that personalization can be accomplished through simple mathematics story problems and that problem-posing can serve as an important tool in this context. in addition, many studies confirm that problem-posing can serve as a means of associating mathematical concepts with daily life situations and can thus contribute to mathematics learning (abu-elwan, 2002; dickerson, 1999; english, 1998). therefore, the investigation of content knowledge through problem-posing should provide us with an accurate assessment of whether prospective teachers have the skills that their students are expected to acquire. a fundamental mathematical domain in middle school mathematics involves the concept of functions, particularly functional relations of the form y=mx+b (brenner et al., 1997; ministry of national education (mone), 2013; nctm, 2000). according to wilkie (2014), many real-world applications are modeled as functions and significant emphasis is placed on functional thinking in mathematics courses during the later years of schooling. in addition, functional relationships play a key role in building algebraic thinking. the lines of the graphs used for problem-posing in this study have the form y=ax+b. therefore, problems involving such graphs may provide significant evidence for understanding how prospective teachers perceive visually presented functional relations. theoretical framework problem-posing and the classification of problem posing activities problem-posing, also referred to as problem generation or problem finding, is defined as the process of generating new problems or reformulating existing ones (akay, 2006; leung, 1993). in his classification of problem types, pehkonen (1995) includes problem-posing in the category of open problems. different theoretical frameworks are offered in the literature for classifying problem-posing activities, each using different criteria (e.g., christou, mousoulides, pittalis, pantazi-pitta & sriraman, 2005; contreras, 2007; silver, 1995; stoyanova & ellerton, 1996). silver (1995), taking into account the relationship between problem-posing and problem-solving, argues that problem-posing can happen prior to problem-solving, during problem-solving, or after problem-solving. christou et al. (2005) formulated a classification according to the relationship of the quantitative information of the problem-posing tasks with thinking processes. this classification consists of the following categories: editing quantitative information, selecting quantitative information, comprehending and organizing quantitative information and translating quantitative information. stoyanova and ellerton (1996), on the other hand, classify problem-posing activities as those taking place in free situations, structured situations and semi-structured situations. in free problem-posing, students are asked to generate problems on the basis of a given natural situation, without any limitations (e.g., generating a shopping problem). in semistructured problem-posing, students are given open-ended situations and are asked to pose problems on the basis of these situations (e.g., problem-posing related to daily life situations to represent the data in a linear graph). the researchers (abu elwan, 1999; christou et al., prospective middle school mathematics teachers’ knowledge of linear graphs / kar 645 2005; stoyanova, 1998) indicated that problem-posing activities related to graphs, openended stories and a picture or diagram were included in the semi-structured problemposing activities. in the current study, prospective teachers were expected to pose problems appropriate to linear graphics related to daily life situations. therefore, these types of problem-posing activities took place in this category. in structured problem-posing, students are asked to generate problems appropriate for a specific solution strategy. this classification is widely used by researchers. representations and transitions between representations representations and transitions between representations play an important role in the teaching of mathematical concepts. ainsworth (2006) draws attention to their importance by stating that two representations are better than one. gagatsis and shiakalli (2004) argue that one representation renders only some aspects of the concept visible, whereas multiple representations complement one another in elucidating a concept. in addition, the ability to translate between different representations has been found to be a strong indicator of conceptual understanding (lesh, post & behr, 1987; panasuk & beyranevand, 2010; stylianou, 2010; you & quinn, 2010). according to harries and suggate (2006), the use of multiple representations provides different views of mathematical concepts, while transitions between representations improve understanding. the importance of using representations can be seen in mathematics curricula. bal (2014) argues that representations are part of the skill set emphasized in mathematics curricula, which includes communication, association and problem-solving, while nctm (2000) emphasizes that mathematical knowledge can be developed by the generation, comparison and utilization of representations. in the mathematics curricula of turkey, it is advised that concepts and rules be taught by using different types of representations that are associated with one another. representations are classified into internal and external representations (goldin & shteingold, 2001; harries & suggate, 2006; lesh, et al., 1987). internal representations, as creations of the human mind, are employed to assign meaning to mathematical concepts and operations (goldin & shteingold, 2001; hjalmarson, 2007). external representations, on the other hand, are used to represent objects and things outside of the human mind (goldin & shteingold, 2001; lesh et al., 1987). mathematical equations, algebraic expressions, graphs and geometric shapes are examples of external representations. external representations are further subdivided into transformations and translations (lesh et al., 1987). transformations take place within a single representation, whereas translations take place between two or more representations (lawler, 2000). for example, if a linear function of the form ax+by=c is represented in different ways, this is called transformation. if a graph is turned into an algebraic representation or an algebraic representation is used to model a real life situation, this is called translation (you, 2006). hence, a translation will always involve two modes of representation: a given source representation and a specified target representation (adu-gyamfi, bosse & stiff, 2012). in this study, the ability to translate between representations was limited to translations between linear graphs and daily life situations. these activities correspond to the translation subgroup of external representations. this limitation was enforced for a number of reasons. first, the visual representation of a functional relationship expresses information much more clearly than an algebraic expression. knuth (2000) makes the point that graphical representation displays an infinite number of points and presents information very clearly, whereas information is shown indirectly in algebraic representations. the current study aimed to examine prospective teachers’ conceptual knowledge in situations where they are faced with a clear presentation of information. second, it was observed that most studies examining the ability to translate between international electronic journal of elementary education vol.8, issue 4, 643-658, june 2016 646 representations through problem-posing have focused on problem-posing based on symbolic representations (e.g., işık & kar, 2012; kar, 2015; macallister & beaver, 2012; toluk-uçar, 2009). only a limited number of studies on problem-posing using graphs have included a linear equation in the form of y=ax+b (e.g., cai et al., 2013). to the best of the author’s knowledge, no studies have examined problem-posing skills by using graphs that emphasize a functional relationship and in which two lines intersect with one another. third, no studies focusing on the errors in problems posed for linear graphs have been found. this study was therefore limited in such a way that it will help to fill these gaps in the literature. studies on problem-posing involving linear graphs many studies involving translations between symbolic, tabular and graphical representations have been carried out (e.g., adu-gyamfi et al., 2012; cunningham, 2005; friedlander & tabach, 2001). in a study conducted with 43 students attending an algebra course, adu-gyamfi et al. (2012) classified the errors made when translating between symbolic, tabular and graphical representations into three groups: implementation errors, interpretation errors and preservation errors. implementation errors arise from computational mistakes (for example, calculating x as −6for 2x+3=y and y=15). in interpretation errors, mathematical concepts are misinterpreted (for example, misinterpreting the slope when formulating the equation representing data in a table). finally, preservation errors arise when some of the characteristics of the representation are translated correctly and others are not (for example, when drawing a graph for the equation 3y-6x=9, the steps required for drawing the graph are followed correctly, but the line cuts the x axis at the wrong point). this type of error is usually made towards the end of the translation process. cunningham (2005) examined the types of transfer problems employed by algebra teachers in their teaching and assessment and found that graphic to numeric transfer problems occurred less frequently than any other type of transfer problem. however, the number of studies investigating the relation of linear functions and in particular, linear graphs to daily life situations by problem-posing is limited (cai et al., 2013; huang & kulm, 2012; işık & kar, 2012; walkington & bernacki, 2014; you, 2006; you & quin, 2010). in some research (e.g., you, 2006; you & quin, 2010), the prospective teachers’ skill at translating between the representations has been investigated using quantitative methods. in such research, the translation from algebraic representations into daily life situations is taken into consideration. for example, you (2006) found that most elementary and middle school prospective teachers (97 out of 104) were able to solve story problems symbolically, but only 18 were able to generate story problems for the equation y = 6x +2. the author also determined that pre-service teachers had difficulty understanding the relationship expressed in a linear function (for example, kim has six times as many soccer balls as bob and then kim got two more). similarly, huang and kulm (2012) found that preservice teachers were not very successful at creating story problems for given non-linear graphs and that they used visual judgment rather than logical reasoning. according to bosse, adu-gyamfi and cheetham (2011), mathematics teachers believe that student achievement in terms of generating verbal stories on the basis of a table or a graph is very low, and they make infrequent use of such activities in their teaching environments. cai et al. (2013) examined 11th-grade students’ problem-posing and problem-solving skills by using a linear graph of the formy=ax+b and a system of equations expressed in algebraic form. the researchers found that most errors were related to expressing the starting point of the graph and the slope. in addition, they found that only 16.6% of the students were able to pose problems about a graph involving an equation of the form y=ax+b. işık and kar (2012) investigated the errors in the problems posed for equation systems by prospective mathematics teachers and found that most errors fell into the categories pertaining to prospective middle school mathematics teachers’ knowledge of linear graphs / kar 647 incorrect translation of mathematical notations into problem statements, ignorance of the realism of problems, failure to establish a relation between the variables, and a lack of conceptual information about equations. walkington and bernacki (2014) examined the difficulties students in grades 6-10 faced when posing and solving problems involving linear equations. they found that students primarily had difficulties using precise language and conceptualizing a functional relationship between unknown quantities. instead of posing problems defining a general linear relationship, students posed problems supplying a specific value for one quantity and asking the solver to calculate the other (e.g., cost=1.25×songs− 15). background information: education system in turkey and middle school mathematics teaching the education system in turkey consists of five hierarchical levels: kindergarten, primary school (grades 1-4), middle school (grades 5-8), high school (grades 9-12) and university. there is a standard curriculum for each of these different levels of education. the middle school mathematics curriculum consists of five learning domains: numbers and operations, algebra, geometry and measurement, data analysis and probability. in each of these learning domains, teachers are advised to use problem-posing and problem-solving together. the curriculum adheres to the belief proffered by gonzales (1998) that problem-posing is the fifth step of problem-solving. generating new problems that resemble problems already solved and posing realistic problems involving given situations are also emphasized (mone, 2013). middle school mathematics teachers in turkey graduate from four-year bachelor’s degree programs offered by the education faculties of universities. entry into these programs is competitive and determined by a national university entrance exam, which is designed to measure 12 years of learning. students are placed in middle school mathematics teaching programs according to their level of achievement on the entrance exam. teachers who graduate from the four-year program teach mathematics to students in grades 5-8 in middle school (ages 12-15). the curricula of middle school mathematics teaching programs consist of courses on general education, pedagogy and mathematical content knowledge. during their junior year at university, students are also offered a course on the methods of mathematics teaching. this course introduces the concepts that are to be taught in middle school classes and provides applied training about the activities to be used when teaching these concepts. in their senior year, prospective teachers also participate in actual teaching at schools and are able to observe and perform in-class teaching activities. method participants prospective teachers in the fourth year of a middle school mathematics teaching program at a university in the eastern part of turkey were informed about the aim of the present study. at the end of this information stage, 93 prospective teachers agreed to participate in the study on a voluntary basis. each participant was assigned an identification code by the researcher (pt1, pt2, …pt93). prospective teachers who participated in the study had experience with linear graphs from their past schooling, particularly middle and high school. all participants in the study had also received instruction on how to use these graphs in teaching during their third-year course on the methods of mathematics teaching. in this course, they performed problem-posing activities involving linear graphs and discussed the errors they made. in addition, as fourth-year students, they had observed lessons and analyzed the work of students in middle schools. data collection and analysis international electronic journal of elementary education vol.8, issue 4, 643-658, june 2016 648 this study used the problem-posing test (ppt) to collect data (see table 1). the ppt was administered to prospective teachers during their class hours and they were given as long as they needed to complete the test. the prospective teachers were asked to pose real life problems that could be represented by the given graphs. all the lines in the graphs were located in the first quadrant of the coordinate plane, but the graphs differed in certain respects. in the first item, there was a single line that started from the positive y axis. in the second and third items, the graphs contained two intersecting lines, but their starting points were different. both of the lines in the second item started from the positive y axis, whereas in the third item, one of the lines started from the (0.0) point. more graphs could have been included, but the number of items was kept low to allow for in-depth analyses. graphs included in the ppt were selected from among those most commonly encountered in the curricula and in textbooks. table 1. ppt items and their characteristics items characteristics graph starts from the positive y axis. lines start from the positive y axis. the two lines intersect. one of the lines starts from the positive y axis and the other starts from point (0, 0). the two lines intersect. story problems involving daily life situations posed on the basis of given graphs should express the starting point of the graph, the functional relationship between x and y and linearity. thus, cai et al. (2013) argue that these problem-posing activities help to assess conceptual understanding, more so than procedural knowledge. prior to administering ppt in this study, it was subjected to 20 prospective teachers as a pilot study. it was applied to prospective teachers and then they were interviewed. in the pilot study, the numeric values applied in the graphs were included in the ppt. in the pilot study carried out during development of the ppt, prospective teachers stated that the numeric data on the graphs made it more difficult for them to devise stories. in addition, prospective teachers indicated that they could pose more appropriate problems if they determined the numbers prospective middle school mathematics teachers’ knowledge of linear graphs / kar 649 themselves. since this study is concerned with participants’ conceptual understanding, what matters is not the type of story but the concepts used to describe the situation. there were no numbers on the graphs. in their stories, participants were free to choose any numbers as long as they were consistent with the structure of the graph. problems posed on the basis of the graphs given in the ppt were analyzed using a threepoint scale: failing (0 points), weak (1 point) and good (2 points). this scale was determined following a careful review of the literature on graphs, equations and problemposing (e.g., cai et al., 2013; işık, kar, i̇pek & işık, 2012; kar, 2015; luo, 2009; you, 2006). the failing category contained responses that failed to associate the graph with a daily life situation, or failed to provide an answer at all. responses in the weak category did associate the graph with a daily life situation, but the contextual structure of the problem being posed did not correspond to the structure of the graph. at this stage, the participants’ responses were subjected to content analysis and error categories were created on the basis of previous work in the field (see section 2.3; for an explanation of error categories and sample responses, see section 5.1). if the elements and constructs in the source representation were successfully articulated in the target representation, the translation was considered to have been successful (adu-gyamfi et al., 2012). thus, the good quality category contained conceptually valid problems that associated the graphs with daily life situations, the solution to which involved drawing the given graphs. repeated-measures anova was used to test whether prospective teachers’ levels of achievement in posing problems varied between the different graphs. the bonferroni post hoc test was conducted to identify the items that were responsible for the differences observed. two researchers, who agreed in their evaluations 92% to 100% of the time, analyzed the problems posed by the participants. scores were then compared and a consensus was reached regarding the assessment of problems previously scored differently. results findings related to errors on ppt items responses placed in the failing category failed to pose problems involving daily life situations that represented the graph. pt2, for example, posed the following problem: “draw a graph for the 𝑦 = 𝑓(𝑥) = 𝑥 + 1 function.” pt2 simply created a function that fit the structure of the graph and directly asked that the graph be drawn. responses in the weak category involved a daily life situation, but the contextual structure of the problem did not correspond to the structure of the graph. five types of errors (e1–e5) were identified in the responses placed in this category: e1, failure to include linearity; e2, failure to express the starting point(s) of the line(s); e3, incompatible story; e4, failure to include a question root; e5, logical error. sample responses for these error types are provided in table 2. responses with the e1error failed to construct stories that express linearity. in the first problem in table 2, the participant is asked about the amount of increase in the height of the water in a bucket being filled with a tap that pours water at a constant rate relative to time. due to the shape of the bucket, however, the time-height graph that results will not be the same as the graph given in the first item of the ppt. given that the bucket narrows towards its opening, the increase in the height of the water will be faster over time. in other words, the graph would be parabolic instead of linear. thus, this problem contains an e1 error. responses with the e2 error failed to express the starting point(s) of the line(s) in their daily life stories. the second problem in table 2, which was prepared in response to the first item on the ppt, involved a year-height graph, but failed to express the initial height of the tree. additionally, the expression “grows two meters each year” is not international electronic journal of elementary education vol.8, issue 4, 643-658, june 2016 650 sufficient to express linearity, because it was not made clear whether the tree had a constant rate of growth over the span of the year. thus, this problem also represents an e1 error. table 2. sample responses classified as weak problems weak problems type of error 1. the bucket in the figure is partially filled with water. this bucket is being filled with water at a constant rate. draw a graph that shows the height of the water in the bucket over time. (pt29) e1 2. there is a tree that grows two meters each year. how tall will this tree be at the end of four years? (pt50) e1 and e2 3. ahmet and ali are saving their money. ali has 10 liras. ahmet initially has no money. ali saves 2.5 liras a day, whereas ahmet saves 5 liras a day. how many days will it take for ahmet to have more money than ali? (pt49) e3 4. vehicle a starts accelerating from the fifth second onwards and increases its speed. (pt61) e1 and e4 5. (distance-time graphics) there are two vehicles, a and b. these vehicles are located 10 and 15 meters away from the starting point. vehicle a has a velocity of 5m/s and vehicle b has a velocity of 7 m/s. when will these vehicles meet? (pt22) e5 responses with the e3 error failed to express continuity, as required by the graphs. the third problem in table 2, which was prepared in response to the third item of the ppt, included this mistake. this problem related a story involving an amount of money saved daily. since days are represented in natural numbers, the graph resulting from this problem will not be the same as the given graph (that is to say, the graph showing ahmet’s total savings, for example, will consist of pairs in the form of (0, 0), (1, 2.5), (2, 5), …). responses with the e4 error failed to include a question root. the fourth problem in table 2, prepared in response to the first item of the ppt concerning the acceleration of a vehicle, did not include a question root. moreover, it was not made clear how the acceleration proceeded. thus, it was not certain that the resulting graph would be a linear one and the problem therefore also contains an e1 error. responses with the e5 error contained stories that were logically inconsistent with the graphs given. the fifth problem in table 2, prepared in response to the second item of the ppt, involved a road-time graph. the story stated that prospective middle school mathematics teachers’ knowledge of linear graphs / kar 651 there was a five-meter gap between the two vehicles. since vehicle a moves at a higher speed than vehicle b, it is clear that the gap between the two vehicles will become larger over time and the resulting lines will not intersect. thus, this problem contains an e5 error. prospective teachers’ achievement in posing problems by graph type table 3 reports the distribution of the failing, weak and good problems posed by prospective teachers for each of the three linear graphs on the ppt. table 3. distribution of scores for problems posed by prospective teachers failing weak good mean score total item 1 4 (4.3) 56 (60.2) 33 (35.5) 1.31 93 (100) item 2 10 (10.8) 64 (68.8) 19 (20.4) 1.1 93 (100) item 3 12 (12.9) 60 (64.5) 21 (22.6) 1.1 93 (100) * data presented in the form of frequencies (percentages). table 3 shows that less than 36% of the problems posed for each of the items included in the ppt were categorized as good. for the second and third graphs on the ppt, achievement levels were even lower. these findings indicate that the prospective teachers had low levels of success in posing conceptually valid problems. when mean scores were compared, it could be seen that participants were more successful for the first item compared with the second and third items. thus, the inclusion of two lines instead of one resulted in lower levels of achievement. results of the repeated-measures anova showed that the prospective teachers’ levels of achievement varied significantly by graph type [f(5.297, 173.653)=3.20, p=.007]. results of the bonferroni post hoc test showed that there were significant differences between the scores for the first and second items (p=.008<.05) and between the scores for the first and third items (p=.039<.05). the difference between the scores for the second and the third items, on the other hand, was not statistically significant (p=1.00>.05). these findings indicate that prospective teachers had greater difficulty posing problems for the second and third graphs than for the first graph. these findings also indicate that the variation in the starting points of the graphs in the second and third items (containing two lines each) did not have a statistically significant impact on problem-posing achievement. more than 60% of the problems posed were categorized as weak for each of the items included in the ppt. these findings indicate that the prospective teachers were successful in associating graphs with daily life situations, but had difficulties reproducing the conceptual structure of the graphs in their stories. table 4 reports the distribution of the five types of errors identified in weak responses by graph type. table 4. distribution of error types identified in weak problems weak responses e1 e2 e3 e4 e5 total item 1 (56 responses) 52(68.4) 13(17.1) 10(13.2) 1(1.3) 76 (100) item 2 (64 responses) 53(68.8) 8(10.4) 3(3.9) 4(5.2) 9(11.7) 77 (100) item 3 (60 responses) 44(55.7) 14(17.7) 9(11.4) 5(6.3) 7(8.9) 79 (100) total (180 responses) 149(64.2) 35(15.1) 22(9.5) 10(4.3) 16(6.9) 232 (100) * data presented in the form of frequencies (percentages). table 4 shows that the 180 responses in the weak category contained 232 errors. on average, there were 1.29 errors per response. the table also shows that e1 and e2 errors international electronic journal of elementary education vol.8, issue 4, 643-658, june 2016 652 were the most common, and that e4 was the least common. these findings therefore indicate that failure to express linearity and failure to express the starting point(s) of the line(s) were the most common errors made when posing problems involving linear graphs. the findings also show that the failure to include a question root in the problem was the least common error. discussion and conclusion many studies and teaching documents support the inclusion of multiple assessment tools in the process of learning (mone, 2013; nctm, 2000; van de walle, karp & bay-williams, 2009). the fact that individual assessment tools suffer from different problems of validity and reliability renders the use of a variety of combined assessment tools essential. the use of different assessment tools is also necessary to create a broader outlook on mathematical understanding (hiebert & carpenter, 1992). in this context, problem-posing has been increasingly emphasized in recent years as an alternative assessment tool (lavy & shriki, 2007; silver & cai, 2005; ticha & hospesova, 2009). problem-posing gives teachers a good idea of their students’ skills, attitudes and conceptual learning (silver & cai, 2005). problem-posing can also be used to assess both teachers' and prospective teachers’ grasp of mathematics subjects, as well as that of students (işık & kar, 2012; kılıç, 2013; mcallister & beaver, 2012). this research utilized problem-posing for assessment purposes in order to investigate prospective teachers’ mathematical understanding of linear graphs. the low percentage of responses categorized as failing for each graph on the ppt indicates that most prospective teachers at least make an attempt to associate graphs with daily life situations. previous studies have shown that fewer students than prospective teachers make this attempt. for example, cai et al. (2013) found that only 63.3% of 11th graders made an attempt to pose a problem for the graph task, while işık et al. (2012) found that most prospective elementary teachers (varying between 87.5% and 100% for different items) made an attempt to associate graphs with daily life situations. considered together, these results indicate that the level of knowledge and experience possessed by the problem-poser has a significant impact on success in problem-posing. prospective teachers generally had difficulties using daily life stories to express the data in the graphs (more than 60% of responses for each item were categorized as weak). the most common error made by the participants in this category was the failure to express linearity. this type of error arises when there is a failure to include statements expressing the linear relationship between x and y. this is also the most common type of error made in research carried out with both students and prospective teachers (e.g., işık et al., 2012; walkington & bernacki, 2014; you, 2006). for example, you (2006) found that prospective teachers tended to construct stories that involved a single pair of values (x, y) instead of defining a general linear relationship. in the present study, however, most participants were able to express a general relationship between x and y (e.g., the car accelerates over time), but failed to express that the slope was constant (e.g., there was no statement expressing that the change in the speed of the car was constant). in addition, it was found that some participants constructed stories involving discrete quantities (see the third sample problem in table 2). this type of error indicates that the participants were exclusively concerned with posing problems reproducing the overall shape of the graph. other studies (berg & philips, 1994; capraro, kulm & capraro, 2005) have also found that students make the error of drawing discrete graphs for continuous data or continuous graphs for discrete data. the translation of mathematical expressions into verbal statements requires strong linguistic skills. thus, a lack of sufficient linguistic skills may be another reason for prospective teachers’ low levels of achievement. many studies (capraro & joffrion, 2006; prospective middle school mathematics teachers’ knowledge of linear graphs / kar 653 macgregor & stacey, 1993) show that participants have difficulty explaining the relationship between mathematical notations and verbal expressions in story problems. clement (1982) found that students sometimes assign meanings to the variables in an algebraic expression that are inconsistent with verbal expression (some students, for example, wrote the equation 6y=x to represent a verbal expression that in fact denoted 6x=y). similar issues were encountered in this study. data swapped places in the translation from graph to verbal expression, leading to failure in terms of representing the graph accurately. this may have resulted from a failure to understand the graph, or it might be an indicator of linguistic problems. lack of knowledge and experience regarding problem-posing result in low levels of success (kar, 2015; luo, 2009). prospective teachers’ lack of experience with problem-posing involving linear graphs was likely another reason for the lack of success observed in this study. participants in the study did have some experience with problem-posing activities from various courses, but their levels of achievement were nevertheless low. this indicates that the development of problem-posing skills, similar to the development of problemsolving skills, takes time. another finding of the study was that participants were more successful in posing problems for the first item of the ppt and had similarly low levels of achievement for items two and three. the first item of the ppt included a single line, whereas items two and three included two intersecting lines and thus, contained a larger amount of mathematical data compared to the first item. from these observations, we can conclude that as the number of associated variables on a graph increases, success in posing appropriate problems for the graph declines. other studies on translations between representations have reached similar conclusions. for example, adu-gyamfi et al. (2012) found that translations involving representations with larger amounts of data resulted in more errors being made. a teacher’s knowledge has an important effect on student achievement (ball et al., 2008; hill, rowan & ball, 2005; kulm, 2008). in other words, if teachers do not have the necessary knowledge and experience regarding the concepts they are supposed to teach, they will have difficulty teaching these concepts. the findings of this study show that prospective teachers have low levels of success in posing problems involving linear graphs and that their efforts were hampered by five types of errors. the distribution of the errors indicates the most important deficiency as a lack of conceptual knowledge, which will also likely be reflected in the problem-posing activities these teachers will organize in future. the findings of this study do not provide conclusive evidence about whether a lack of conceptual knowledge, linguistic problems, or a lack of experience with problem-posing led to the different types of errors. thus, it is recommended that future studies supplement these findings with interviews, so that the causes of the errors can be probed further. levels of success did not differ significantly between some of the items on the ppt. similar additional studies conducted with linear graphs that have different structures will make it possible to enhance understanding of this field. the sample size of the study does not allow for generalization of the conclusions to all prospective teachers in turkey, a country with low levels of achievement in international comparisons. similar studies with larger sample sizes should be conducted and results should be compared with results obtained in countries with higher levels of achievement in international comparisons. the literature fails to provide a sufficient explanation for the difficulties faced by students in translating between representations (adu-gyamfi et al., 2012). this study aimed to contribute to the literature by examining the errors made by prospective teachers when translating linear graphs into daily life situations. errors can be viewed as opportunities for improving learning; moreover, the errors determined in this study can provide guidance to teachers and researchers when designing learning environments involving problem-posing. international electronic journal of elementary education vol.8, issue 4, 643-658, june 2016 654 furthermore, problem-posing has recently received increased attention, but studies indicate that problem-posing is still only an emerging topic in mathematics education research (kitchings, 2014). according to cai et al. (2013), practitioners are interested in making problem-posing a more prominent feature of classroom instruction. the findings of this study contribute to this effort by providing information on the use of problem-posing for the purpose of translating between representations. • • • references abu-elwan, r. (1999). the development of mathematical problem posing skills for prospective middle school teachers. in a. rogerson (ed.), proceedings of the international conference on mathematical education into the 21st century: social challenges, issues and approaches, (pp. 1-8), cairo egypt. abu-elwan, r. (2002). effectiveness of problem posing strategies on prospective mathematics teachers’ problem solving performance. journal of science and mathematics education, 25(1), 5669. adu-gyamfi, k., stiff, l., & bosse, m. j. (2012). lost in translation: examining translation associated with mathematical representations. school science and mathematics, 112(3), 159–170. ainsworth, s. (2006). deft: a conceptual framework for considering learning with multiple representations. learning and instruction, 16, 183-198. akay, h. (2006). the examination of the effect of mathematics ınstruction with problem posing approach on students’ academic achievement, problem solving ability and creativity (unpublished doctoral dissertation). university of gazi, ankara. aslan-tutak, f., & adams, t. l. (2015). a study of geometry content knowledge of elementary preservice teachers. international electronic journal of elementary education, 7(3), 301-318. bal, a. p. (2014). the examination of representations used by classroom teacher candidates in solving mathematical problems. educational sciences: theory & practice, 14(6), 2349-2365. ball, d. l., hill, h. h., & bass, h. (2005). knowing mathematics for teaching: who knows mathematics well enough to teach third grade, and how can we decide? american educator, 29(1), 14-22. ball, d. l., lubienski, s. t., & mewborn, d. s. (2001). research on teaching mathematics: the unsolved problem of teachers’ mathematical knowledge. handbook of research on teaching, 4, 433-456. ball, d. l., thames, m. h., & phelps, g. (2008). content knowledge for teaching: what makes it special? journal of teacher education, 59(5), 389-407. berg, c. a., & philips, d. g. (1994). an investigation of the relationship between logical thinking and the ability to construct and interpret line graphs. journal of research in science teaching, 31(4), 323-344. bossé, m. j., adu-gyamfi, k., & cheetham, m. (2011). translations among mathematical representations: teacher beliefs and practices. international journal of mathematics teaching and learning. 1-23. retrieved from http://www.cmit.plymouth.ac.uk/journal/default.htm brenner, m. e., mayer, e. r., moseley, b., brar, t., durán, r., reed, b. s., & webb, d. (1997). learning by understanding: the role of multiple representations in learning algebra. american educational research journal, 34(4), 663-689. cai, j., moyer, j. c., wang, n., hwang, s., nie, b., & garber, t. (2013). mathematical problem posing as a measure of curricular effect on students’ learning. educational studies in mathematics, 83, 57-69. prospective middle school mathematics teachers’ knowledge of linear graphs / kar 655 capraro, m. m. & joffrion, h. (2006). algebraic equatıons: can middle-school students meaningfully translate from words to mathematical symbols? reading psychology, 27, 147–164. capraro, m. m., kulm, g., & capraro, r. m. (2005). middle grades: misconceptions in statistical thinking. school science and mathematics journal, 105(4), 1–10. christou, c., mousoulides, n., pittalis, m., pitta-pantazi, d., & sriraman, b. (2005). an empirical taxonomy of problem posing processes. zdm, 37(3), 149 – 158. clement, j. (1982). algebra word problem solutions: thought processes underlying a common misconception. journal for research in mathematics education, 13, 16–30. contreras, j. (2007). unraveling the mystery of the origin of mathematical problems: using a problem-posing framework with prospective mathematics teachers. the mathematics educator, 17(2), 15-23. cunningham, r. f. (2005). algebra teachers’ utilization of problems requiring transfer between algebraic, numeric, and graphic representations. school science and mathematics, 105(2), 73-81. dickerson, v. m. (1999). the impact of problem-posing instruction on the mathematical problemsolving achievement of seventh graders (unpublished doctoral dissertation). university of emory, atlanta. english, d. l. (1998). children’s problem posing within formal and informal contexts. journal for research in mathematics education, 29(1), 83-106. friedlander, a., & tabach, m. (2001). promoting multiple representations in algebra. in a. cuoco (ed.), the roles of representation in school mathematics. 2001 yearbook of the national council of teachers of mathematics (pp.173-184). reston, va: national council of teachers of mathematics. gagatsis, a., & shiakalli, m. (2004). ability to translate from one representation of the concept of function to another and mathematical problem solving. educational psychology, 24, 645–657. goldin, g., & shteingold, n. (2001). systems of representations and the development of mathematical concepts. in a. cuoco (ed.), the roles of representation in school mathematics. 2001 yearbook of the national council of teachers of mathematics (pp. 1-23). reston, va: national council of teachers of mathematics. gonzales, n. a. (1998). a blueprint for problem posing. school science and mathematics, 94(2), 78 85. goos, m. (2013). knowledge for teaching secondary school mathematics: what counts? international journal of mathematical education in science and technology, 44(7), 972-983. harries, t., & suggate, j. (2006). exploring links across representations of numbers with young children. international journal for technology in mathematics education, 13(2), 53-64. hiebert, j., & carpenter, t. p. (1992). learning and teaching with understanding. in d. a. grouws (ed.), handbook of research on mathematics teaching and learning (pp. 65-97). new york: macmillan. hill, h. c., rowan, b., & ball, d. l. (2005). effects of teachers' mathematical knowledge for teaching on student achievement. american educational research journal, 42, 371-406. hjalmarson, m. (2007). mathematical representations. gwenanne m. salkind george mason university edci 857 preparation and professional development of mathematics teachers. retrieved from http://mason.gmu.edu/~gsalkind/portfolio/products/857litreview.pdf huang, r., & kulm, g. (2012). prospective middle grade mathematics teachers’ knowledge of algebra for teaching. the journal of mathematical behavior, 31, 417-430. işık, c., & kar, t. (2012). the analysis of the problems posed by the preservice teachers about equations. australian journal of teacher education, 37(9), 92-113. international electronic journal of elementary education vol.8, issue 4, 643-658, june 2016 656 işık, c., kar, t., i̇pek, a. s., & işık, a. (2012). difficulties encountered by pre-service classroom teachers in constructing stories about line graphs. international online journal of educational sciences, 4(3), 644-658. kar, t. (2015). analysis of problems posed by sixth-grade middle school students for the addition of fractions in terms of semantic structures. international journal of mathematical education in science and technology, 46(3), 1-16. doi: 10.1080/0020739x.2015.1021394 kılıç, ç. (2013). pre service primary teachers’ free problem posing performances in the context of fractions: an example from turkey. the asia pacific education researcher, 22(4), 677-686. kitchings, c. n. (2014). problem posing in middle-grades mathematics classes (unpublished doctoral dissertation). university of georgia, athens, georgia. knuth, e. j. (2000). student understanding of the cartesian connection: an exploratory study. journal of research in mathematics education, 31, 500-508. kulm, g. (2008). teachers’ mathematics knowledge. school science and mathematics, 108, 2-3. lavy, i., & shriki, a. (2007). problem posing as a means for developing mathematical knowledge of prospective teachers. paper presented at the meeting of 31st conference of the international group for the psychology of mathematics education, seoul. lawler, g. m. (2000). the use of written scaffolds to instruct linear functions in a reform algebra course using graphic calculators (unpublished doctoral dissertation). university of new york. lesh, r., post, t., & behr, m. (1987). representations and translations among representations in mathematics learning and problem solving. in c. janvier (ed.), problems of representation in the teaching and learning of mathematics (pp. 33-40). hillsdale, nj: erlbaum associates. leung, s. s. (1993). the relation of mathematical knowledge and creative things to the mathematical problems posing of prospective elementary school teachers on tasks differing in numerical information content (unpublished doctoral dissertation). university of pittsburg. luo, f. (2009). evaluating the effectiveness and insights of pre-service elementary teachers’ abilities to construct word problems for fraction multiplication. journal of mathematics education, 2(1), 83–98. ma, l. (2010). knowing and teaching elementary mathematics: teachers' understanding of fundamental mathematics in china and the united states. newyork, nj: lawrence erlbaum associates. macgregor, m., & stacey, k. (1993). cognitive models underlying students' formulation of simple linear equations. journal for research in mathematics education, 24(3), 217-232. mcallister, c. j., & beaver, c. (2012). identification of error types in preservice teachers' attempts to create fraction story problems for specified operations. school science and mathematics, 112(2), 88-98. mewborn, d. (2001). teachers content knowledge, teacher education, and their effects on the preparation of elementary teachers in the united states. mathematics teacher education and development, 3, 28-36. ministry of national education [mone], (2013). middle school mathematics curriculum: grades 5-8, ankara. national council of teachers of mathematics (nctm). (2000). principles and standards for school mathematics. reston, va: nctm. panasuk, r., & beyranevand, m. (2010). algebra students' ability to recognize multiple representations and achievement. international journal for mathematics teaching and learning, 1-22. retriewed from http://www.cimt.plymouth.ac.uk/journal/panasuk.pdf pehkonen, e. (1995). introduction: use of open-ended problems. international reviews on mathematical education 27(2), 55-57. prospective middle school mathematics teachers’ knowledge of linear graphs / kar 657 rizvi, n. f. (2004). prospective teachers’ ability to pose word problems. international journal for mathematics teaching and learning, 166, 1-22. silver, e. a., & cai, j. (2005). assessing students’ mathematical problem posing. teaching children mathematics, 12(3), 129-135. silver, e. a. (1995). the nature and use of open problems in mathematics education: mathematical and pedagogical perspectives. international reviews on mathematical education, 27, 67–72. stoyanova, e. (1998). problem posing in mathematics classrooms. in a. mcintosh, & n. ellerton (eds.), research in mathematics education: a contemporary perspective (pp.164-185). perth: mastec publication. stoyanova, e., & ellerton, n. f. (1996). a framework for research into students’ problem posing. in p. clarkson (ed.), technology in mathematics education (pp.518–525). melbourne: mathematics education research group of australasia. stylianou, d. a. (2010). teachers’ conceptions of representation in middle school mathematics. journal of mathematics teacher education, 13, 325–343. doi 10.1007/s10857-010-9143-y ticha, m., & hospesova, a. (2009). problem posing and development of pedagogical content knowledge in preservice teacher training. paper presented at the meeting of cerme 6, lyon. toluk-uçar, z. (2009). developing pre-service teachers understanding of fractions through problem posing. teaching and teacher education, 25(1), 166-175. van de walle, j. a., karp, k. s., & bay-williams, j. m. (2009). elementary and middle school mathematics: teaching develop-mentally (7th ed.). boston: pearson education. walkington, c., & bernacki, m. (2014). students authoring personalized “algebra stories”: problemposing in the context of out-of-school interests. paper presented at the annual meeting of the american educational research association. walkington, c., sherman, m., & howell, e. (2014). personalized learning in algebra. mathematics teacher, 108(4), 272-279. wilkie, k. j. (2014). upper primary school teachers’ mathematical knowledge for teaching functional thinking in algebra. journal of mathematics teacher education, 17, 397–428. doi: 10.1007/s10857-013-9251-6 you, z. (2006). preservice teachers’ knowledge of linear functions within multiple representation modes (unpublished doctoral dissertation). texas a&m university. you, z., & quinn, r. j. (2010). prospective elementary and middle school teachers’ knowledge of linear functions: a quantitative approach. journal of mathematics education, 3(1), 66-76. international electronic journal of elementary education vol.8, issue 4, 643-658, june 2016 658 www.iejee.com this page is intentionally left blank http://www.iejee.com/ microsoft word iejee_5_4_griffin_wiley_britt_salas international electronic journal of elementary education, 2012, 5(1), 63-78. issn:1307-9298 copyright © iejee www.iejee.com the role of clear thinking in learning science from multiple-document inquiry tasks thomas d. griffin ∗∗∗∗ university of illinois at chicago, chicago, il, usa jennifer wiley university of illinois at chicago, chicago, il, usa m. anne britt northern illinois university, dekalb, il, usa carlos r. salas university of illinois at chicago, chicago, il, usa received: 21 august 2012 / revised: 6 november 2012 / accepted: 6 november 2012 abstract the main goal for the current study was to investigate whether individual differences in domaingeneral thinking dispositions might affect learning from multiple-document inquiry tasks in science. middle school students were given a set of documents and were tasked with understanding how and why recent patterns in global temperature might be different from what has been observed in the past from those documents. understanding was assessed with two measures: an essay task and an inference verification task. domain-general thinking dispositions were assessed with a commitment to logic, evidence, and reasoning (clear) thinking scale. the measures of understanding were uniquely predicted by both reading skills and clear thinking scores, and these effects were not attributable to prior knowledge or interest. the results suggest independent roles for thinking dispositions and reading ability when students read to learn from multiple-document inquiry tasks in science. keywords: thinking dispositions, learning from text, climate change, earth science, multipledocument inquiry tasks. introduction the internet has become a primary means by which people search for information to answer many science-related questions. adults read internet sources to help them understand phenomena in the world around them. they read to learn about the development of new ∗ thomas d. griffin, department of psychology, university of illinois at chicago, 1007 w. harrison st (m/c 285), chicago, il, 60607 usa, phone: (312) 996-8190, e-mail: tgriffin@uic.edu. international electronic journal of elementary education, 2012, vol.5, issue 1, 63 78. 64 technologies. they read to form beliefs on policy issues as well as to inform health-related decision-making. to form an understanding, they need to integrate information that is spread out across numerous documents and sources. hence, the ability to learn about scientific phenomena from multiple documents is a critical skill for life-long learning. yet, instruction in this area is not well represented in science classrooms, and research identifying potential sources of variance in how well students can engage in multiple-document inquiry learning on science topics has been limited. the main goal for the current study was to investigate whether individual differences in either reading skills or domain-general thinking dispositions might affect learning from multiple-document inquiry tasks in science. what processes are needed to learn from multiple-document inquiry tasks? one reason why learning from multiple documents is so complex is because it requires all of the processes necessary for comprehending individual informational texts, plus an additional set of processes that become particularly important when readers are confronted with information from more than one text. according to theories of text comprehension (kintsch, 1998; kintsch & van dijk, 1983), understanding even a single informational text requires the construction of several levels of representation. at the lowest level, a reader creates a surface representation, which generally consists of a fleeting episodic trace capturing the exact words and format of the text. at the next level of processing, the reader attempts to develop the text-based representation. this is essentially a propositional representation of the ideas presented in each clause or sentence. basic word and sentence-level reading processes contribute to the construction of this text-based representation. in addition, to learn from informational text, the reader must attempt to develop yet another level of representation, referred to as the situation model by kintsch. on this level, the reader attempts to connect ideas between the sentences and with prior knowledge to develop a coherent understanding of the content that is being described. when the goal for reading informational science texts is to develop an understanding of how or why a phenomenon occurs, then the situation model can be thought of as a causal chain or mental model of the phenomena being described (kintsch, 1994; trabasso & van den broek, 1985; wiley, griffin & thiede, 2005). yet, in many learning situations, readers are presented with more than a single text from which to obtain information (britt, perfetti, sandak, & rouet, 1999; perfetti, rouet, & britt, 1999). one framework for describing the cognitive processes involved in multiple-document comprehension is the md-trace model (multiple documents task-based relevance assessment and content extraction) proposed by rouet and britt (2011). according to the md-trace model, students begin multiple-document reading by creating an interpretation of the task (called the task model). this task model includes the goals and subgoals for reading (e.g., why is the text being read? what is the question to be answered? what does developing an argument or explanation entail?) and plans to reach those goals (e.g., find evidence or causes). in other words, the task model includes the goals for reading and the basic steps that should be taken to achieve the desired outcome. depending on the reading context, including the reader’s interpretation and the instructions that are given, task models will range from cursory to clearly delineated. although reading goals may affect learning in single text contexts, they become even more critical to consider in multiple-document contexts. learning from multiple documents instead of a single document also requires another level of representation (a documents model) that captures the relation of information across the document set, and information about each document, in addition to the representations of the content of individual texts. the documents model (britt, perfetti, sandak, & rouet, 1999; the role of clear thinking in learning science from multiple-document inquiry tasks / griffin, wiley, britt & salas 65 perfetti, rouet, & britt, 1999) has been proposed to capture these two needs. one part of a documents model, the intertext model, includes information about the sources of the various documents (e.g., who wrote it), and notes relations among the documents (e.g. the presence of corroborating or conflicting statements). the documents model also contains the integrated model, which serves as the representation of the situation or phenomena described across documents. according to the md-trace model, the extent to which readers develop integrated models or intertext models from multiple-document contexts will be partially determined by the goals readers have in their task model. with a single text, comprehension can be driven by a text’s intended purpose, structure, or argument. with multiple texts, the reader must impose selection and organization in order to form a model that integrates the information from different texts (rather than simply constructing distinct models of each text). a reader’s goals guide the process of reading and evaluating the individual texts, selecting relevant information, and reassembling what is selected into a new coherent model. thus, the interpretation of the task and the goals a reader sets for reading are a critical determinant of multiple-document comprehension. what leads to successful learning from multiple-document inquiry tasks in science? a burgeoning area of investigation at the intersection of literatures on subject-matter learning and learning from informational texts has been exploring what conditions facilitate student understanding from multiple-document inquiry activities in science (braten, britt, stromso,& rouet, 2011; cerdan & vidal-abarca, 2008; goldman, braasch, wiley, graesser, & brodowinska, in press; mason, boldrin, & ariasi, 2010; sanchez, wiley & goldman, 2006; wiley, ash, sanchez & jaeger, 2011; wiley, goldman, graesser, sanchez, ash, & hemmerich, 2009). there are numerous factors that are likely to impact learning from multiple documents. these factors can include features of the set of sources that are provided, as well as the nature of the inquiry task that is given. one general approach within this literature has been to provide students with a set of informational texts as reading material, often through the guise of the results of an internet search. as a goal for the processing of the informational texts, students are generally tasked with learning about how or why a phenomenon occurred such as “what caused the eruption of mt. st. helens?” or “how do bacteria resist the effects of antibiotics and which biological mechanisms explain this phenomenon and its transmission to other bacteria?” or “what caused the extinction of dinosaurs?” similar to studies on learning from multiple sources in history (i.e. wiley & voss, 1996; 1999), when students are prompted to use the text sets to generate a causal argument or explanatory model of a phenomenon, it results in better learning from the activity. for example, cerdán and vidal-abarca (2008) found that prompting students to read texts in order to explain how resistance to antibiotics develops resulted in a deeper and more integrated understanding than did asking students very specific questions that could be answered by searching for, finding, memorizing, and reproducing isolated bits of information within the text set. these results are generally consistent with the idea that students’ understanding may benefit from multiple-document activities to the extent that students engage in constructive processing that builds connections across ideas in order to form a coherent, integrated model of the phenomena (britt & rouet, 2012; wiley & voss, 1996; 1999). yet, further research has demonstrated that not all learners take advantage of this opportunity, especially when the reading material requires selective use of information (wiley, ash, sanchez & jaeger, 2011). for example, in wiley et al (2009) which provided students with texts from both reliable and unreliable sources about volcanic eruptions, the ability to evaluate the sources was seen as a gatekeeper to the development of an accurate international electronic journal of elementary education, 2012, vol.5, issue 1, 63 78. 66 mental model of the phenomena. in the first study, better learning was related to the ability to differentiate reliable from unreliable sources. in the second study, the presence of a preinquiry instructional module on source evaluation was manipulated (see also sanchez, wiley, & goldman, 2006). in this module, called “seek”, students were taught to evaluate reliability of each document by considering not only the source of each document, but also whether any evidence was presented that could be related to an explanation of the phenomenon prompted by the inquiry question, and also how information in the document related to other knowledge about the phenomena. participants completed the seek module on an unrelated topic prior to the inquiry task. thus, this instructional manipulation stressed the need to consider the reliability of source information as well as the importance of thinking about explanations, evidence, and integration with both prior knowledge and the information in other documents. the main result of this study was that students given seek instruction demonstrated better learning from the subsequent multiple-document inquiry task on volcanic eruptions. together these two studies demonstrated that multipledocuments learning is predicted by individual differences in evaluating sources and content of the documents, and that learning is improved by instruction that targets skills related to evaluation based on source, evidence, and coherence with other information. additional lines of investigation have further explored what the better learners were doing in the first wiley et al (2009) study by using eyetracking and think-aloud methodologies. wiley, et al. (2011) found that when students were asked to write an argument explaining the causes of volcanic eruptions, the best learners showed more selective reading behaviors. these participants were more likely to skim pages, but would often go back and thoroughly re-read a page if it contained conceptually relevant information. eyetracking data showed that these better learners were also more likely to spend a greater proportion of their time on the specific sentences of a relevant page that (i.e., regions of interest, rois) were most critical for forming an explanation. lastly, while all readers looked at the illustrations that accompanied the texts, the better learners tended to look at the conceptual images more so than decorative images. these findings suggest that the students that were more goaldirected, strategic, and selective in their reading and use of available information learned more and created better causal explanations. these better learners seemed to be more engaged in the process of creating an integrated mental model, as they showed better use of conceptual illustrations, and more integrated reading patterns. importantly, the best learners were not simply spending more overall time reading—but instead they were more selective with their reading efforts. for the best learners, it appears that they responded to an argument-writing task by directing their attention to the most relevant information for the construction of an accurate mental model or explanation. an additional finding in this study is that when students were given an instruction to write a report rather than an argument, they were generally less selective in which information they read and included in their essay. the findings of the think-aloud study (goldman, et al. in press) where readers were asked to simply think aloud about what they were doing while viewing the documents, also suggested that better learners were more selective in what they read and how they utilized the information. better learners made more comments related to their evaluation of source credibility and reliability, especially in relation to why they went to certain pages and not others or why they were leaving a page before they read all of its content. their comments revealed a more strategic approach to reading in which they referenced their inquiry task, their current understanding, and what they still needed to accomplish. consequently, they incompletely read pages that they judged would not further their understanding and finished pages that they judged would further their understanding. in addition, the better the role of clear thinking in learning science from multiple-document inquiry tasks / griffin, wiley, britt & salas 67 learners were more selective in how they used the information. they engaged in more sensemaking, self-explanation and comprehension-monitoring processes while they were reading reliable sites than they did on unreliable websites. these behaviors are usually related to incorporating information into mental models. self-explanation helps in both the construction of models (chi, 2000) and as a source of cues for evaluating the quality and completeness of one’s own mental model (griffin, wiley, & thiede, 2008). thus, these findings suggest that better readers were more selective and integrated the more reliable information into their mental models. interestingly, the think-aloud evidence available from this study does not suggest that the better learners had better a priori knowledge of which sites might be considered more reliable and useful. rather, the think-aloud comments suggested that readers assessed whether their understanding was increasing and what additional information was needed to achieve the inquiry goal. this type of evaluation of the sites in terms of goal-relevant content resulted in learners being more strategic in their reading and spending a greater proportion of their time on the more reliable sites, hence resulting in better learning from the inquiry task. other work on learning from internet-inquiry tasks also suggests that differences in how students approach the evaluation of information quality can affect their learning. following an internet inquiry task on dinosaur extinction, mason, boldin, and ariasi (2010) asked learners how they decided which information they found on the internet was true. they found that students who were more likely to appeal to comparisons of information from multiple sources and to scientific evidence as a basis for evaluation were those who were more likely to learn the scientifically accepted conception of extinction from the activity. this study also administered the conley, pintrich, vekiri and harrison (2004) self-report measure of epistemic beliefs about science. they found that scores on the justification subscale were related to whether students learned the scientifically accepted view of extinction. this subscale assesses beliefs about the nature of science and the importance science places on experiments, replication, and the source of scientific ideas. similarly, other work from braten and stromso (2010) has also shown that some features of epistemological beliefs (specifically about the to-be-learned topic such as climate change) can predict who might learn most effectively from multiple-document inquiry tasks, and that readers who engage in more source evaluation behaviors develop better understanding from multiple documents (braten, stromso & britt, 2009). together, these results suggest that multiple-document inquiry tasks provide the opportunity for readers to engage in more integrative processing and model construction. yet, the extent to which readers are able to take advantage of this opportunity depends on whether readers selectively process and integrate the most reliable, central, and relevant information. several lines of research suggest that individuals differ in how they approach inquiry tasks and whether they engage in evaluation, selective reading and integration. the reviewed research suggests that beliefs about science matter, as do instructions to use the information in order to form causal arguments and instructions to evaluate evidence, relevance, and source reliability. individual differences in general thinking dispositions may be a promising source of influence on the likelihood that a person will engage in successful learning from multiple documents. international electronic journal of elementary education, 2012, vol.5, issue 1, 63 78. 68 delineation of two classes of individual differences: capacities and dispositions. expanding on the work of reasoning theorists such as baron (1985), stanovich and west (1997) delineated two classes of individual differences, capacities and dispositions that may be useful to consider in this context. what and how a person does on any cognitive task is determined by a combination of what they are capable of doing and what they are disposed towards doing. capacity constraints have been a primary focus within cognitive psychology. these range from more basic processing abilities and constraints that are not likely teachable, such as working memory capacity, to more teachable factors including reading skills such as decoding and word knowledge. individual differences in capacity constraints would be expected to play a role in learning from multiple documents just as they would in learning from single passages. although rarely included in studies on learning from multiple documents, reading ability has been shown to be a significant predictor of multipledocument comprehension in at least one study (mason, boldrin, & ariasi, 2010). in contrast, dispositional individual differences are those that relate to a person’s goals, their orientation towards the task, and their willingness towards applying whatever relevant skills and capacities they have to the processes required for effective thinking, learning, and task performance. griffin and ohlsson (2001) showed that people vary in whether they report forming their belief on a topic in terms of considering relevant evidence versus deferring to their affective preferences, and this in turn predicts people’s willingness to revise their belief in light of new evidence. more recently, griffin (2008) reported findings which suggest that people have a general disposition towards whether they consider evidence or affect when forming their beliefs. people reported the extent to which they based their beliefs on either considering evidence or relying upon faith. the beliefs varied across eight different topics that were both religious and non-religious. the topics were largely unrelated in content as indicated by the fact that what a person believed on each topic did not predict what they believed on the other topics. however, the degree to which people relied upon faith versus evidence to arrive at whatever belief they held on a topic was correlated with their reliance on faith versus evidence for all other topics. this pattern of consistency across distinct topics implicates a general thinking disposition relating to intellectual values. a reader’s goals when attempting to learn from multiple documents might be affected by many factors, including their interest in the topic, as well as by general intellectual values. in a multiple-documents inquiry context, effective learning requires engaging in evidencebased reasoning in the service of argument construction. a general disposition of valuing evidence-based thinking in the evaluation of beliefs and claims would seem to orient one toward attempting to construct the kind of coherent argument that such an inquiry task requires. thus, having an evidence-based disposition may play a unique role above and beyond learners’ capacities and skills in determining whether they engage in the extratextual processing required for developing an integrated model of phenomena across texts as required by multiple-document inquiry tasks. in previous studies on domain-general thinking dispositions, stanovich and colleagues have shown that a general disposition towards actively open-minded thinking (aot) predicts cognitive performance on higher-order thinking tasks over and above measures of cognitive capacity, such as the sat, raven’s matrices, and nelson-denny reading comprehension test (e.g., stanovich & west, 1998; for a recent review see stanovich, 2012). this research has employed various versions of a 41-item aot scale comprised of several subscales. some items tap moral authoritarianism and openness to others’ values (e.g., “i believe we should look to our religious authorities for decisions on moral issues.”; “there are a number of people i have come to hate because of the things they stand for.”). however, other items the role of clear thinking in learning science from multiple-document inquiry tasks / griffin, wiley, britt & salas 69 focus upon a general openness to intellectual inquiry, evidence, and belief revision (e.g., “people should always take into consideration evidence that goes against their beliefs.”; “one should disregard evidence that conflicts with your established beliefs.”). the present research is interested in these latter items assessing a disposition that is more directly related to the kind of evidence-based thinking that should impact a multiple-documents inquiry task in science. the aot scale has typically been employed as a composite with a single score that is used to predict performance on higher-order cognitive tasks assessing logical reasoning, rational judgment, normative decision making, and informal reasoning processes such as syllogistic reasoning, probabilistic reasoning, statistical reasoning, covariation detection, and argument evaluation (stanovich & west, 1997; 1998). individuals with high aot scores evaluate objective argument quality more accurately than those with lower aot scores, and their evaluations are less biased by consistency with prior beliefs, even when controlling for cognitive ability (stanovich & west, 1997; west, stanovich, & toplak, 2008). sá, kelley, ho, and stanovich (2005) found that people low in aot were more likely to generate arguments that simply reiterated their personal theory rather than providing supporting evidence. identifying and incorporating relevant information across multiple texts is likely to be impacted by some of the same factors that impact the kind of argument evaluation and construction tasks employed in these studies. although most of the studies on aot have involved adults, one study has found that thinking dispositions can predict performance on several standard reasoning tasks with children (kokis et al., 2002). however, none of the prior studies have used the aot measure to examine the effects of thinking dispositions in a classroom learning task where the goal is to acquire knowledge and understanding in a content area. prior research shows that this thinking disposition impacts performance on tests of one’s reasoning proficiency, but has not examined the impact on the content learning and knowledge acquisition that partially depend on such skills. thus, it is informative to examine individual differences in this thinking disposition in a real classroom context where the goal of the task is science learning via selective integration of information across multiple information sources and inclusion of that information in the form of an explanatory argument. the present study the purpose of the current study was to explore the effects of individual differences in both capacity and dispositional constraints on learning from a multiple-document inquiry task in science. middle school students were given a set of documents about the global temperature system and were asked to write an essay explaining how and why recent patterns in global temperature are different from what has been observed in the past. understanding of the science topic was assessed both by considering the quality of the essays that were written as well as by performance on an inference verification task. the main question for the current study was whether individual differences in domain-general thinking dispositions might have unique effects from reading skill on the understanding that results from a multiple-document inquiry task in science. method participants participants in this study were 59 seventh grade students from 3 science classes in an urban public middle school in the united states. the average age was 13.31 years (sd = .64). the sample was 57% female. self-reported ethnicity was 22% hispanic, 27% african american, international electronic journal of elementary education, 2012, vol.5, issue 1, 63 78. 70 10% asian, 59% white, 30% native american/pacific islander and 25% other. (students were able to select multiple ethnicities and 5 students did not select any.) materials and measures global temperature document set. all participants were given a set of 7 documents containing information related to the causes of global temperature change, based on material that has been used in previous studies with older students about the causes of ice ages (sanchez & wiley, 2006; sanchez & wiley, 2009). five text-based documents covered several main topics including ice ages, the carbon cycle, the greenhouse effect, solar radiation, and energy from fossil fuels. the document set also included a graph of co2 concentrations over the last 400,000 years, presented as its own document. in addition, students were provided with seventh document, titled “changes in global temperatures”, which provided textual background on the methods used to assess global temperatures. this document also included a graph of average global temperatures over the last 400,000 years, and a second graph showing the increases in average global temperatures from 1870 to 2010. the texts were excerpted from several online sources from the united states geological survey, the public broadcasting service, the nasa earth observatory, the environmental protection agency, as well as an extension module from an earth science textbook series (bennington, 2009). to adapt the texts for younger grade levels, vocabulary and sentence structures were simplified. the final text-based documents were on average 326 words long (range: 208-475), with an average in flesch reading ease of 62.36, and an average fleschkincaid grade level of 7.9. the documents were presented to students on pieces of paper contained in a pocket folder, with each of the 7 documents printed on a separate page. inquiry task essay prompt and essay coding. one main source of information about student understanding was the essays that students wrote in response to the inquiry prompt. students were asked to “use this set of documents to write an essay explaining how and why recent patterns in global temperature are different from what has been observed in the past.” student responses to this essay prompt were evaluated for the presence of 5 critical target concepts that directly relate to recent changes in global temperature, and thus address the inquiry question students were asked. these concepts were: 1. we are in an unusually long warming period. 2. co2 levels in the atmosphere are at their highest in at least 400,000 years. 3. fossil fuel burning releases co2. 4. co2 is a greenhouse gas. 5. greenhouse gases in the atmosphere cause warming. all essays were evaluated for the presence of the target concepts by two independent coders, who produced a high level of interrater reliability (krippendorf’s α = .90, p < .05). any differences were resolved through discussion. inference verification task. as another measure of student understanding, a sentence judgment task was created in which students were asked to indicate which of a list of statements seemed true based on the texts they had just read. this test (based on sanchez & wiley, 2006; wiley & voss, 1999) consisted of 18 statements that represented potential connections or inferences that could or could not be made based on the information in the document set. some example items are “in the past 100 years, both fossil fuel use and co2 levels have increased” and “increases in fossil fuel use increase the amount of heat that escapes into space.” the first is an example of a conclusion that is supported by the documents but requires connections across documents. the second is an example of a the role of clear thinking in learning science from multiple-document inquiry tasks / griffin, wiley, britt & salas 71 statement that is false based upon connecting multiple ideas across documents, namely ideas 3, 4, and 5 that were coded in the essays. the items represented 8 correct and 8 incorrect inferences. for every correct and incorrect inference appropriately identified, the students received a single point. an overall proportion score was computed for the task, and higher levels of performance indicated better understanding of the inferences that could be made from the documents. clear thinking scale. students’ clear thinking refers to their commitment to logic, evidence, and reasoning. the 5-item scale assesses the extent to which students place value and importance on reasoning about evidence when forming and revising beliefs. the construct is measured at the most domain general level. the scale incorporates items from the flexible thinking scale (stanovich & west, 1997) and the belief identification scale (sa’, west, & stanovich, 1999), which were revised by kokis et. al.,(2002) to be used with children. the items were selected based upon the criteria that they directly ask about belief revision in the face of new evidence or information. the items used for the clear thinking scale were: 1. i never change what i believe in even when someone shows me that my beliefs are wrong. 2. people should always consider evidence that goes against their beliefs. 3. it's important to change what you believe after you learn new information. 4. people shouldn't pay attention to evidence that contradicts their strongly held beliefs. 5. to decide what is true, you often have to ignore your emotions and stick just to the evidence. students were asked to respond to these items using a 1-6 scale with 1 meaning strongly disagree and 6 meaning strongly agree. the scale was scored by subtracting the average rating for the negatively worded items (1 and 4) from the average of the positively worded items. a difference score between the weighted averages means that the combined influence of the positively worded items on the total score is the same as the combined influence of the negatively worded items. this avoids the problem of giving more weight to positively worded items as a group, which creates a response bias effect such that people who simply anchored all of their ratings at a higher value would receive a higher score. descriptive student and teacher surveys. due to the family educational rights and privacy act (ferpa), official student standardized test scores could not be obtained. to obtain measures of reading skill in lieu of test scores, a teacher survey asked teachers to indicate each student’s level of reading skill relative to their grade level as low, medium or high. a student self-report survey was created to collect basic descriptive information including gender, date of birth, and ethnicity. students were also asked to rate on a 1-to-5 scale their level of interest in science, interest in the topic, and prior knowledge about the topic. procedure students participated in the inquiry activity as part of their normal science classes. all materials for the inquiry activity were distributed to students in folders, including the inquiry task essay prompt, blank writing pages and the document set. students were asked to read along as the inquiry task essay prompt instruction was read out loud. international electronic journal of elementary education, 2012, vol.5, issue 1, 63 78. 72 the full instructions for the reading and writing task were: the primary purpose of reading in science is to understand the causes of scientific phenomena. this means your goal for reading is to understand how and why things happen. to reach an understanding of a new topic in everyday life, we often need to gather information from multiple sources. in today’s task your goal is to learn about the causes of global temperature changes from several documents. you will have to piece together important information across the documents to construct a good understanding. no one text will provide the answer. this task is interesting because you are the one making the connections across documents and coming up with an explanation. no author has already done the work for you. it is also important that you use information from the documents to support your explanation of the causes. your task is to use this set of documents to write an essay explaining how and why recent patterns in global temperature are different from what has been observed in the past. be sure to use specific information from the documents to support your conclusions and ideas. students had access to the documents as they wrote the essays. then, the essays and document sets were collected and students completed the inference verification task, without access to the documents. these were collected and students completed a final booklet including the clear thinking scale and the self-report descriptive student surveys. teachers were asked to fill out the teacher survey while students worked on the inquiry task. the activity was done over two 50 minute periods. results descriptive statistics the descriptive statistics for all variables in the study are displayed in table 1. all variables showed normal distributions and high variance covering the range of possible values. the mean clear thinking score was greater than 0, reflecting that most students had at least slight agreement with an evidence-based disposition. however, there was high variability and many students had negative scores and disagreed with an evidence-based disposition. table 1. means, standard deviations, and ranges for all measures measure mean sd observed range possible range clear thinking 1.19 1.93 -2.67 4.67 -5.00 5.00 reading skill 2.32 0.78 1.00 3.00 1.00 3.00 prior knowledge 3.46 1.18 1.00 5.00 1.00 5.00 science interest 2.98 1.42 1.00 5.00 1.00 5.00 topic interest 2.88 1.25 1.00 5.00 1.00 5.00 essay concepts 1.83 1.66 0.00 5.00 0.00 5.00 inference test 0.70 0.14 0.44 1.00 0.00 1.00 correlations among measures of understanding and individual differences as shown in table 2, there was a significant positive correlation between the two outcome measures of understanding. students with greater conceptual coverage in their essays (essay concepts) also tended to have higher scores on the inference verification task (inference test), despite the fact that the texts were only available during the essay writing. table 2 also shows that both these measures of understanding were predicted by clear thinking dispositions and by reading skill, and that prior knowledge predicted inference test performance. the relationship between prior knowledge and essay concepts was trending in the same direction but weaker and non-significant (p = .15). the role of clear thinking in learning science from multiple-document inquiry tasks / griffin, wiley, britt & salas 73 table 2. pearson correlations among clear thinking, reading skill, prior knowledge, science interest, topic interest, number of essay concepts, and inference test scores measure 1 2 3 4 5 6 1. clear thinking 2. reading skill .26* 3. prior knowledge .27* .27* 4. science interest .11 .01 .08 5. topic interest .13 .05 .01 .73** 6. essay concepts .36** .46** .19 .13 .07 7. inference test .39** .42** .30* .13 .14 .45** note. n = 59. *p < .05. **p < .01. students’ interest in both science and the specific topic failed to predict performance on the essays and the inference test, and were also unrelated to clear thinking and the other predictors. however, the two interest measures were highly correlated with each other. in addition, interest levels differed for male and female students. consistent with prior findings (for a meta-analysis, see weinburgh, 1995) males had significantly higher interest in both science and the topic (ms = 3.32 and 3.42) than females (ms = 2.58 and 2.66), ts(57) = 2.33 and 2.02, ps < .05. gender did not relate to any of the other predictors or to either outcome measure. unique effects of thinking dispositions and reading skill on understanding the main question for the current study was whether individual differences in domaingeneral thinking dispositions might have unique effects from reading skill on the understanding that results from a multiple-document inquiry task in science. to examine this question, reading skill and clear thinking scores were entered simultaneously into a regression predicting the number of key explanatory concepts in the essays. as seen in the top half of table 3, the regression resulted in a significant model accounting for 27% of the variance in essay concepts. the beta tests showed that both clear thinking and reading skill each accounted for significant unique variance. the inclusion of key explanatory concepts increased with reading skill. in addition, regardless of reading skill, students with a stronger general disposition towards evidence-based thinking were more likely to incorporate the key explanatory concepts into their essays. table 3. regression analyses predicting inference test scores and number of essay concepts from clear thinking and reading skill scores predictor r2 f value b sem β t value (dv) essay concepts model .27 10.29* clear thinking .22 .10 .25* 2.13 reading skill .84 .25 .39* 3.31 (dv) inference test model .26 9.81* clear thinking .02 .01 .30* 2.51 reading skill .06 .02 .34* 2.87 note. n = 59. *p < .05. another regression was conducted in which reading skill and clear thinking scores were entered simultaneously to predict inference test performance. the results reported in bottom half of table 3 were very similar to the essay concepts results. the overall model was international electronic journal of elementary education, 2012, vol.5, issue 1, 63 78. 74 significant and accounted for 26% of the variance in test performance. the beta tests show that both clear thinking and reading skill each accounted for significant unique variance. inference test performance was better for students with more reading skill. more importantly, regardless of reading skill, students with a stronger general disposition towards evidence-based thinking were more likely to correctly identify statements that could and could not be inferred by integrating the information from the multiple documents. since prior knowledge of the topic was related to inference performance and clear thinking (see table 2), this analysis was rerun adding prior knowledge as a control predictor. the results did not change, except for a slight increase in the total variance explained from 26% to 28%. conclusions across two measures of student understanding, the results of the present study demonstrate the influence of both reading skill and a domain-general thinking disposition on learning science from multiple-document inquiry tasks. these influences were independent from each other and from self-reported ratings of prior topic knowledge, interest in the topic and interest in science. of these individual differences, only the interest ratings were not related at all to understanding. although reading skill and prior topic knowledge were not assessed with standardized measures, the measures that were used did predict understanding as expected and were correlated with each other, suggesting they are capturing variance in their respective constructs. the two measures of understanding (essay concepts and inference tests) similarly correlated with reading skill and with clear thinking, but correlated only modestly with each other. in addition, the inference test but not the essay concepts were significantly related to prior knowledge. the lack of relation between essays and prior knowledge makes sense given that the documents were available during writing, so students did not need to rely upon retrieval from long term memory in order to construct a more complete argument. thus, the ivt and essay measures reflect somewhat different aspects of multiple-documents comprehension. yet, a motivating disposition towards considering evidence (clear thinking) related to both of these different aspects of comprehension independently from reading skill, prior knowledge, and topic and domain interest. this is consistent with stanovich’s (2012) distinction between individual differences in what a person might be capable of (e.g., reading skill) versus what a person might be disposed to do. this study demonstrates that both are required for successful learning from multiple-document inquiry tasks in science. although the current study does not directly test the md-trace model, the finding that clear thinking scores predicted middle school students’ learning from multiple documents is consistent with the importance of the task model. bråten et al (2011) hypothesized that epistemic beliefs contribute to the creation of a task model and it is likely that thinking dispositions function in much the same way. it is expected that readers who are disposed to using evidence and reasoning to form and update their beliefs will have a very different task model from those who do not. as a result, they will create different subgoals to guide reading. for example, students with an evidence-based disposition will be expected to seek coherence across explanatory elements and look for evidence to support claims. these subgoals will lead to the integration of more of the key causal concepts from the document set into their mental model of climate change. the present results also highlight that a task model may be more than what a learner thinks is expected and required for learning, but may also include their personal goals related to their desire to learn and update their views versus to protect and maintain their existing views. an interesting direction for future research would be to investigate the manner in which thinking dispositions influence the development of a task model. the role of clear thinking in learning science from multiple-document inquiry tasks / griffin, wiley, britt & salas 75 limitations and future directions the working assumption behind the present findings is that students’ dispositions towards evidence-based thinking impacts how they approach a multiple-documents inquiry task, such dispositions make students more likely to engage in the kind of integrative, coherencebuilding, argumentation processes that have been shown to improve learning in these contexts (e.g., wiley et al., 2011). the previously reviewed literature shows there is much variance in reading behaviors and strategies when readers are faced with multiple documents. thinking dispositions may be a generalized individual difference that contributes to this variance. however, we note that the current study only measured the learning outcomes that were presumed to result from these different behaviors, but did not include any on-line measures of processing and reading strategies to verify actual differences in processing. griffin and ohlsson (2001) speculated that people who had previously formed a belief on a topic via evidence-based reasoning rather than relying on affective preferences may be better able to represent new belief-relevant concepts. thus, an alternative to differences in how readers are actively engaging in the task is differences in how their past reasoning on the topic impacts their ability to represent the concepts. future research is needed to provide evidence that readers vary in their processing during reading in ways that might mediate the observed learning outcomes. implications for instruction the fact that such a general thinking disposition was able to show relations to learning on a specific topic within science is pedagogically useful. the trend in research on the related construct of epistemology has been toward measuring more domain-specific rather than more general thinking dispositions (e.g., hofer, 2006). in fact, braten and stromso (2010) have even argued for using topic specific epistemology, and have demonstrated that it can be used as a successful predictor of science learning for multiple documents on that topic. predicting learning does seem to benefit from measuring epistemology in more specific ways (for a review, see muis, bendixen, & haerle, 2006). however, from a pedagogical perspective, identifying general dispositions that could improve learning may be more pragmatically useful as targets for instruction. domain-specific and topic-specific dispositions imply that separate pedagogies would be needed to target the development of thinking dispositions within each domain or on each specific topic. any benefit of such interventions would be limited to that domain or topic. the present results suggest that there are more general thinking dispositions regarding the value of evidence that can also have a substantial impact on learning. there has been little work on interventions targeting an evidence-based disposition. the fact that the disposition itself has some domain generality does not imply that topic-specific learning activities would be ineffective in fostering a dispositional change. topic specificity may be necessary in order to expose students to examples of such thinking and to have them engage in tasks that require it. long-term impact may prove difficult if this disposition reflects core values related to commitment to evidence versus the perceived value of sticking to one’s beliefs. such values would seem to be shaped by the social reinforcement students receive in many areas of life outside of school, from their home, religious upbringing, media, and popular culture. on the other hand, there appears to be little existing effort in schools to directly and explicitly foster an evidence-based disposition. thus, even minor interventions could notably increase students’ exposure to the importance and utility of adopting such a disposition. this is another direction for future research. if such a general international electronic journal of elementary education, 2012, vol.5, issue 1, 63 78. 76 disposition can be effectively encouraged, developed or leveraged through instruction, it has the potential to impact learning across topics and domains more generally. • • • acknowledgement the research reported here was supported by the institute of education sciences, u.s. department of education in part by grant r305b07460 improving metacomprehension and self-regulated learning from scientific texts from the casl program to thomas d. griffin and jennifer wiley and in part by grant r305f100007 reading for understanding across grades 6 through 12: evidence-based argumentation for disciplinary learning from the reading for understanding research initiative to anne britt, thomas griffin and jennifer wiley. the authors thank karyn higgs, kris kopp, brent steffens, andrew taylor and the other members of project readi for their assistance and contributions. the opinions expressed are those of the authors and do not represent views of the institute of education sciences or the u.s. department of education. thomas d. griffin. research professor, department of psychology, university of illinois at chicago. his research interests include contexts and individual differences that impact learning in science and history, the impact of affect-based beliefs on the comprehension and acceptance of scientific theories, and the factors that determine metacomprehension monitoring accuracy for explanatory texts. jennifer wiley. professor, department of psychology, university of illinois at chicago. her research interests include contexts that improve learning from informational texts in history and science, and the role of individual differences, especially in working memory capacity and spatial ability in comprehension processes. m. anne britt. professor, department of psychology, northern illinois university. her research interests include argument comprehension and production, and learning and reasoning from multiple documents. carlos r. salas. graduate student researcher, department of psychology, university of illinois at chicago. his research interests include text comprehension, argumentation, and belief formation. references baron, j. (1985). rationality and intelligence. cambridge, england: cambridge university press. bennington, j. b. (2009). the carbon cycle and climate change. cengage learning. bråten, i., britt, m.a., strømsø, h.i., & rouet, j.f. (2011). the role of epistemic beliefs in the comprehension of multiple expository texts: towards an integrated model. educational psychologist, 46, 48-70. bråten, i., & strømsø, h.i. (2010). effects of task instruction and personal epistemology on the understanding of multiple texts about climate change. discourse processes, 47, 1-31. bråten, i., strømsø, h.i., & britt, m.a. (2009). trust matters: examining the role of source evaluation in students’ construction of meaning within and across multiple texts. reading research quarterly, 44, 6–28. britt, m.a., perfetti, c.a., sandak, r., & rouet, j-f. (1999). content integration and source separation in learning from multiple texts. in s. r. goldman, a. c. graesser, & p. van den broek (eds.). narrative comprehension, causality, and coherence: essays in honor of tom trabasso (pp. 209-233). mahwah, nj: lawrence erlbaum associates. the role of clear thinking in learning science from multiple-document inquiry tasks / griffin, wiley, britt & salas 77 britt, m.a., & rouet, j-f. (2012). learning with multiple documents: component skills and their acquisition. in m.j. lawson & j.r. kirby (eds.), the quality of learning: dispositions, instruction, and mental structures. cambridge university press. cerdán, r., vidal-abarca, e. (2008). the effects of tasks on integrating information from multiple documents. journal of educational psychology, 100, 209-222. chi, m. t. h. (2000). self-explaining expository texts: the dual processes of generating inferences and repairing mental models. in r. glaser (ed.), advances in instructional psychology (vol. 5, pp. 161-238). mahwah, nj: erlbaum. conley, a. m., pintrich, p. r., vekiri, j., & harrison, d. (2004). changes in epistemological beliefs in elementary science students. contemporary educational psychology, 29, 186–204. goldman, s .r., braasch, j. l .g., wiley, j., graesser, a. c., & brodowinska, k. (in press). comprehending and learning from internet sources: processing patterns of better and poorer learners. reading research quarterly. griffin, t. d. (2008). faith: serving emotional epistemic goals rather than evidence coherence. in v. sloutsky, b. love, and k. mcrae (eds.), proceedings of the 30th annual conference of the cognitive science society, pp. 2059-2064. austin, tx: cognitive science society, inc. griffin, t. d., & ohlsson, s. (2001). beliefs versus knowledge: a necessary distinction for explaining, predicting and assessing conceptual change 364-369 in j. d. moore and k. stenning (eds.), proceedings of the 23rd annual conference of the cognitive science society, pp. 364-369. austin, tx: cognitive science society, inc. griffin, t. d., wiley, j., & thiede, k. w. (2008). individual differences, rereading, and self-explanation: concurrent processing and cue validity as constraints on metacomprehension accuracy. memory & cognition, 36, 93-103. hofer, b. (2006). domain specificity of personal epistemology: resolved questions, persistent issues, new models. international journal of educational research. 45, 85-95. kintsch, w. (1994). learning from text. american psychologist, 49, 294–303. kintsch, w. (1998). comprehension: a paradigm for cognition. new york: cambridge university press. kintsch, w., & van dijk, t. a. (1978). toward a model of text comprehension and production. psychological review, 85, 363-394. kokis, j., macpherson, r., toplak, m., west, r. f., & stanovich, k. e. (2002). heuristic and analytic processing: age trends and associations with cognitive ability and cognitive styles. journal of experimental child psychology, 83, 26-52. mason, l., boldrin, a., & ariasi, n. (2010). epistemic metacognition in context: evaluating and learning online information. metacognition and learning, 5, 67–90. muis, k. r., bendixen, l. d., & haerle, f. c. (2006). domain-generality and domainspecificity in personal epistemology research: philosophical and empirical questions in the development of a theoretical model. educational psychology review, 18, 3-54. perfetti, c.a., rouet, j-f., & britt, m.a. (1999). towards a theory of documents representation. in h. van oostendorp & s.r. goldman (eds.) the construction of mental representations during reading (pp. 99-122). mahwah, nj: lawrence erlbaum associates. rouet, j-f., & britt, m.a. (2011). relevance processes in multiple document comprehension. in m.t. mccrudden, j. p. magliano, & g. schraw (eds.), text relevance and learning from text (pp. 19 52 ). greenwich, ct: information age publishing. international electronic journal of elementary education, 2012, vol.5, issue 1, 63 78. 78 sá , w., kelley, c., ho, c., & stanovich, k. e. (2005). thinking about personal theories: individual differences in the coordination of theory and evidence. personality and individual differences, 38, 1149-1161. sá, w. c., west, r. f., & stanovich, k. e. (1999). the domain specificity and generality of belief bias: searching for a generalizable critical thinking skill. journal of educational psychology, 91, 497-510. sanchez, c., & wiley, j. (2006) effects of working memory capacity on learning from illustrated text. memory & cognition, 34, 344-355 sanchez, c., & wiley, j. (2009). to scroll or not to scroll: interactions of text presentation and working memory capacity. human factors, 51, 730-738. sanchez, c. a., wiley, j., & goldman, s. r. (2006). teaching students to evaluate source reliability during internet research tasks. proceedings of the 7th international conference of the learning sciences, bloomington, in. stanovich, k. e. (2012). on the distinction between rationality and intelligence: implications for understanding individual differences in reasoning. in k. holyoak & r. morrison (eds.) (pp. 343-365), the oxford handbook of thinking and reasoning. new york: oxford university press. stanovich, k. e., & west, r. f. (1997). reasoning independently of prior belief and individual differences in actively open-minded thinking. journal of educational psychology, 89, 342-357. stanovich, k. e., & west, r. f. (1998). individual differences in rational thought. journal of experimental psychology: general, 127, 161-188. trabasso, t., & van den broek, p. (1985). causal thinking and the representation of narrative events. journal of memory and language, 24, 595–611. weinburgh, m. (1995). gender differences in student attitudes toward science: a meta-analysis of the literature from 1970 to 1991. journal of research in science teaching, 32(4), 387-398. west, r. f. stanovich, k. e., & toplak, m. e. (2008). heuristics and biases as measures of critical thinking: associations with cognitive ability and thinking dispositions, journal of educational psychology, 100, 930-941. wiley, j., ash, i.k., sanchez, c.a., & jaeger, a. (2011). clarifying readers’ goals for learning from expository science texts. in m. mccrudden, j. magliano, & g. schraw (eds.), text relevance and learning from text (pp. 353-374). greenwich, ct: information age publishing. wiley, j., goldman, s., graesser, a., sanchez. c., ash, i., & hemmerich, j. (2009). source evaluation, comprehension, and learning in internet science inquiry tasks. american educational research journal, 46, 1060-1106. wiley, j., griffin, t. d., & thiede, k. w. (2005). putting the comprehension in meta-comprehension. journal of general psychology, 132, 408-428. wiley, j., & voss, j. f. (1996). the effects of "playing" historian on learning in history. applied cognitive psychology, 10, 63-72. wiley, j., & voss, j. f. (1999) constructing arguments from multiple sources: tasks that promote understanding and not just memory for text. journal of educational psychology, 91, 301-311. international electronic journal of elementary education, 9(3), 693-708, march, 2017 issn:1307-9298 copyright © iejee www.iejee.com empowering children with adaptive technology skills: careful engagement in the digital information age miftachul huda a  kamarul azmi jasmi a aminudin hehsan a mohd ismail mustari a masitah shahrill b bushrah basiron a saikou kawsu gassama a a universiti teknologi malaysia, malaysia b universiti brunei darussalam, brunei darussalam received: 5 january 2017 / revised: 15 february 2017 / accepted: 25 february 2017 abstract the technological advancement occurring in the world today has a potency to create positive and negative side effects especially on children. the positive side may entail increased empathy and acceptance of diversity through modelling the prosocial behaviours, while the negative can be manifest in aggressive behaviour, risky sexual behaviour, and substance abuse. these challenges pose a need to draw a particular attention to children’s rights of protection and care in relation to the use of technology. this paper sought to critically explore the way in which children adapt technology skills and the way they respond towards media influences. the findings revealed that the adaptive technology skills are needed in providing a considerable guidance for child protection and careful engagement towards digital information as part of their rights to natural growth and development. keywords: adaptive technology skills, careful engagement, digital information, media influences, and children’s protection and rights introduction the advances in technology in the 21st century entails that children are naturally growing within arenas of higher levels of information access. it is critical for children to adapt their capacity to get knowledge and to develop their skills through technology usage. the dynamics of technological revolution has both positive and negative effects. studies have shown that it may potentially affect children negatively manifesting in aggressive behaviour, risky sexual attitude, and substance abuse. on the other hand, positive impact may include promoting empathy and acceptance of diversity through modelling prosocial behaviours (o'keeffe & clarke-pearson, 2011). since children feel inquisitive and would always like to inquire, and to reflect on what to do with information acquired, there is a need to provide considerable guidelines on technology use so as to support them their potential and capacities. it is noted that children face numerous risks as they interact on various social  corresponding author: miftachul huda, faculty of islamic civilisation, universiti teknologi malaysia, email: halimelhuda@gmail.com http://www.iejee.com/ international electronic journal of elementary education, 9(3), 693-708, march, 2017 694 media platforms. for example, they grapple with drawing up a picture of emerging conceptions in their digital world and the philosophical struggles understanding their childhood experiences. this study posits a need in guiding children appropriately in facing the challenges in digital environment like cyber bullying, social media depression, accessing inappropriate content (christensen & aldridge, 2012). furthermore, it is necessary to take into account that the need for protecting on media platforms is inextricably linked to children’s rights protections. the advocacy role relating to media influences, and the theoretical framework for protecting children will be examined by proposing a reference model for child adaptability to technology usage. this approach is preventive action, taking care when using technology in ways that can be applied through guiding and filtering issues and ideas coupled with organised and cautious parental involvement and counselling (tirri & husu, 2002). there is a need for paying serious attention to controlling their behavioural dependence on technology usage by ensuring parental involvement and taking care of the children’s adaptation process on appropriate uses of technology (buckingham, 2013a). there is also a need for collaboration between parents, practitioners, and policymakers so as to maximize prosocial aspects of social media for the appropriate development of children and adolescents. this approach can be achieved by well-designed guidelines which develops their skills to question, analyse and evaluate, whilst accessing information freely but under the control of their seniors (buckingham, 2013b). regardless of the challenges in nurturing an appropriate culture of technology use free from violating children’s rights, children must be guided wisely in the way that makes them engage and reflect on the accessed information. this will allow them to be mindful of adult materials which can hamper their moral and cognitive development. the need to maximise benefits and reduce the harm emanating from the media and technology uses, has become a core point of discussion on how to utilize technologies as an attempt to empower children’s adaptive technological skills. in the light of the aforementioned, there is an urgent need to promote an empowerment process that takes care of the children and their natural growth within various digital media platforms. thus, this paper aims to explore the attempt to empower the adaptive technology skills among children and their awareness about media technologies and influences. literature review information pattern in the digital era in the digital era, information can spread fast around the world through digital devices and by creating machine-readable formats which users can easily get access to (agichtein et al, 2008). there are many distinct features for which electronic messages can be transmitted. this includes social media which enables sharing of information and ideas thus generating databases into a form with particular interest in expression through virtual means. since the model of the touch screen has been widely developed, this feature seems unique compared to traditional media forms, and is easily accessible (mcnaughton & light, 2013). there are a number of digital media forms which are integrated into the model of computing. the outstanding value of digital media in society is creating interaction skills combined with further technological advances manifest in platforms such as instagram, facebook, linkedin, twitter, among others. in the digital era, this trend seems nuanced to the process of transmitting information where intellectual assets may also be included as an enduring value (raju, 2014). it is noted that connecting the users by rethinking new approaches and perspectives is needed before going into the action stage. prior to application, the way to manage information pattern represented by the current situation refers to competitive achievement widely organised to create the way to interact with empowering children with adaptive technology skills / huda, hehsan, jasmi, mustari, shahrill, basiron, & gassama 695 others especially in cyberspace, the virtual world (dabbagh & kitsantas, 2012). it refers to how the objects of the virtual world can be achieved with an open content movement widely seen as the activity by which the users do. in addition, there are two models of information patterns which can be exerted to transfer the knowledge information to other users in exploring the virtual world. these two, are namely, explicit and tacit (nonaka & von krogh, 2009). explicit information refers to the access of verbal patterns of information which are easily transmitted from one set of media to another. on the other hand, tacit patterns of information refer to actually performing the knowledge acquisition by addressing the reliability of ‘content fit’ in the digital era (stevens, millage & clark, 2010). apart from that, the outstanding potential concern which would lead to being readily articulated to communicate with others in the context-specific value to transform needs to be adjusted to knowledge with the ability of combining such types of information (holste & fields, 2010). moreover, it can be generated to give a right path of guidance on the communication pattern in the cyber environment with consistently stressing the moral engagement. in order to guide the processes and provide the perceived value of information pattern, focusing on individual and social concerns should be given more attention, applying the right path of knowledge. the media information has a significant, wide-ranging and complex impact on society and culture through processes explicitly driven by digital transitions (koltay, 2011). in light of this, it is necessary to keep accessible the information systems in the digital media, including developed devices, which can be utilized to become prominent parts of people’s everyday life, particularly the youth and children. whilst, engaging in social media there may be need to encourage notions of critical thinking, bringing awareness to both positive and negative aspects and being mindful of the essential component of feelings, its merits and demerits, in a digital society. in an attempt to consider the potencies of digital information, the moral engagement has to be the basis of integrating media technology in a creative way. it is to enhance the children’s ability to have an opportunity to access information online more enjoyably and in an appropriate way (shenton & fitzgibbons, 2010). it is noted therefore that adopting digital tools in the process of transforming and transmitting information is required so as to channel it in a fun way, in line with ethical engagement suitable for children’s consumption. the way and manner of adapting moral principles to digital tools should be integrated with professional skills so as to drive the process in getting feedback on digital information. the trends on mobile information transmissions in the social media allow the reaping of rich benefits in a safe digital environment. this immersion in digital information and access, often leads the creation of a virtual reality. in view of these interactions with digital technologies, it is critical to prepare children and the youth to effectively access educational information to support their varied educational inquiries. the way to use digital technology wisely and safely should therefore be directed at developing social skills in order to discourage addictive behaviour and exposure to the negative side-effects related to technology use, especially on the net (strasburger et al, 2013). in order to promote effective child protection during teaching and learning, the supervision, guidance and advice from teachers and parents should provide an ethical supportive structure, in line with the framework proposed herein., creating awareness of such adverse issues on digital media such as on web links containing both good and bad data could be pivotal to the needs of children and their protection. hence it would constitute an integral part of the cultural and daily livelihoods with potential value of reaping the positive benefits for the greater good of the society. child adaptive skills on media information since today’s generation is bombarded by media technology, where people live, interact and communicate with each other using digital devices, all information can be easily accessed international electronic journal of elementary education, 9(3), 693-708, march, 2017 696 and forwarded using various digital devices. these include the internet, email, twitter and facebook. moreover, there are a number of digital devices which can be used to support daily information transmissions in this cyber age, which includes tablets, smartphones, computers or ipad.. it is generally accepted that the use of such portable digital devices to enable people in their daily interactions should be cautiously made accessible to children especially the digital gaming devices (o'keeffe, & clarke-pearson, 2011). children often feel entertained by a variety of internet devices, which enables them further access to useful information and innovative devices. as a result, the increasing demand to use play digital devices among children has been clearly observed in the last decade (strasburger et al., 2013) hence, the need for a wiser approach in designing suitable adaptive processes and skills. this also applies to social media information transmissions which arise as a necessity in today’s everyday life. furthermore, the adaptive skills on media information transmissions among the youth, both at home and in community settings have to be developed in a purposeful way. these skills could be taught and promoted within the school curriculum whilst childhood education settings may also provide nurturing of adaptive skills, appropriately embedded within technology applications (campbell et al., 2006; hutinger & johanson, 2000). the uses of technology applications both at home and in early childhood settings may require the need for trained practitioners (mettler et al., 2011; shepherd, 2004). in order to implement adaptive processes, the use of applications should be more widely prioritized in transmitting information useful to children in line with their age and development needs. with regards to an attempt to see the essence of potential value which childhood possess, bringing many benefits is to facilitate growth and identity which enables people to connect with each other. there is also a need to appropriately guide the usage of media information by paying attention to insightful guidelines, since this may enhance the child’s personal identity and value. whilst children may want to watch whatever picture or image they like, infusing a positive ethical dimension into the development of their technological skills may enhance their personal and social awareness. collaboration between t the home, the school and other stakeholders is pivotal in helping the children develop such necessary skills through fun whilst accessing a variety of service application to strengthen their critical thinking skills. there are wide ranges of media information platforms featured amongst educational services to enable children in getting enhanced skills along with critical thinking abilities in any subject, for instance, in maths and these are found to be appealing (cemalcilar et al., 2005; christensen & aldridge, 2012). , it is noted that providing the young generation with unlimited access to the internet via tablets and smartphones has both opportunities and challenges. according to a number of studies, the youth and children showed that such devices can change their thinking skills and capability to absorb information (collis et al., 2013; druin, 2009). however, the approach to avoiding prolonged exposure to ‘useless’ online materials should be regarded as an attempt to keep a balance between providing wider opportunities and positive conscious control. providing children with tools which may assist them on the use of technology as a creative substance can help their development in the digital age (buckingham, 2013a). it is noted that as long as the adaptive process goes together with consistent surveillance under a guardian or parent, the potency children have can develop well in utilizing the digital devices and media information platforms. challenges and opportunities of adaptive technology skills the emergence of widespread technological use and innovation in the media information has both positive and negative effects in children’s daily lives. the emerging innovations with some new features and digital tools enable individuals from all generations and walks of life to connect with each other and interact like never before. these benefits of technology empowering children with adaptive technology skills / huda, hehsan, jasmi, mustari, shahrill, basiron, & gassama 697 use can concisely be taken into satisfying the human needs in terms of activities in their social system (anshari et al., 2017; chayko, 2014; chen & hwang, 2014; fenwick & edwards, 2016; huda et al., 2016). for instance, downloading music and watching movies at home can be carried out with an array of digital technologies. these may include mobile devices combined with interactive social networks. the virtual interaction may also go well through facebook, myspace, and twitter. as a result, such distinct characteristics do enhance the positive aspects of technology in society by promoting social good and constructing general approaches underlying self-performance and awareness of these technologies (davies, coleman & livingstone, 2015). in this regard, digital technology has all aspects constructed into worldwide communication system and integrated into the activity which humans may fulfil by adapting it to the social system. an attempt to engage the process with adaptive technology skills should be committed to the time management together with planning (huda et al., 2017). this refers to the way to support assessing the multi-channels of knowledge sources to extract new insights of value combined with strengthening the management skills into adaptive technology competencies. despite digital technological advances, there are numerous side-effects which are quite apparent and pervasive in the digital social media. among them are a number of unlikely phenomena, such as depression amongst the users (dehue, bolman, & völlink, 2008; erdurbaker, 2010; gillespie, 2008). depression has negative effects, which seem to have spread amongst someusers including both adults and children, culminating in aggression, natural and unnatural sexual behaviour, substance abuse, disorderly eating habits, and academic difficulties (strasburger et al., 2010).the number of these challenging issues is likely to increase, leading to other dangerous elements threatening children and their development (collis et al., 2013). another instance of the negative effects of technology is how quickly it is pushing younger generations into early maturity in society (davies et al., 2015). children can now see whatever they want online, can learn anything or even view sensitive images which have no relevance to their needs. it is noted that when they are free from proper supervision, children usually access such prohibited material (gillespie, 2008). through parental guidance and supervision, it is hoped that the uncensored sources from the internet, would be safely under parental control and appropriate safeguards followed (gentile, 2014). the need to strengthen the impactful power of control under adult guidance should be given due consideration both at school and at home. apart from some potential harm highlighted above, there are also evidences that media can be beneficial to the youth such as increasing empathy and acceptance of diversity through modelling prosocial behaviours (buckingham, 2013); and also developing children’s early literacy skills through educational programming (mcloughlin & lee, 2010). when societies know more about technological developments and innovations, they are abler to take advantage of it (luo et al, 2010).. knowledgeable uses of technology tools in every field can result in the utilization of its latest innovations leading to enhanced interactions (keith, & martin, 2005). the options users have in social media become more challenging due to increased impact of daily communication. this can be seen from making facebook friends to posting status, sending emails or pictures to catch up with others instantly. the mode of communication and discussion is often transparent about its broadcast content. the ability to adapt and navigate communication technologies may enhance one’s performance thereby transforming the way the users interact with others in an appropriate, ethical and safer way wisely. international electronic journal of elementary education, 9(3), 693-708, march, 2017 698 analysis and discussion the need to strengthen adaptive skills with moral engagement there are several emerging challenges of technology usage in the social media such as facebook, linkedin, and even messenger, which impact upon behavioural challenges such as student-teacher relationship (campbell, 2009); and cyber bullying (erdur-baker, 2010). there is therefore an urgent need to adopt strategies on technology uses so as to resolve the challenge faced by institutions such as schools. the role of information communication and technology (ict) in contribution to the social system and technical structures is engaged into the framework of feedback on strategy and application in technology adoption (fenwick & edwards, 2016). understanding of the alignment with critical discourse will enable users to integrate the modes of interacting with others in an appropriate way. this can be supported by the model of decision making integrated into the feedback levels coming from the decisionlearning structures. as a valuable contribution with the main focus on adopting and utilizing tool in the digital era, creating feedback levels would demonstrate inherent contribution along with technical tools of ict. in line with exposing the creative way considered to be the professional adoption towards the use of technology tools, the role of strategic planning has a significant role to engage both the educational technology which may be implemented in the school program and the society concerned (fenwick & edwards, 2016). it means that such course redesign which takes a strong leader that is willing to challenge in doing difficulties enables the users to assess what is needed as an attempt to pay attention in creating the vision by communicating with the team of experts. in this regard, the vision is needed to build the way of implementation skills through teaching practices and methods. hence, the resources are essential in the framework to address dimensions of technology integration and course redesign. in terms of considering the integration among the dimensions about the professional adoption, there needs to be strong commitment in terms of time instructors invest in integrating the technology tool into the course in the model (gupta & hee-woong, 2007). the level of current approaches lies in the core model which underlies organisational initiatives in providing the motivational basis (kaplan & haenlein, 2010; koltay, 2011). in particular, the personal values and goals indicate that a causal relationship between time commitment and competence has a particular essence to the identification of skill development. the skills set out in integrating technology may enhance pedagogically sound way to signify the quality course design and teaching expertise. in using the instructional design model as a guide to serve as a frame of reference, the professional competence can lead to the redesigning of a variety of instructions which can support multi-media experts. ethical norms in digital information seem to have been important achieving a good life and contributed to the adoption-derived information technology standard (spence, 2011). this goal, specifically, can be useful in terms of personal and social orientation. the moral principle states that the moral worth of an action is determined solely by its usefulness, summed among all beings. in this social context, the credibility of information is an important prerequisite for creating good life among all the users. this ethical norm becomes important in guiding to the right path by evaluating and applying digital information. as a result, social media can be used to transmit information by consistently following ethical considerations in dealing with individuals and society. the normal human interactions from face to face or live and direct way can be accomplished through digital information technology. since information sharing can make life easier, for example by allowing us to build relationships with one another, create business contacts, or get news updates, carelessness in information sharing on the other hand can cause confusion and even conflict if users spread information which is not true. it is not uncommon for hostility arises among groups as a result of misinformation (rainey & goujon, 2011). in addition, there can be empowering children with adaptive technology skills / huda, hehsan, jasmi, mustari, shahrill, basiron, & gassama 699 dangerous consequences when there is no way for users to discriminate among news, fact, prejudgment, and gossip (sensoy et al., 2013; wakunuma & stahl, 2014). these ideas are relevant to both face-to-face interactions and interactions via technology adoption. consequently, disseminating information presents a particular challenge, which necessitates possessing good ethical foundation. in short, attainment of the social good should become an ultimate goal of the digital information acquisition with cautious consideration for ethical engagement. careful engagement with children’s learning in digital information the digital information which refers to the current extent of ict has undergone such concerns in driving the societal benefit and also educational process by using the technology’s transformational impact. with regard to seeing as their opportunities, children may have exciting ideas to present key information which can be explicitly seen in the social and cultural issues. by tapping their finger on the screen of a tablet for instance, they can learn in creating pictures to express their skills of social and individual concern through their own way (hutinger & johanson, 2000). moreover, the skills which they adopt actually refer to the performance of personal skills on digital media management process, which enables them to live interaction to the virtual one. with regard to the educational field, the features would enable to enhance teaching and learning with the technology developments such as mind training software, computers, digital cameras, projectors and even powerpoint presentations and 3d visualization tools. all these refer to the additional sources which can support instructors to help children engage with learning process through grasping a concept easily. integrated with visual explanation into making interactive and interesting circumstance, the distinction of making learning enjoyable enables them in getting some opportunities by empowering the technology tool in the particular design such as curriculum design, informal one, and housing setting, informal one (raju, 2014; shenton & fitzgibbons, 2010). with this regard, circulating the learning materials through maximizing the facilities such as using the internet can make easier in enduring educational system. it aims to ensure the process of enriching learning experiences and possibilities among the students (othman et al., 2016) to get involved in enhancing and transforming the curriculum which the stakeholders have prepared (mettler et al., 2011). as a result, achieving students raise the standard of the learning process dependent on the way which they can organise appropriately. it refers to how they can perform moral engagement to adopt digital tool integrated with creative ways. in this regard, the existing contributions are actually needed to elucidate in further on how the ethical engagement with adaptive skills can contribute to drive the process of transforming the technology in the education in a right way. apart from that, exposing to internet use in learning to operate and manage students’ way of accessing information refers to the useful ways on how to describe technology tool which enables them to stand strong in term of self-esteem and confidence (jackson et al., 2010). it should be engaged in the global competition where many matters to do characterised by technology, since its functionality in terms of social and economics aspect is needed to the personal ability based knowledge to the school environment and daily life. an effort to control such changes and diffusion needs to take into account building sustainable relationship into the social and educational aspect which technology can help facilitate. in the digital era, understanding of the basic rule to adopt is required to see its impact which can become an inextricable link between technology and the advanced knowledge. it is important to note about this era where virtual acts are essentially higher than direct one relies on the knowledge which is produced frequently adapted to the changing environment. the attainment of mixing and matching rates of knowledge is regarded to be the development for a dynamic and ever-lasting world. moreover, it can be seen as the international electronic journal of elementary education, 9(3), 693-708, march, 2017 700 control of humans in the understanding of the knowledge embedded within the human genome about how we relate to the world which is generated in association with technology and freely available on the internet to raise the questions about our assumptions of control (rainey & goujon, 2011). driving evolutionary system which enables to make knowledge increasingly become more mobilised to manage, software agents with replacing the search function of unknowingly matching words will exchange knowledge without human intervention. in the digital era, both implications and innovations in shaping the human experience increasingly affect to engage with media entertainment and information to connect with each other and in collective (rainey & goujon, 2011). this would potentially result in the effects on humans of increased engagement with media platforms, services, and content. with the key insights into the emerging media consumer’s behaviours, it would be how hyper-connection to the digital tool could be affected by various aspects of human lives with a wide range of individuals, organizations, and societies. in line with changing condition involving transforming how to interact, develop, work, and play affected by digitization involving media, entertainment, and information with newer generations leading to the evolution, there has emerged such challenges on this opportunities (shepherd, 2004). with regard to the challenges of technology use among the humans, paying particular attention to adapting the ethical basis of the emerging technology to reach positive implication for both individuals and society is needed for providing ethical based instructional suggestion with adaptive skills. in order to reach this, attempts to help users better understand the importance and implications of virtual interaction in actions within the digital identities in digital media should be engaged with the way to use properly and wisely. it is necessary to take into account referring to foster positive implications of increased digital media use and address potential issues by proposing a reference model of ethically based instruction associated with adaptive skills in the digital era within emerging technologies and converging technologies. cooperative engagement of children on awareness of digital information digital information refers to the knowledge which is usually utilised in transmitting information through application of a wide range of digital devices, like mobile, tablet, laptop or pc (shepherd, 2004). all these can be engaged into the network which has a particular value to generate devices in digital literacy. it indicates that digital skills to operate and use in a daily life have an outstanding initial representation about the certain subject of what to conduct in a network device. in the case of children, since they have the potency to follow what they see and listen to, careful awareness in guiding them in virtual interaction should be taken into consideration in particular about each single activity in fun and creative way. encouraging kids to have creative and critical thinking comes across within and outside of class (strasburger et al., 2013; voogt et al., 2013). the extent of encouraging them can be facilitated through letting them play games with others which enable to cooperative experience (dabbagh et al., 2016). this would lead to the solidarity among them. it can be seen there is a valuable insight to take care in guiding them in terms of the format of communication and information content. with regard to the way to choose in developing initiative amongst the national approach to the internet regulation, imposing internet filtering can also be applied with controlling access to the subject content. the technical approach as an attempt to control the message consequently offers the multiple ways which can be employed at this stage. creating the filtering techniques to deploy the restricted access in network level may be carried out, where amongst guardians, parents, educators, and other authorities may also have to access the tools into the programs generated in the digital devices. those are the following stages (internet and children, 2016). proxies and software that can allow or block specific sites and protocols (including anti-virus protection, empowering children with adaptive technology skills / huda, hehsan, jasmi, mustari, shahrill, basiron, & gassama 701 email spam filters, pop-up blockers, anti-spyware, cookie deletion software, etc.); content filtering software that finds and blocks specific content or websites; and configuration options to set site privacy and monitoring features (e.g., google safe search filter, privolock). moreover, cooperating children with awareness in digital information can effectively apply their knowledge and experience gained. in addition to setting up the continual basis where the access to search for material content has the connectivity and flexibility to adjust the children, it is necessary to integrate children’s material needs together with digital media devices. apart from that, the way to conceive the children’s ability to access a variety of such internet would enable them to see the exact risk. as an important role to help them understand something useful to their growth development by safety on the internet use, the role of awareness in digital information (gackenbach, 2011) should be embedded in protecting them against the threats such as inappropriate content or cyberbullying (dehue, bolman & völlink, 2008; erdur-baker, 2010). in this regard, internet safety education here refers to the conceptual framework which considers the critical engagement in protecting them towards the cyber threat, which enables them to have a careful engagement infrequently accessing the internet (edwards et al., 2016). as part of the role in enabling the children’s safety on the internet use, the way to help them understand the risk and safety online will make them more careful towards the digital information. by allowing them to get access online, the continual control would become a necessity to go through in a safe and responsible way with such opportunities. an attempt in the commitment to keep away immoral attitude such as due to the addition of online use would help better understanding on impacts of online use apart from its benefit (luciana, 2010). it is important to keep in mind that the balance between exposing online with continual control is important to see its unprecedented danger. as a result, the idea of the resilience-based school of thought at this stage to deal with the way of adaptive capacity is an essential skill which drives the ability to adapt to changed circumstances in fulfilling the proper content. in short, addressing the issues related to the safety in online access is necessarily integrated with cooperating children awareness. strengthening children adaptability with appropriate information content in the digital era, most of the people in the modern society interact and communicate with others using their digital device (westerman et al., 2014). such produced information contains sometimes inappropriate content or even sensitive images backgrounds like graphic sexual images, inappropriate aggression or fighting. there is need for every individual to have some degree of responsibility in nurturing appropriate content for the children. this would allow children grow at their age when they would still be active in gaming and making fun (saint-jacques et al., 2006). an attempt to strengthen adaptability among the children with management ability can actually be applied by every individual in school and home environment. in order to achieve this designed approach of engagement with appropriate information content, the individuals must take responsibility for their own actions in nurturing their children (collis et al., 2013; strasburger et al., 2013). as a result, considering creative and fun approach should be related to positive attitudes and interests of children in an attempt to keep appropriate information content for children’s adaptability. in line with appropriate information content approach, this should widely strengthen the protective action against misuse of information. in particular, it refers to enhancement of the accuracy of information relevant to globally accepted standards of representation for children and the basis of preventive action to empower children to cope with the challenges (gentile, 2014). with regard to the extent of information inquiry from both speaking and listening skills, the quality of interpersonal relationships seems to rely on learning process international electronic journal of elementary education, 9(3), 693-708, march, 2017 702 to achieve high quality communication (saint-jacques et al., 2006). at this point of view, the demands of personal needs directly relate to getting appropriate information content to underlie the interaction pattern with continuous inquiry to verify and maintain the information content (westerman et al., 2014). this also applies to the children where they need personal satisfaction by getting access to recreational, gaming or cartoon images. in order to facilitate this approach, which is free from the risks, avoids key weaknesses and emphasises virtues to ensure information quality, is achieved widely amongst users and offers alternative and efficient opportunities for solving challenging issues (strasburger et al., 2010). the acquisition of knowledge on how to develop child adaptability needs to deal with establishing a productive interpersonal inquiry under parental guidance. to meet the demands and address the challenges of everyday life, this allows the children to enhance their communication skills in an efficient manner for positive adaptability behaviour. enhancing moral responsibility in media information for children moral responsibility here refers to the personal and social trait which is usually applied in the way to transmitting and delivering information (wakunuma & stahl, 2014). in order to achieve the responsibility performances, children protection can be engaged in the supervision which is taken in maintaining the continual basis integrated with a significant essence towards any situation which represents the inner state of the users. this involvement should concern the moral aspect in any particular state. in line with children, the parental involvement here needs to concern into the main persons responsible for their children’s caring (saint-jacques et al., 2006). in terms of maintenance, education, and supervision, the initiative here should intervene only in exceptional cases in which the child’s interests and rights require protection (tirri & husu, 2002). any situation which should be reported to represent a serious threat to the child’s security or development means the inclusion and-or active participation of mothers and fathers in activities. in this regard, parental involvement, amongst tasks, services and decision-making throughout the process of child protection needs the involvement which enables parents to interact with all those who provide support services to their child. apart from that, the different social work perspective with referring to the concept of parental involvement has to strengthen the process as an attempt to empower moral responsibility. towards the shortcomings, parent involvement is the springboard for a process of empowerment into the children protection by focusing on parents’ skills and strengths. in line with moral responsibility which parental concern is used to describe the interaction between themselves with their children, amongst both individuals and groups in producing the exchange ideas, the children may have opportunities to grow up with their surrounding with digital devices including interactive social media (collis et al., 2013; druin, 2009). producing and sharing any matter through digital devices indicated transforming the manner to enable children in trying to interact with their parents, peers, as well as how they make use of technology. the moral responsibility which children may show is the way for cooperating together with other peers in their group. towards the emotional response which is adaptively quick and reliable to individuals in any condition, preparing basic emotion such as empathy with others or cooperative assisting with others becomes a significant factor in drawing attention away from the experience of child protection (gentile, 2014). the increasing concern is about the restructuring of social work and child protection. emphasising rules and procedures are collaborated with inter-professional collaboration for information as a way of managing the most complex emotions (burnett & daniels, 2015). in particular, the risks of sharing the emphasis on procedures, audit, and new rules in a greater accountability require more active cognitive processing to develop moral emotions from other emotions. it is commonly known as one self-directed and three other-directed. in addition, the invaluable tool for enhancing the moral skills for the empowering children with adaptive technology skills / huda, hehsan, jasmi, mustari, shahrill, basiron, & gassama 703 children to get access online would lead to the moral responsibility in achieving information with assisting interactive social networking sites such as the social media which made a vital aspect of their life. further reflections careful engagement in digital information to promote child protection careful engagement has an outstanding value to drive users in a particular way in the digital information. it can make us aware that the moral issues have to be an ultimate attainment which should pay serious attention, especially amongst the youth and children. in particular, empowering the way to deal with children’s protection is significant to play a constructive role with perceiving problems arising from internet use (keith & martin, 2005). as a result, amongst guardians, parents and educators at this stage, the approach to empower careful engagement is mostly the one which has an effective way to deal with techniques. for instance, there are some approaches which can apparently endure legal boundaries to empower children and young people in the age with an appropriate pattern such as language and means approach to make them know the way to safeguard them in particular and their communities or friends in general (collis et al., 2013; edwards et al., 2016). in addition, empowerment to the careful engagement covers the approach in expectations of moral and cultural norms by elucidating openly with teaching them about the significant essence of digital information with making a fun approach (druin, 2009; hutinger & johanson, 2000). in working together amongst educator, parent, government and private sector, the extent of this empowerment would enable them to help to prepare a preventive action by recognising these boundaries and norms to the careful engagement into the child protection. in addition, both young people and children need to be well prepared with the preventive action to face the cybercrime by managing their use of digital information and device. here, the consistent control as a significant role to strengthen their confidence and expertise should be combined with careful engagement and appropriate anticipation towards the impact which may happen. as a result, for paying particular attention into this expectation, keeping aware of potential risks has to be integrated with the anticipation approach to effectively convey the information reliable as role models and sources of advice (rainey & goujon, 2011). with the emerging strategies in an effective way for evading the victims from offensive, the ability to make under control in managing the way of use of digital information will likely become an outstanding value to make sure the children in a safe condition. in order to take an active role, to keep children and young people from the sexually explicit materials, for instance, available online should be equally anticipated into the careful engagement (saint-jacques et al., 2006). by educating those in the way which can be communicated privately into the subsequent stages which parents are responsible for their children’s safety (gentile, 2014), paying serious attention to the children’s potential growth is an entirely developed plan with understanding the way of the cultural basis used in digital information. in particular, careful engagement here refers to the extent of keeping the rights to child protection. child protection as the rights to natural growth development child protection here refers to the way of protecting children from the impact of digital information. the number of issues in digital device and information such as cybercrime emerges to be seen such huge generational gaps in attitudes in terms of confidentiality, privacy and an individual’s rights which encourage parental involvement with the struggle to keep up (saint-jacques et al., 2006). the potential of addressing safety and privacy against cyberbullying and online predation points out the necessity to encourage parents to engage with their children and to discuss their online activities. in terms of psychological international electronic journal of elementary education, 9(3), 693-708, march, 2017 704 and technical aspect, the parents may find such factors equally important to empower children by equipping them with online literacy skills (koltay, 2011). with regard to the level of experience in getting involved amongst guardians, parents, and educators, to keep children out of harm’s way may include teaching about the introductory use of the digital device. as a result, empowerment of children against abuse and vulnerability will make positive benefit to access a filtered and purified internet. hence effective child protection can be achieved with a deep understanding of the requirement around rooms and houses to engage directly with children (gentile, 2014). here is the agenda which is firmly circumventing the filters deployed on the computer. in line with the proposed child protection regarded as the right to natural and positive growth of children who access internet with continual guidance of parents and instructors for the purpose of filtering and purifying content for safety of consumption the domain is intended to protect children to solve the underlying problems through a mediated discussion as they may encounter where this is to help educate children about how to behave online (strasburger et al., 2013). this enhancement of individual and collective creativity refers to the development and growth of ideas and endeavours (keith & martin, 2005). in particular, the children can be helped through expansion of one’s online connections shared, gaming sites, for instance, with more diverse backgrounds. it covers such communication in fostering the one’s individual identity with social skills to become an important step for children to let them grow naturally in the way which they can search for as many opportunities as possible through cooperative and creative ways. conclusion in the light of the forgoing discussion, careful engagement of media information with the expressed purpose of child protection and rights of accessing media is critical in this age of globalization. hence the proposed framework on how to transmit valuable information within an ethical context from empirical literature relating to the growth and development of the child is all the more necessary. the protection of children digital era is significant and would be difficult to achieve without parental or instructor’s guidance to promote positive impact of technology on their lives rather than negative one. this paper elaborated the current situation on the moral responsibility for the parental engagement on how to protect children from media influences by creating awareness and mediated supervision of media usage by parents and instructors. this paper provided a considerable guideline on child protection from the harmful impact of media influences to grow naturally with positive attitude in a good social environment. this is expected to contribute in disseminating the children’s rights to get useful insight from the media. the finding reveals that encompassing what children should do mainly on adapting their capacity to get knowledge and skill development towards the technology usage should be linked with providing a considerable guidance on taking care of the child protection aspects by forestalling its harmful impact. this can be enhanced as the children’s rights to enable them grow naturally with positive impact of media influences. in addition to the consideration of a distinct guideline to protect innocent childhood from something which harmful, this attempt would lead teachers to mediate between competing private and public interests within ethical conflicts. all practices concerning children’s attitude and behaviour can be considered as primary precursor to evaluate and balance all the elements necessary to make a decision on a specific situation for a specific individual child or group of children. in addition to the understanding on child adaptability to media technology usage, this approach will contribute in creating awareness of information and engagement of children to promote cooperative behaviour and reaction on transitions and changes. thus there is the need for monitoring both changes and transitions, helping children to find a way by giving accurate information to assist with making smooth transitions that allows them to know what will empowering children with adaptive technology skills / huda, hehsan, jasmi, mustari, shahrill, basiron, & gassama 705 happen next and what behaviour is expected. hence, this paper contributed to natural and positive growth of children by promoting ethical consideration of their protection and innocence through parental or instructor guidance for purpose of achieving appropriate media content for children. • • • references agichtein, e., castillo, c., donato, d., gionis, a., & mishne, g. (2008). finding high-quality content in social media. in proceedings of the 2008 international conference on web search and data mining (pp. 183-194). acm. anshari, m., almunawar, m. n., shahrill, m., wicaksono, d. k., & huda, m. (2017). smartphones usage in the classrooms: learning aid or interference?. education and information technologies, 1-17. buckingham, d. (2013a). beyond technology: children’s learning in the age of digital culture. cambridge: polity press. buckingham, d. (2013b). media education: literacy, learning and contemporary culture. london: john wiley & sons. burnett, c., & daniels, k. (2015). technology and literacy in the early years. in understanding digital technologies and young children: an international perspective. (eds) garvis, s. london and new york: routledge. campbell, m. a. (2005). cyber bullying: an old problem in a new guise? australian journal of guidance and counselling, 15(1), 68-76. campbell, p. h., milbourne, s., dugan, l. m., & wilcox, m. j. (2006). a review of evidence on practices for teaching young children to use assistive technology devices. topics in early childhood special education, 26(1), 3-13. cemalcilar, z., falbo, t., & stapleton, l. m. (2005). cyber communication: a new opportunity for international students’ adaptation?. international journal of intercultural relations, 29(1), 91110. chen, b., & bryer, t. (2012). investigating instructional strategies for using social media in formal and informal learning. the international review of research in open and distributed learning, 13(1), 87-104. chayko, m. (2014). techno‐social life: the internet, digital technology, and social connectedness. sociology compass, 8(7), 976-991. chen, n. s., & hwang, g. j. (2014). transforming the classrooms: innovative digital game-based learning designs and applications. educational technology, research and development, 62(2), 125. children and the internet (2016). http://www.internetsociety.org/sites/default/files/bpchildrenandtheinternet-20129017-en.pdf (accessed on november 1, 2016) christensen, l., & aldridge, j. (2012). critical pedagogy for early childhood and elementary educators. new york and london: springer science & business media. collis, b. a., knezek, g. a., lai, k. w., miyashita, k. t., & pelgrum, w. j. (2013). children and computers in school. new york and london: routledge. international electronic journal of elementary education, 9(3), 693-708, march, 2017 706 dabbagh, n., & kitsantas, a. (2012). personal learning environments, social media, and selfregulated learning: a natural formula for connecting formal and informal learning. the internet and higher education, 15(1), 3-8. dabbagh, n., benson, a. d., denham, a., joseph, r., al-freih, m., zgheib, g., ... & guo, z. (2016). gamebased learning. in learning technologies and globalization (pp. 31-35). new york and london: springer international publishing. davies, c., coleman, j., & livingstone, s. (eds.). (2015). digital technologies in the lives of young people. new york and london: routledge. dehue, f., bolman, c., & völlink, t. (2008). cyberbullying: youngsters’ experiences and parental perception. cyberpsychology & behavior, 11(2), 217-223. druin, a. (2009). mobile technology for children: designing for interaction and learning. burlington, usa: morgan kaufmann. erdur-baker, ö. (2010). cyberbullying and its correlation to traditional bullying, gender and frequent and risky usage of internet-mediated communication tools. new media & society, 12(1), 109125. edwards, s., nolan, a., henderson, m., skouteris, h., mantilla, a., lambert, p., & bird, j. (2016). developing a measure to understand young children’s internet cognition and cyber-safety awareness: a pilot test. early years, 36(3), 322-335. fenwick, t., & edwards, r. (2016). exploring the impact of digital technologies on professional responsibilities and education. european educational research journal, 15(1), 117-131. gackenbach, j. (ed.). (2011). psychology and the internet: intrapersonal, interpersonal, and transpersonal implications. burlington, usa: academic press. gentile, d. a. (2014). media violence and children: a complete guide for parents and professionals: a complete guide for parents and professionals. santa-barbara, california: preager, abc-clio. gillespie, a.a. (2008). child exploitation and communication technologies, rusell house publishing. gupta, s., & hee-woong, k. (2007). developing the commitment to virtual community: the balanced effects of cognition and affect. information resources management journal, 20(1), 28. holste, j. s., & fields, d. (2010). trust and tacit knowledge sharing and use. journal of knowledge management, 14(1), 128-140. huda, m., anshari, m., almunawar, m. n., shahrill, m., tan, a., jaidin, j. h., & masri, m. (2016). innovative teaching in higher education: the big data approach. the turkish online journal of educational technology, 15(special issue), 1210-1216. huda, m., maseleno, a., shahrill, m., jasmi, k. a., mustari, i., & and basiron, b. (2017). exploring adaptive teaching competencies in big data era. international journal of emerging technologies in learning (in press). hutinger, p. l., & johanson, j. (2000). implementing and maintaining an effective early childhood comprehensive technology system. topics in early childhood special education, 20(3), 159-173. jackson, l. a., von eye, a., fitzgerald, h. e., zhao, y., & witt, e. a. (2010). self-concept, self-esteem, gender, race and information technology use. computers in human behavior, 26(3), 323-328. kaplan, a. m., & haenlein, m. (2010). users of the world, unite! the challenges and opportunities of social media. business horizons, 53(1), 59-68. keith, s., & martin, m. e. (2005). cyber-bullying: creating a culture of respect in a cyber world. reclaiming children and youth, 13(4), 224. koltay, t. (2011). the media and the literacies: media literacy, information literacy, digital literacy. media, culture & society, 33(2), 211-221. luciana, r. p. (2010). one minute more: adolescent addiction for virtual world. procedia-social and behavioral sciences, 2(2), 3706-3710. empowering children with adaptive technology skills / huda, hehsan, jasmi, mustari, shahrill, basiron, & gassama 707 luo, q., zhang, s., zhao, j., & hu, m. (2010). a lightweight and portable approach to making concurrent failures reproducible. in international conference on fundamental approaches to software engineering (pp. 323-337). springer berlin heidelberg. mcloughlin, c., & lee, m. j. (2010). personalised and self-regulated learning in the web 2.0 era: international exemplars of innovative pedagogy using social software. australasian journal of educational technology, 26(1), 28-43. mcnaughton, d., & light, j. (2013). the ipad and mobile technology revolution: benefits and challenges for individuals who require augmentative and alternative communication. augmentative and alternative communication, 29(2), 107-116. mettler, e., massey, c. m., & kellman, p. j. (2011). improving adaptive learning technology through the use of response times. grantee submission. nonaka, i., & von krogh, g. (2009). perspective-tacit knowledge and knowledge conversion: controversy and advancement in organizational knowledge creation theory. organization science, 20(3), 635-652. o'keeffe, g. s., & clarke-pearson, k. (2011). the impact of social media on children, adolescents, and families. pediatrics, 127(4), 800-804. othman, r., shahrill, m., mundia, l., tan, a., & huda, m. (2016). investigating the relationship between the student’s ability and learning preferences: evidence from year 7 mathematics students. the new educational review, 44(2), 125-138. raju, j. (2014). knowledge and skills for the digital era academic library. the journal of academic librarianship, 40(2), 163-170. rainey, s., & goujon, p. (2011). toward a normative ethics for technology development. journal of information, communication and ethics in society, 9(3), 157-179. saint-jacques, m. c., drapeau, s., lessard, g., & beaudoin, a. (2006). parent involvement practices in child protection: a matter of know-how and attitude. child and adolescent social work journal, 23(2), 196-215. shenton, a. k., & fitzgibbons, m. (2010). making information literacy relevant. library review, 59(3), 165-174. shepherd, j. (2004). why the digital era?. social and economic transformation in the digital era, 1-18. pennsylvania, usa: igi global. stevens, r. h., millage, j., & clark, s. (2010). waves of knowledge management: the flow between explicit and tacit knowledge. american journal of economics and business administration, 2(1), 129. spence, e. h. (2011). information, knowledge and wisdom: groundwork for the normative evaluation of digital information and its relation to the good life. ethics and information technology, 13(3), 261-275. stefanone, m. a., lackaff, d., & rosen, d. (2010). the relationship between traditional mass media and “social media”: reality television as a model for social network site behavior. journal of broadcasting & electronic media, 54(3), 508-525. strasburger, v. c., hogan, m. j., mulligan, d. a., ameenuddin, n., christakis, d. a., cross, c., ... & moreno, m. a. (2013). children, adolescents, and the media. pediatrics, 132(5), 958-961. strasburger, v. c., jordan, a. b., & donnerstein, e. (2010). health effects of media on children and adolescents. pediatrics, 125(4), 756-767. tirri, k., & husu, j. (2002). care and responsibility in the best interest of the child: relational voices of ethical dilemmas in teaching. teachers and teaching: theory and practice, 8(1), 65-80. voogt, j., erstad, o., dede, c., & mishra, p. (2013). challenges to learning and schooling in the digital networked world of the 21st century. journal of computer assisted learning, 29(5), 403-413. international electronic journal of elementary education, 9(3), 693-708, march, 2017 708 wakunuma, k. j., & stahl, b. c. (2014). “tomorrow’s ethics and today’s response: an investigation into the ways information systems professionals perceive and address emerging ethical issues.” information systems frontiers, 16(3), 383-397. wall, d. s. (2007). cybercrime: the transformation of crime in the information age. cambridge, uk: polity press. westerman, d., spence, p. r., & van der heide, b. (2014). “social media as information source: recency of updates and credibility of information.” journal of computer-mediated communication, 19(2), 171-183. international electronic journal of elementary education, 9(3), 627-644, march 2017 issn:1307-9298 copyright © iejee www.iejee.com developing the irrational beliefs in mathematics scale (ibims): a validity and reliability study deniz kaya a  a ministry of national education, turkey received: 11 september 2016 / revised: 12 october 2016 / accepted: 10 january 2017 abstract the purpose of this study is developing a valid and reliable scale intended to determine the irrational beliefs of students in mathematics. the study was conducted with a study group consisting of 700 students in 2015-2016 academic year. expert opinions were received for the content and face validity of the scale, and the exploratory factor analysis (efa) and confirmatory factor analysis (cfa) were applied. after the efa was applied a structure was obtained consisting of 20 items, which explained 53.86% of the total variance and the four factors. the findings obtained from the cfa showed that the structure consisting of 20 items and the four factors related to the irrational beliefs in mathematics scale (ibims) had adequate consistency indices (x2/df=2.50, rmsea=.056, srmr=.056, gfi=.92, agfi=.90, cfi=.92, ifi=.90, pnfi=.76). the total internal consistency coefficient of the scale was calculated as .81, and the internal consistency coefficients of the items of finding reasons, perfection, being conditioned and inclinations for being accepted were calculated as .85, .78, .71 and .66 respectively. the test-retest measurement reliability was found to be .75. the discrimination of the items in the scale was made with the total corrected item correlation by comparing the 27% lower-upper group comparisons. keywords: developing scale, irrational belief, mathematics, reliability, validity introduction today, with the increasing importance of basic mathematical skills and competencies, many countries have felt the necessity of re-designing their educational policies, and have performed profound changes for this purpose. especially, the research results of the program for international student assessment [pisa], trends in international mathematics and science study, [timss], international association for the evaluation of educational achievement [iea], progress in international reading literacy study [pirls], national council of teachers of mathematics [nctm] and similar research institutions and programs that assess the knowledge and skills of students provide us with important clues for this purpose. in this context, the reports prepared by research institutions, and the widespread belief suggesting that the individuals and societies that can use mathematics in an efficient manner will have a voice in increasing the opportunities that will shape their futures made many educationalists to understand the factors that influence  address for correspondence: deniz kaya, ministry of national education, izmir, turkey. phone: +90 507 9465098, e-mail:denizkaya38@gmail.com http://www.iejee.com/ international electronic journal of elementary education, 9(3), 627-644, march, 2017 628 mathematical success better (nctm, 2000). no doubt, understanding the relations of these factors with each other in a better manner contributes greatly to the development of a desired and qualified mathematics teaching. however, the results of international tests show that the students of many countries are not at the desired level in terms of mathematical success (martin, mullis & kennedy, 2007; ministry of national education, [mne], 2014; mullis, martin, robitaille & foy, 2009; organization for economic cooperation and development, [oecd], 2014; yalcin & tavsancil, 2014). this situation has made the social, cognitive and affective characteristics that influence mathematical success become the focal point of many studies (bandura, 1997; bloom, 1998; bruner, 1977; campbell & ramey, 1994; de villiers, 1994; piaget, 2013; schunk & zimmerman, 1998; senemoglu, 2009; wynn, 1992). it has been expressed in hypothetical expressions that cognitive characteristics (reasoning, problem solving, perception, memory, attention, imitation, and creativity) are important factors influencing success (bandura, 1997; bloom, 1998; schunk & zimmerman, 1998). as a matter of fact, the results of many studies conducted on mathematics support these hypothetical opinions (arslan & altun, 2007; byron, 1995; cai, 2003; eurydice, 2012; higgins, 1997; isik & kar, 2011; ozsoy, 2005; schonfeld, 1992; van de walle, 2004). as it is already known, the majority of the behavioural characteristics that are supposed to be included in the educational needs of a student are defined as the cognitive characteristics (ozcelik, 1998). in this context, cognitive characteristics have had their place among the important study topics of the literature researchers. in recent years, we can conclude that many academic studies, the majority of which are conducted on psychoanalytic hypotheses, have focused in the cognitive structures of students (corey, 2005; çivitci, 2006; ellis & dryden, 1997; turkum, 1996; white, 2003; wong, 2008). the concept of irrational beliefs in which the cognitive structure and processes are determined with various assessment methods and which is one of the study areas of the rational emotive behaviour therapy (rebt) is one of these concepts. the idea of irrational beliefs is based on the rebt philosophy led by albert ellis in 1955. the most distinctive characteristics of this approach is that it associates the events, emotions, beliefs, evaluations and reactions of individuals with the influence of the psychological difficulties they undergo. in other words, ideas, emotions and behaviours influence each other at a major scale and act in a mutual cause and effect relationship. in this context, he defends the notion that humans are born with strong inclinations that are rational and irrational, beneficial and destructive (ellis, 1999). according to this understanding, the reason that influence the spiritual health of humans in a negative manner is not bad environmental conditions, but the individuals’ turning themselves into dysfunctional beings in emotional and behavioural terms. ellis (1993) stated that individuals felt disappointment and being prevented when they failed or when they were not approved, and increased their discomfort by misconceptions, deductions and interpretations in the direction of their irrational desires. in this context, the rebt, which is one of the cognitive behavioural therapy models, tried to explain its main idea on psychological problems with the concept of irrational beliefs. with the broadest meaning, irrational beliefs are defined as the cognition that lack empirical reality, which includes the expressions like “strict, dogmatic, unhealthy, and inharmonious”, and which prevent the behaviour of reaching life goals and include compulsion and desire, and are not considered to be correct in logical terms (can, 2009; dryden & neenan, 1996; ellis, 1999; ellis & dryden, 1997; ellis & harper, 1997; walen, diguiseppe & dryden, 1992). irrational beliefs generally develop when individuals convert the events and their desires about themselves into compulsory desires/demands (corey, 2005; nelson-jones, 1982). the typical characteristics of these beliefs is that although they are dysfunctional, and do not have a logical and empirical validity, they are accepted as if they were real, and have self developing the irrational beliefs in mathematics scale / kaya 629 defeating patterns (corey, 2005). for this reason, these thoughts trigger absolustic expectations about humans and events, awfulizing the negative results of an event in an excessive manner, and being vulnerable to any discomfort at a significant level (abrams & ellis, 1994; corey, 2005). when the literature is scanned, it is observed that there have been conducted studies reporting that there are positive and significant relations between irrational thinking and faulty thinking (webber & coleman, 1988), problem-solving skills (bilge & arslan, 2000), low self-respect (daly & burton, 1983), anxiety level (çivitci, 2006; lorcher, 2003), failure in classes (bozkurt, 1998; dilmac, aydogan, koruklu & deniz, 2009), depression (mclennan, 1987; nelson, 1977), stress (amutio & smith, 2008), anxiety to establish communication (altintas, 2006; ambler & elkins, 1985), anger (ford, 1991), cancelling academic work (bridges & roig, 1997), gender (bozkurt, 1998; yurtal-dinc, 1999), avoidant and postponing decision-making style (can, 2009), aggressiveness (kilicarslan, 2009), exam anxiety (boyacioglu, 2010; guler, 2012) and self-efficacy (alcay, 2015). for example, bilge and arslan (2000) conducted a study by using different variables and examined the relation between problem solving skills and irrational thinking on 767 students whose irrational thinking levels varied. at the end of the study, it was observed that as the income levels of the families of the university students and their perceived academic success levels increased, and as their satisfaction on the educational medium they were studying at increased, and as the irrational belief levels in the residential units decreased, this situation influenced the problem-solving skills of the students in a positive manner. çivitci (2006) conducted another study and examined the irrational belief levels of 405 students according to some socio-demographical characteristics. the findings of the study showed that the irrational belief levels of the students varied according to the educational status of their parents, perceived parent attitudes, perceived academic success and to the number of the siblings; however, it did not vary according to the grade, age, gender, employment status of the mother, and the structure of the family. bridges and roig (1997) examined the relation between irrational beliefs and delaying academic tasks in 195 university students. according to the study results, there is a significant relation between the “avoiding problems” sub-dimension, which is one of the sub-scales of the irrational beliefs, and the delaying academic tasks and duties variable. in another study conducted by altintas (2006) on 395 secondary education students, it was reported that there is a significant relation between the communication skills of the teenagers from high schools and their irrational beliefs. when the gender variable is considered, it was determined that the irrational belief levels of female students were significantly higher than those of male students. on the other hand, yurtal-dinc (1999) conducted a study on 560 university students and examined their irrational beliefs (the need for approval, high expectations, the inclination of blaming someone, emotional irresponsibility, excessive anxiety, being addict, helplessness, and perfectionism) according variables like parents’ attitudes (democratic, protective-demanding and authoritarian), gender, and the department they studied at. according to the data obtained in the study, the mean scores of the general irrational beliefs and high expectations sub-scale differed in favour of those with authoritative parent attitudes; and the helplessness sub-scale mean scores differed in favour of those with protective-demanding parent attitudes. the mean scores of the subscale of the inclination of blaming someone were observed to be higher in males than in females; and in the need for approval sub-scale, the mean scores of the students who were at the social sciences department, were found to be higher than those studying at science education departments. daly and burton (1983) conducted a study in which they examined the relation between irrational beliefs and self-respect and included 251 university students in their study. according to the data obtained, a negative and significant relation was found between the irrational beliefs and self-respect variables. in international electronic journal of elementary education, 9(3), 627-644, march, 2017 630 addition, it was reported that the irrational beliefs that predicted the low self-respect were the desire for being approved, high expectations, excessive anxiety and avoiding problems. the findings of the study conducted by boyacioglu (2010) on 557 students indicate that there are positive and significant relations between the illogical beliefs of the students and exam anxiety. in this context, it was determined in previous studies that as the illogical belief levels of the students increased, so did their exam anxiety levels. nelson (1977) examined the relation between irrational beliefs and depression in 156 university students. the correlation analyses revealed that depression had a significant relation with high expectations, excessive anxiety, helplessness and irrational beliefs, and there were low-level gender differences between the female and male students. ford (1991) conducted a study with 110 subjects and investigated the relation between anger and irrational beliefs. at the end of the study it was determined that there is a significant relation between constant anger, angry nature, perception of injustice, the provocation factor among individuals and the irrational beliefs. can (2009) conducted another study with 750 students and reported that there was a negative relation in the postponing, panic and avoidant sub-scales in the decision-making scale of the students whose irrational belief scale scores were low; and there was a positive relation between the self-respect sub-dimensions. amutio and smith (2008) conducted a study on 480 university students to determine the relation between the irrational beliefs and stress, and the results of this study revealed that there was a positive and significant relation between stress and irrational beliefs. when the studies in the literature are examined in general terms, it is observed that the irrational beliefs were examined by considering them together with many variables (grade, gender, attitudes, residential areas, monthly income levels, etc.). when the field of mathematics education is considered, it is observed that there are limited studies conducted on the irrational beliefs of students. in addition to this, there are no scales that are specific to the irrational beliefs in mathematics education/teaching field. in this context, it is expected that this scale will bring a new insight to the studies that will be conducted on mathematics education. as a matter of fact, mathematics classes are considered as being boring and abstract subjects by many students and are not loved much (aksu, 1985). on the other hand, it is also known that irrational beliefs have the quality of preventing individuals from reaching their goals and their happiness by influencing their emotions and thoughts in a negative manner. in this context, it is considered that examining many factor groups that influence the mathematical success of students together with the irrational beliefs in order to understand this issue better and to contribute to the solution of problems. method the model of the study the general scanning model was adopted in the study. the scanning models imply a research approach that aims to define an existing or past situation as is (karasar, 2005). the study was designed in the descriptive scanning model and was conducted in two steps. in the first step, the ibims was developed; and in the second step; the scale, which was developed, was applied to another group to obtain evidence on the functionality of the scale. the study group the study group consisted of 700 students who were studying at the 6th, 7th, and 8th grades of a state secondary school in the city centre of izmir in 2015-2016 academic year. 331 of the students were female (47.3%), and 369 were male (52.7%). in determining the number of the students that would constitute the study group, the criteria, which was developing the irrational beliefs in mathematics scale / kaya 631 recommended by tabachnick and fidell (2001) for factor analyses as 300 people “good”, 500 people “very good” and 1000 people “perfect” was applied. in addition to this, classes from various grades were also included in the study group to increase the representation power of the scale for similar groups and to obtain a wide variance in terms of age. table 1. the frequency table of the study group 6th grade 7th grade 8th grade total n % n % n % n % female 129 48.1 112 48.5 90 44.8 331 47.3 male 139 51.9 119 51.5 111 55.2 369 52.7 total 268 38.3 231 33 201 28.7 700 100 data collection tool the hypothetical data on the research/studies conducted on irrational beliefs within our country and abroad were examined with literature scanning method. as a result of this scan, it was determined that the studies conducted for the purpose of measuring the irrational beliefs of students in mathematics were inadequate, and there were no scales to measure the irrational beliefs of secondary school students. in this context, an item pool consisting of 33 expressions was formed by considering the rebt hypothetical structure suggested by albert ellis in mid-1950s, and the irrational beliefs scale, which was developed by jones (1969). the initial form, which consisted of 33 items, was presented to 5 experts (2 academicians, 1 mathematics teacher, and 2 psychology education) who had knowledge on this field and who were informed about the study to receive their viewpoints and to ensure the content and face validity. in order to receive the expert viewpoints, an assessment tool consisting of three items was used. in this assessment tool, the experts were asked to choose one of the options “suitable”, “must be corrected” and “not suitable”. by combining all the assessment tools as one assessment tool, the issue of how many experts approved each possible option of the items was determined. in this context, the content validity of the items was determined with the “(the number of the experts who answered positively/the number of total experts)-1” formula for each item (veneziano & hooper, 1997). after this calculation, four items whose content validity ratios were below 0.80 were excluded from the study. in addition to this, three items which were considered to have similar meanings, and another two items which were considered to cause misconceptions were determined and excluded from the scale. after the necessary changes were made in accordance with the expert viewpoints, a grammar teacher was consulted in order to ensure the understand ability of the scale in terms of language and typos. as a result, the draft scale, which had 24 items, was designed in a 5point structure, which consisted of statements “i definitely do not agree (1), i do not agree (2), i am indecisive (3), i agree (4) and i definitely agree (5)”. the possible highest score that could be received from the scale is 120, and the lowest score is 24. the scores’ being high shows that the irrational beliefs of the student are at higher levels, and the scores’ being low indicates that the irrational beliefs of the student are at the lower levels. as a last item, the draft form was applied as a pre-application to 30 students, who were selected randomly, studying at a state school in izmir in order to determine the item/items that were not understood and to detect the spelling mistakes and approximate response time. according to the data, it was determined that there was no misunderstandings and spelling mistakes in the draft form. the sixth grade students completed the scale in approximately 25 minutes. since the scale would be applied to upper grades (7th and 8th grades), this time was considered to be adequate. the draft form was applied in classroom medium after explaining the purpose of the study to the participants. international electronic journal of elementary education, 9(3), 627-644, march, 2017 632 the collection and analysis of the data in order test the validity and reliability of the ibims which was prepared as a draft form, it was pre-applied to 700 students, who were in the first study group by the authors of the study. the kaiser-meyer olkin (kmo) coefficient was applied to determine whether the sampling size was suitable for factorization or not, and the barlett test of sphericity was applied to determine whether or not the data were from multivariate normal distribution. the validity investigations of the scale were performed by examining the structural validity. for the structural validity, the factorial structure of the scale was determined by using the explanatory and confirmatory factor analyses. the efa is applied to determine the association between the unknown latent variables and the observed variables (çokluk, sekercioglu & buyukozturk, 2014). this analysis is defined as being explanatory or a discoverer for researchers who do not have any ideas on the issue of under which factor the items perform measurements in reality (byrne, 1994). as a matter of fact, it is expected in factor analysis, which is performed to locate the variable in the factor group in question, that the factor loads are high. when the literature is scanned it is observed that there is a widely-held belief that an item must have at least 0.30 minimum size for the factor load of the relevant item. according to tabachnick and fidell (2001), the load value of each variable must be evaluated at or over 0.32 as a basic rule. in addition to these, the explained total variance in single-factor designs being minimum 30% is considered to be adequate (buyukozturk, 2011), while it is expected to be over 41% in multi-factorial designs (kline, 2005). the cfa, on the other hand, is beneficial in efforts to develop, organize and review the measurement scales (floyd & widaman, 1995). according to kline (2005), in the cfa results of a measurement model, the correlation predictions among the factors, the loads under the factors to which the indicators are connected, and the amount of the measurement error for each indicator are given. cfa is the most influential analysis used to assess whether a pre-defined factor model fits the data (çokluk et al., 2014). many fit indices are used in order to determine the adequacy of the model tested in cfa (jöreskog & sörbom, 1993). in this study, the chi-square goodness test, goodness of fit index (gfi), adjusted goodness of fit index (agfi), comparative fit index (cfi), incremental fit index (ifi), parsimony normed fit index (pnfi), standardized root mean square residual (srmr), and root mean square error of approximation (rmsea) were examined for cfa. in these goodness indices, gfi, agfi, cfi, ifi and pnfi being >.90, rmsea and srmr being <.08 are considered as criteria, which is generally the situation (çokluk et al., 2014; tabachnick & fidell, 2001). the internal consistency (alpha) and test-retest reliability coefficients were examined for the reliability investigations of the ibims. for item analysis, the significance of the difference between the item average values of the upper 27% and lower 27% groups and the corrected item-total scores correlation were examined by using the t-test. the spss 22.0 and amos 24.0 programs were used for the validity and reliability analyses of the scale. findings in this part, the findings on validity and reliability tests, which were conducted to develop the ibims are provided. explanatory factor analysis firstly, the internal consistency of the scale was examined for the suitability of structural validity. as a result of the analysis, no items were detected with low item-total correlation in the 24-item scale. the kmo and barlett sphericity tests were applied to determine whether the 24-item scale fit the factor analysis or not. the kmo value, which is used to determine whether or not the data and the sampling size are adequate and suitable for the developing the irrational beliefs in mathematics scale / kaya 633 selected analysis, was found to be .87. in addition, the barlett sphericity test, which is used to check whether the data come from multi-variate normal distribution or not, was applied and the result was found to be significant (x2=4234.6, p<.01). it is necessary that the kmo measurement test result is .60 and over, and the result of the barlett sphericity test is statistically significant (jeong, 2004). since the values obtained as a result of the aforementioned analyses fit the basic hypotheses at a good level, it was decided that the factor analysis could be conducted (tavsancil, 2010). since the factor loads show the correlation between the item to be measured and the main structure, the relevant dimensions that appeared as a result of the basic component analysis and the factor loads were examined. since 3 items showed high load values in two or more factors, and because they did not fit the factor that was supposed to measure a certain attribute, they were excluded from the scale by receiving the expert opinion. in situations where the difference in cross items was below .20, which contributed greatly to the content validity of the sub-dimensions, the items that fit the sub-dimension was preferred (plotnikoff, 1994). another item was also excluded from the scale because it had low factor load (<.30). after these processes, the last form of the ibims was given as 20 items. the rotated components matrix, which was converted with varimax method, and which was obtained as a result of the factor analysis, is given in table 2, and the eigenvalue graphics is provided in figure 1. the varimax method, which is one of the vertical spinning methods, was preferred in order to ensure that the factor variances would have high value with a few variables. according to the analysis results, there are six items about the “inclination for finding reasons” factor of the irrational beliefs in mathematics, which are given in table 2. it is observed that the factor load values of the items vary between .568 and .840, and explain 22.18% of the total variance. in the second factor, there are six items of “inclination for perfection” factor of the irrational beliefs in mathematics, and the factor load values of the items vary between .541 and .734, and explain 18.03% of the total variance. in the third factor, there are five items of the “inclination on being conditioned” of the ibims and the factor load values of the items vary between .578 and .735, and explain 7.26% of the total variance. in the fourth factor, there are three items of “inclination for being accepted” factor of the ibims, and the factor load values of the items vary between .715 and .730, and explain 6.37% of the total variance. as a result, the total variance explained by these four factors is 53.86%. table 2. explanatory factor analysis results of the scale item common factor variance factor load values* factor 1 factor 2 factor 3 factor 4 7 .756 .840 5 .716 .821 6 .602 .764 2 .568 .738 1 .511 .691 8 .517 .568 .437 16 .579 .734 18 .488 .695 14 .501 .692 12 .517 .673 .207 17 .452 .645 15 .375 .541 .241 9 .639 .735 .279 11 .479 .250 .641 3 .452 .637 10 .450 .603 .220 international electronic journal of elementary education, 9(3), 627-644, march, 2017 634 table 2 (cont.). explanatory factor analysis results of the scale 4 .406 .578 23 .631 .264 .730 22 .552 .724 24 .582 .246 .715 eigenvalue (total=10.773) 4.437 3.607 1.454 1.275 explained variance (total=53.865) 22.184 18.037 7.269 6.375 *values below ±0.20 are not given. figure 1. scree plot confirmatory factor analysis the structure of the ibims, which consisted of 20 items and four factors, was tested by using the cfa. this analysis was made over 484 students, who were selected randomly from the sampling group (n=700) that were used in efa work. the findings obtained as a result of analyzing the model with cfa are given below. the chi-square/sd value (411.502/164=2.50) was found to be showing that the cfa results have a good fit [rmsea=.056, srmr=.056, gfi=.92, agfi=.90, cfi=.92, ifi=.90, pnfi=.76]. the standard values for the indices: the gfi and agfi values must be between 0 and 1. although there is no consensus on these values in the literature, if the values are over 0.85 and 0.90, this is the evidence of a good fit (kline, 2005; schumacker & lomax, 1996). the rmsea values also vary between 0 and 1. the more these values are close to 0, the more they indicate a fit. the x2/df ratio is a good fit indicator, and if it is below 2, this shows a perfect fit (jöreskog & sörbom, 1993; kline, 2005). as a result, all the standard fit indices show that the factor structure of the model is approved. table 3. the fit indices and standard fit criteria for the proposed model fit indices good fit acceptable fit scale values x2/df ≤3 ≤5 2.50 rmsea ≤.05 ≤.08 .056 srmr ≤.05 ≤.08 .056 gfi ≥.95 ≥.90 .92 agfi ≥.90 ≥.85 .90 cfi ≥.95 ≥.90 .92 ifi ≥.95 ≥.90 .90 pnfi ≥.95 ≥.50 .76 x2=411.512, sd=164, 90% probable confidence interval=[.049, .063] for rmsea the t-test values of the four-factor model obtained as a result of cfa are given in table 4. when the findings in table 4 are examined it is observed that the t-test values for inclination for finding reasons [f1] sub-scale vary between 13.08 and 23.79; for developing the irrational beliefs in mathematics scale / kaya 635 inclination for perfection [f2] sub-scale vary between 11.86 and 14.89; for the inclination for being conditioned [f3] sub-scale vary between 8.90 and 12.70; and for inclination for being accepted [f4] sub-scale vary between 13.31 and 19.00. the t values’ being over 1.96 shows that they are significant at .05 level; and being over 2.58 shows that they are significant at .01 level (jöreskog & sörbom, 1993; kline, 2005). in this context, it was determined that all the t values obtained in cfa were significant at .01 level. for this reason, the t values obtained in cfa confirm that the number of the participants is adequate for factor analysis, and reveal that there are no other items to be excluded from the model. table 4. the t-test values obtained from cfa for ibims f1 f2 f3 f4 item no t value item no t value item no t value item no t value 1 (7) 23.79* 7 (12) 14.89* 13 (3) 11.60* 18 (22) 14.31* 2 (5) 22.15* 8 (14) 14.19* 14 (4) 8.90* 19 (23) 19.00* 3 (6) 17. 47* 9 (15) 11.86* 15 (9) 12.70* 20 (24) 13.31* 4 (2) 15.53* 10 (16) 14.37* 16 (10) 11.54* 5 (1) 13.08* 11 (17) 13.55* 17 (11) 10.60* 6 (8) 14.33* 12 (18) 13.19* *p<.01 (the numbers in the brackets denote the item numbers in the draft scale) reliability study the reliability of the measurement values obtained with ibims was computed with the cronbach alpha reliability and test-retest reliability methods. the cronbach alpha reliability coefficients of the measurements was found to be .85 for inclination for finding reasons sub-scale; .78 for inclination for perfection sub-scale; .71 for the inclination for being conditioned sub-scale; .66 for inclination for being accepted sub-scale; and .81 for the whole of the scale. the test-retest reliability of the scale was tested on 56 students after three weeks, and was found as .75. when the fact that the measurements whose reliability coefficients are .70 and over are accepted as reliable is considered (fraenkel, wallen & hyun, 2012), it is possible to suggest that the reliability coefficients are adequate. the results on reliability analysis are given in table 5. table 5. reliability coefficient values of the scale sub-dimensions cronbach’s α finding reasons inclination .85 perfection inclination .78 the inclination for being conditioned .71 inclinations for being accepted .66 the whole of the scale .81 test-retest measurement .75 in order to determine the distinctiveness level of the items in ibims, and to see its prediction power for the total scores, the corrected item total correlation was computed, and 27% bottom-up group comparisons were applied. the results obtained in the item analysis are given in the table below. when the findings in table 6 are examined it is observed that the t values of the differences in item scores of the 27% bottom-up groups varies between 5.78 and 15.94 for all items (p<.01). in addition to this, when the results of the item total correlation are examined, it is observed that the items in the scale have values between .36 and .78. the t value of the differences between the bottom-up groups being significant is considered as a proof for the distinctiveness of the item (erkus, 2012). based on these findings, it is possible to suggest that all the items in the scale are distinctive. international electronic journal of elementary education, 9(3), 627-644, march, 2017 636 table 6. the corrected item-total correlations of the scale, and the t values on 27% lowerupper group difference item no item total correlation t item no item total correlation t 1 .46 13.868* 11 .49 11.359* 2 .45 14.346* 12 .36 5.789* 3 .42 14.236* 13 .53 14.774* 4 .38 10.898* 14 .50 12.595* 5 .40 11.501* 15 .52 12.177* 6 .50 13.151* 16 .57 15.940* 7 .43 8.918* 17 .50 12.559* 8 .42 8.698* 18 .62 10.438* 9 .50 11.636* 19 .78 14.538* 10 .38 7.395* 20 .65 11.387* *p<.01 criterion validity in order to ensure the criterion validity of the scale, the application was made with 116 female (48.74%) and 122 male (51.24%) students, 238 students in total. 95 (39.1%) of the 238 students who participated in the study were 6th graders; 65 (27.3%) were 7th graders; and 78 (33.6%) were 8th graders. the scores of the students received from the ibims were examined firstly according to the gender and the t-test results for the validity study of the scale. the results are given in the table below. table 7. the t-test results according to gender variable group n mean std. deviation df t p female 116 2.669 .514 236 -2.591 .010 male 122 2.846 .534 when table 7 is examined, it is observed that the scores received by the students from ibims vary according to gender (t(236)=-2.591; p<.05). according to this result, the mean score received by male students (m=2.84) is more than that of the female students (m=2.66), and therefore, we can conclude that the irrational beliefs in mathematics are more. the relation between the irrational beliefs of the students on mathematics and their academic grades were examined for another scale validity examination. the academic success criteria of the students consist of the average grades (all subjects at school) in the report card in the semester before the study was conducted. in this context, three terms were evaluated for the 6th grade, five terms were evaluated for the 7th grade, and seven terms were evaluated for the 8th grade. the measurements were made according to the 5point system. the average of the academic success levels of the students was 3.24; the standard deviation was 1.22; the mode was 4 and the median was 3. the issue of whether or not the irrational beliefs in mathematics of the students varied according to academic grades of the students was analysed with the anova test for repetitive measurements. on the other hand, in order to determine the source of the possible differences that might appear between the variables that showed normal distribution, the post hoc “bonferroni test”, which is one of the multiple comparison tests, was used. the results obtained are given in the table below. developing the irrational beliefs in mathematics scale / kaya 637 table 8. the anova results according to the academic grades on ibims variance source sum of squares df mean square f p the source of the significant difference between groups 3.286 4 .822 3.006 .019* 1-5** within groups 63.681 233 .273 total 66.967 237 *significant at p< .05 level. **the measurements in which differences were detected in bonferroni test. when table 8 is examined it is observed that there are statistically significant differences between the mean scores of the students who had different academic grades on irrational beliefs in mathematics (f(4-233)=3.006; p<.05). the source of this significant difference was concluded to be between 1-5 academic grades. in this context, we can suggest that as the academic grades of the students increase, they have less irrational beliefs in mathematics. evaluation of the scores received from ibims there are 20 items in ibims. a 5-point likert scale was used in the scale which consisted of statements “i definitely do not agree (1)” and “i definitely agree” (5). the scale has a 4dimensional structure consisting of 6 items in the inclination for finding reasons and inclination for perfection dimension; 5 items in inclination for being conditioned dimension; and 3 items in the inclination for being accepted. for this reason, the scores that may be received from inclination for finding reasons and inclination for perfection dimensions vary between 6 and 30; the scores that may be received from inclination for being conditioned dimension vary between 5 and 25; and the scores that may be received from inclination for being accepted dimension vary between 3 and 15 (appendix-1). when the scores received from ibims were being assessed, the scores received from the sub-scales were used for the processes. the scores received from the sub-dimensions of ibims being high shows that students have high-level perceptions in that dimension. results in this study, the aim was to develop a measurement scale that would allow obtaining valid and reliable results in irrational beliefs in mathematics of the students. when the ibims was being developed, the rebt hypothetical structure developed by albert ellis (1955) and the irrational beliefs scale developed by jones (1969) were taken into consideration. in addition, expert opinions were received in order to ensure the content and face validity of the scale. in the light of the expert viewpoints, 9 items were excluded from the item pool that was initially formed. in this way, a draft form consisting of 24 items were obtained. the items in the scale were applied to the students in the study group with the 5-point likert scale as i definitely agree (5) → i definitely do not agree (1). the first scale form, which had twenty-four items, was applied to 700 secondary school students consisting of students from sixth, seventh and eighth grades. since three items showed high load values in two or more factors, and since one item showed low factor load value (<.30), they were excluded from the scale. the structural validity of the scale was analyzed with efa and cfa. four items were removed from the scale after efa and a structure consisting of four factors were obtained. the factor load values of the items varied between .54 and .84, and explained 53.86% of the total variance. on the other hand, the total eigenvalue of the scale was found to be 10.77. generally, it is recommended that .50 value is taken as the criterion about the common variance (thompson, 2004). in this context, it is possible to claim that the total common variance of the scale is at a good level. the factors were named as inclination for finding reasons, inclination for perfection, inclination for being conditioned and inclination for being accepted. the total internal consistency coefficient of the scale was found as .81; and the internal consistency coefficients of the inclination for finding reasons, inclination for perfection, inclination for international electronic journal of elementary education, 9(3), 627-644, march, 2017 638 being conditioned and inclination for being accepted were found as .85, .78, .71 and .66 respectively. the test-retest measurement reliability was found as .75. when the fact that the measurement whose internal consistency coefficient is .70 and over are accepted as being reliable is considered (fraenkel, wallen & hyun, 2012), it is possible to claim that the reliability coefficients are at a good level. the factors that were obtained at efa were tested with cfa. the fit values were computed as x2/df=2.50, rmsea=.056, srmr=.056, gfi=.92, agfi=.90, cfi=.92, ifi=.90, pnfi=.76. according to this result, the agfi value has a good fit value, and the rmse, srmr, gfi, cfi, ifi and pnfi values have acceptable good fit values. when the fact that the fit indices computed in cfa are in acceptable limits is considered, it is possible to claim that the structural validity of the measurements obtained from ibims has been achieved. on the other hand, it was determined that the t-test values of the model with four factors obtained as a result of cfa varied between 8.90 and 23.79. the t values being higher than 2.58 shows that it is significant at .01 level (jöreskog & sörbom, 1993; kline, 2005). in this context, all the t values obtained in cfa were found to be significant at .01 level. as a conclusion, the t values obtained in cfa confirmed that the number of the participants in the study was adequate for factor analysis, and revealed that there were no items that needed to be eliminated from the model. the item analysis was performed in order to determine the prediction power of the items for the total score and to determine the distinctiveness levels. in item analysis, the corrected item total correlation was examined, and 27% bottom-up group comparisons were made. after the analysis, it was determined that the corrected item total correlations varied between .38 and .50 for inclination for finding reasons sub-scale; between .36 and .50 for inclination for perfection sub-scale; between .50 and .57 for inclination for being conditioned sub-scale; and between .62 and .78 for inclination for being accepted subscale; and that the t values of the differences between the 27% bottom-up groups was significant for all items included in the scale. these findings indicate that all of the items included in ibims are distinctive. an application was performed with 238 secondary school students who were studying at sixth, seventh, and eighth grades in order to ensure the scale validity of the measurement tool. firstly, the ibims was tested according to the gender variable, and was examined according to the t-test result. according to the findings, the scores received by the students in ibims showed variations according to gender variable (t(236)=-2.591; p<.05). since the mean score obtained by male students (m=2.84) is more than that of the female students (m=2.66), it was concluded that the irrational beliefs in mathematics were more. another scale validity test was performed according to the academic grades of the students. whether the irrational beliefs in mathematics of the students varied according to academic grades or not was analysed with anova. the bonferroni test, which is one of the multiple comparison tests, was applied to the dataset in order to determine the source of possible differences that might appear among the academic grade variables. according to the results, it was concluded that the source of the significant difference stemmed from the academic grades between 1 and 5. in this context, it was also concluded that as the academic success levels increased, the logical beliefs of the students in mathematics decreased (f(4-233)=3.006; p<.05). as a result, it was concluded that the scale, which was developed in the scope of the study, is a scale that produces valid and reliable results, and may be used in determining the irrational beliefs of students in mathematics. recommendations and the limitations of the study when the relevant literature is examined it has been observed that there are no measurement tools in international studies to determine the irrational beliefs of students in mathematics and to obtain conclusions in the light of different variables. it is considered developing the irrational beliefs in mathematics scale / kaya 639 that ibims, which has been developed in this study, will fill the gap in this field in the literature. for this reason, the strongest aspect of this study is that it will ensure that the consideration of irrational beliefs is included in the field of mathematics education. another strong side of the measurement tool is that it provides more than one single proof for the distinctive, structural validity and reliability of the items of the scale. in addition to this, with the help of the scale, it is expected that the concept of irrational beliefs, which is a psychoanalytic approach, will provide the opportunity to know students better in a wide range by handling the mathematics education in this context. by doing so, the cognitive structures that are not accepted to be true in terms of logic, including inclination for finding reasons, inclination for perfection, inclination for being conditioned and inclination for being accepted, developed by students in mathematics will be investigated in a detailed manner. as a matter of fact, the concept of irrational beliefs, which is used frequently in today’s world in psychology education, is handled with some parameters like the level of anxiety (çivitci, 2006; lorcher, 2003), gender (bozkurt, 1998; yurtal-dinc, 1999), anger (ford, 1991), and exam anxiety (boyacioglu, 2010; guler, 2012). however, the notion of irregular beliefs is spread to a wider area that cannot be limited with psychology education. for this reason, one of the greatest contributions of the study, which was conducted on mathematics teaching, to the literature is to provide the instructors with a different practice field. in addition to this, the study was conducted with the students from secondary school level, and this will facilitate the conduction of future similar studies at different educational levels. especially the irrational beliefs of high school and university students developed in mathematics may be investigated and the factor groups that influence the mathematical success may be examined. on the other hand, the irrational beliefs of students in mathematics may be investigated with new studies in terms of gender and grade level as well as in terms of some other variables (educational medium, the success in classes, student-teacher communication, anxiety, school management, income levels, etc.) which may be influential in the beliefs in the classes. it is expected that the scale, which has been developed in the scope of this study, may be used in studies that investigate the factors influencing school success together with sub-dimensions. the study also has some limitations as well as its strong sides mentioned above. the first limitation of the study is the issue of whether the structure obtained with the efa was confirmed or not was examined by conducting the cfa over the same dataset. in this context, the cfa must be tested again over different datasets, and it must not be underestimated that additional proof must be obtained for the confirmation of the structure obtained. studies that will be conducted with multiple method matrix may provide stronger proofs on the validity of the scale. another limitation is the fact that one single educational institution was used in the process of developing the measurement tool. in this context, different educational institutions must be included in future studies, and this will contribute to the structural validity of the scale. • • • references abrams, m., & ellis, a. (1994). rational emotive behaviour therapy in the treatment of stress. journal of guidance and counselling, 22(1), 117-123. aksu, m. (1985). problems in mathematics education and secondary education institutions. ankara: t.e.d. publishing. international electronic journal of elementary education, 9(3), 627-644, march, 2017 640 alcay, a. (2015). examining the relationship between self-efficacy and irrational beliefs of adolescents. unpublished master dissertation, education science institute, dokuz eylül university, izmirturkey. altintas, g. (2006). investigating the relationship of communication skills and irrational beliefs of high school adolescents with respect to some independent variables. unpublished master dissertation, education science institute, gazi university, ankara-turkey. ambler, b., & elkins, m. (1985). an examination of relationship between irrational beliefs and communication apprehension. paper presented at the 71st annual meeting of the speech communication association (denver, co, november 7–10). retrieved from http://files.eric.ed.gov-/fulltext/ed278081.pdf. amutio, a., & smith, j. c. (2008). stress and irrational beliefs in college students. ansiedad y estrés, 14, 211-220. arslan, ç., & altun, m. (2007). learning to solve non-routine mathematical problems. elementary education online, 6(1), 50-61. bandura, a. (1997). self-efficacy: the exercise of control. new york: freeman. bloom, b. s. (1998). human nature and learning in school (trans. d. a. ozcelik) (3th ed.). istanbul: ministry of national education publishing. bilge, f., & arslan, a. (2000). problem solving skills of university students whose irrational thoughts are different. turkish psychological counselling and guidance journal, 2(13), 7-18. boyacioglu, n. e. (2010). irrational beliefs and test anxiety among early adolescents. unpublished master dissertation, health science institute, istanbul university, izmir-turkey. bozkurt, n. (1998). an investigation of automatic thoughts and depression related with school achievements in high school students. unpublished doctoral dissertation, social science institute, dokuz eylül university, izmir-turkey. bridges k. r., & roig, m. (1997). academic procrastination and irrational thinking: a re-examination with context controlled. personality and individual differences, 22(6), 941-944. bruner, j. s. (1977). process of education. cambridge, ma: harvard university press. buyukozturk, ş. (2011). data analysis handbook for social sciences (15th ed.). ankara: pegem academy. byrne, b. m. (1994). structural equation modelling with eqs and eqs/windows: basic concepts, applications, and programming. california: sage publishing. byron, m. k. (1995). confronting the verbal/visual issue: the mathematical problem-solving processes of a small group of female junior secondary students. unpublished doctoral dissertation, the ohio state university, ohio-usa. cai, j. (2003). singaporean students’ mathematical thinking in problem solving and problem posing: an exploratory study. international journal of mathematical education in science and technology, 34(5), 719-737. campbell, f. a., & ramey, c. t. (1994). effects of early intervention on intellectual and academic achievement: a follow-up study of children from low-income families. child development, 65(2), 684-698. can, ö. (2009). examination of the irrational beliefs of university students and decision making style. unpublished master dissertation, social science institute, selçuk university, konya-turkey. corey, g. (2005). psychological counselling psychotherapy theory and applications (trans. t. ergene). ankara: mentis publishing. çivitci, a. (2006). irrational beliefs in adolescents: an investigation according to socio-demographic variables. pamukkale university journal of education, 19, 7-17. developing the irrational beliefs in mathematics scale / kaya 641 çokluk, ö., sekercioglu, g., & buyukozturk, ş. (2014). multivariate statistics for the social sciences: spss and lisrel applications (3th. ed.). ankara: pegem academy. daly, m. j., & burton, r. l. (1983). self-esteem and irrational beliefs: an exploratory investigation with implications for counselling. journal of counselling psychology, 30, 361–366. de villiers, m. (1994). the role and function of a hierarchical classification of quadrilaterals. for the learning of mathematics, 14(1), 11-18. dilmac, b., aydogan, d., koruklu, n., & deniz, m. e. (2009). the explication of primary school students’ perfectionist properties with irrational beliefs. elementary education online, 8(3), 720728. dryden, w., & neenan, m. (1996). dictionary of rational emotive behaviour therapy. london: whurr publishing. ellis, a. (1955). new approaches to psychotherapy techniques. journal of clinical psychology monograph supplement, 11, 1-53. ellis, a. (1993). reflection on rational-emotive therapy. journal of consulting and clinical psychology, 61(2), 199-201. ellis, a. (1999). early theories and practices of rational emotive behaviour therapy and how they have been augmented and revised during the last three decades. journal of rational-emotive and cognitive-behaviour therapy, 17(2), 69-93. ellis, a., & dryden, w. (1997). the practice of rational-emotive behaviour therapy. new york: springer publishing. ellis, a., & harper, r. a. (1997). a guide to rational living. north hollywood. ca: melvin powers books. erkuş, a. (2012). measurement and scale development in psychology. ankara: pegema publishing. eurydice (2012). the european higher education area in 2012: bologna process implementation report. brussels: eacea p9 eurydice. retrieved from http://sgb.meb.gov.tr/eurydice/. floyd, f. j., & widaman, k. f. (1995). factor analysis in the development and refinement of clinical assessment instruments. psychological assessment, 7(3), 286-299. ford, b. d. (1991). anger and irrational beliefs in violent inmates. personality and individual differences, 12(3), 211-215. fraenkel, j. r., wallen, n. e., & hyun, h. h. (2012). how to design and evaluate research in education (8th ed.). new york: mcgram-hill companies. güler, d. (2012). relationship of test anxiety with irrational beliefs and parental attitudes in 12th year high school students. unpublished master dissertation, social science institute, akdeniz university, antalya-turkey. higgins, k. m. (1997). the effect of long instruction in mathematical problem solving on middle school students’ attitudes, beliefs and abilities. journal of experimental education, 66(1), 5-24. isik, c., & kar, t. (2012). the analysis of the problems posed by the pre-service teachers about equations. australian journal of teacher education, 37(9), 93-113. jeong, j. (2004). analysis of the factors and the roles of hrd in organizational learning styles as identified by key informants at selected corporations in the republic of korea. unpublished doctoral dissertation, texas a & m university, usa. jones, r. g. (1969). a factored measure of ellis's irrational belief system with personality and maladjustment correlates. unpublished doctoral dissertation, texas technological college, texas-usa. jöreskog, k. g., & sörbom, d. (1993). lisrel 8: structural equation modeling with the simplis command language. lincolnwood: scientific software international, inc. international electronic journal of elementary education, 9(3), 627-644, march, 2017 642 karasar, n. (2005). scientific research methods. ankara: nobel publishing. kilicarslan, s. (2009). the relationships between irrational beliefs and aggressiveness in adolescent. unpublished master dissertation, social science institute, çukurova university, adana-turkey. kline, r. b. (2005). principles and practice of structural equation modelling (2nd ed.). new york: guilford publications, inc. lorcher, p. s. (2003). worry and irrational beliefs: a preliminary investigation. individual differences research, 1(1), 73-76. martin, m. o., mullis, i. v. s., & kennedy, a. m. (2007). pirls 2006 technical report. chestnut hill, ma: timss & pirls international study center, boston college. mclennan, j. p. (1987). irrational beliefs in relation to self-esteem and depression. journal of clinical psychology, 43, 89-91. ministry of national education [mne] (2014). timms 2011 national report of mathematics and science eight classes. retrieved from http://timss.meb.gov.tr/?page_id=25. mullis, i. v. s., martin, m. o., robitaille, d. f., & foy, p. (2009). timss advanced 2008 international report: findings from iea’s study of achievement in advanced mathematics and physics in the final year of secondary school. timss & pirls international study center, lynch school of education, boston college. national council of teachers of mathematics [nctm] (2000). principles and standards for school mathematics. reston, va: nctm. nelson-jones, r. (1982). the theory and practice of counselling psychology. london: holt, rinehart and winston. nelson, r. e. (1977). irrational beliefs in depression. journal of consulting and clinical psychology, 45(6), 1190-1191. organisation for economic co-operation and development [oecd] (2014). pisa 2012 results: what students know and can do-student performance in mathematics, reading and science (vol. i, rev. ed.). paris: oecd publishing. ozcelik, d. a. (1998). measurement and evaluation. ankara: osym publishing. ozsoy, g. (2005). the relationship between problem solving skills and mathematical achievement. gazi university journal of education, 25(3), 179-190. piaget, j. (2013). the construction of reality in the child (vol. 82). routledge. plotnikoff, r. c. (1994). an application of protection motivation theory to coronary heart disease risk factor behaviour in three australian samples: community adults, cardiac patients, and school children. unpublished doctoral dissertation, the university of newcastle, australia. schoenfeld, a. h. (1992). learning to think mathematically: problem solving, metacognition, and sense-making in mathematics. in d. grouws (ed.), handbook for research on mathematics teaching and learning (pp. 334-370). new york: macmilla. schumacker, r. e., & lomax, r. g. (1996). a beginner's guide to structural equation modelling. mahwah, nj: lawrence erlbaum associates. schunk, d. h., & zimmerman, b. j. (eds.). (1998). self-regulated learning: from teaching to selfreflective practice. new york: guilford. senemoglu, n. (2009). development, learning and teaching from theory to practice. ankara: pegem academy. tabachnick, b. g., & fidell, l. s. (2001). using multivariate statistics (4th ed.). new york: harper collins publishing. tavsancil, e. (2010). measurement of attitudes and data analysis with spss (4th ed.). ankara: nobel publishing. developing the irrational beliefs in mathematics scale / kaya 643 thompson, b. (2004). exploratory and confirmatory factor analysis: understanding concepts and applications. washington dc: american psychological association. turkum, s. a. (1996). cognitive behavioural therapy-based group effect on cognitive distortions and communication skills of counsellors. unpublished doctoral dissertation, social science institute, ankara university, ankara-turkey. van de walle, j. (2004). elementary and middle school mathematics: teaching developmentally. new york: pearson education, inc. veneziano l., & hooper j. (1997). a method for quantifying content validity of health-related questionnaires. american journal of health behaviour, 21(1), 67-70. walen, s. r., digiuseppe, r., & dryden, w. (1992). a practitioner’s guide to rational emotive therapy (2nd ed.). new york: oxford university press. webber, j., & coleman, m. (1988). using rational-emotive therapy to prevent classroom problems. teaching exceptional children, 21(1), 32-35. white, a. h. (2003). cognitive behavioural therapy in children with autistic spectrum disorder. steer, 4(5). retrieved from www.signpoststeer.org. wong, d. f. k. (2008). cognitive behavioural treatment groups for people with chronic depression in hong kong: a randomized wait-list control design. depression and anxiety, 25(2), 142-148. wynn, k. (1992). addition and subtraction by human infants. letters to nature, 358, 749-750. yalcin, s., & tavsancil, e. (2014). the comparison of turkish students' pisa achievement levels by year via data envelopment analysis. educational sciences: theory and practice, 14(3), 961-968. yurtal-dinc, f. (1999). an investigation of the irrational beliefs of university students according to some variables. unpublished doctoral dissertation, education science institute, gazi university, ankara-turkey. international electronic journal of elementary education, 9(3), 627-644, march, 2017 644 appendix irrational beliefs in mathematics scale (ibims) no items levels 1 i hate mathematics because it is a complex subject. 1 2 3 4 5 2 i hate mathematics because it is a difficult subject. 1 2 3 4 5 3 mathematics always makes me anxious. 1 2 3 4 5 4 the homework given by mathematics teachers always makes students feel exhausted. 1 2 3 4 5 5 the most difficult things in life are related to mathematics. 1 2 3 4 5 6 i will never be able to learn mathematics. 1 2 3 4 5 7 i must have perfect mathematics knowledge. 1 2 3 4 5 8 i must succeed in mathematics if i want to have a good profession in the future. 1 2 3 4 5 9 each statement of a mathematics teacher must be definitely true. 1 2 3 4 5 10 everything i do in mathematics classes is important for me to be successful. 1 2 3 4 5 11 i must not make mistakes if i want to be successful in mathematics. 1 2 3 4 5 12 mathematics requires seriousness. 1 2 3 4 5 13 if i am not successful in mathematics, my value will decrease in the eye of the teachers of other subjects. 1 2 3 4 5 14 there are no compensations if i make mistakes in mathematics. 1 2 3 4 5 15 i participate in mathematics classes to make my friends like me more. 1 2 3 4 5 16 when the mathematics teacher does not love me, i am nothing. 1 2 3 4 5 17 all students have to be successful in mathematics. 1 2 3 4 5 18 my family seeing that i am successful in mathematics is very important for me. 1 2 3 4 5 19 everybody must see my efforts in mathematics classes. 1 2 3 4 5 20 my efforts in mathematics classes must always be appreciated. 1 2 3 4 5 1. dimension [inclination for finding reasons]: 1-2-3-4-5-6 2. dimension [inclination for perfection]: 7-8-9-10-11-12 3. dimension [inclination for being conditioned]: 13-14-15-16-17 4. dimension [inclination for being accepted]: 18-19-20 turkish version: matematiğe yönelik akılcı olmayan i̇nançlar ölçeği (myaoi̇ö) no maddeler dereceler 1 matematik karmaşık bir ders olduğu için nefret ediyorum. 1 2 3 4 5 2 matematik zor bir ders olduğu için nefret ederim. 1 2 3 4 5 3 matematik dersi beni her zaman endişelendirir. 1 2 3 4 5 4 matematik öğretmenlerinin verdiği ödevler öğrencileri canından bezdirir. 1 2 3 4 5 5 hayatta en zor şey matematik ile ilgili uğraşılardır. 1 2 3 4 5 6 matematiği hiçbir zaman öğrenemeyeceğim. 1 2 3 4 5 7 her zaman mükemmel bir matematik bilgim olmalıdır. 1 2 3 4 5 8 gelecekte iyi bir meslek sahibi olmak istiyorsam matematikte başarılı olmak zorundayım. 1 2 3 4 5 9 matematik öğretmeninin kullandığı her ifade mutlaka doğru olmalıdır. 1 2 3 4 5 10 matematik derslerinde yaptığım her şey başarılı olmam için çok önemlidir. 1 2 3 4 5 11 matematikte başarılı olmak istiyorsam hata yapmamalıyım. 1 2 3 4 5 12 matematik ciddiyet gerektirir. 1 2 3 4 5 13 matematikte başarılı olamazsam diğer ders öğretmenlerinin gözündeki değerim düşer. 1 2 3 4 5 14 matematikte hata yaparsam bunun telafisi yoktur. 1 2 3 4 5 15 matematik derslerine arkadaşlarımın beni daha çok sevmesi için katılırım. 1 2 3 4 5 16 matematik öğretmeni beni sevmediği zaman ben bir hiçim. 1 2 3 4 5 17 tüm öğrenciler matematikte başarılı olmak zorundadır. 1 2 3 4 5 18 ailemin matematik derslerinde başarılı olduğumu görmesi benim için önemlidir 1 2 3 4 5 19 matematik derslerindeki gayretimi herkes görmelidir. 1 2 3 4 5 20 matematik derslerindeki çabalarım her zaman takdir edilmelidir. 1 2 3 4 5 1. boyut [neden bulma eğilimi]: 1-2-3-4-5-6 2. boyut [kusursuzluk eğilimi]: 7-8-9-10-11-12 3. boyut [şartlanma eğilimi]: 13-14-15-16-17 4. boyut [kabul görme eğilimi]: 18-19-20 international electronic journal of elementary education, 2016, 8(3), 403-424 sensitivity of students to the natural environment, animals, social problems and cultural heritage nuray kurtdede fi̇dan  afyon kocatepe university, turkey received: 25 december 2015 / revised: 7 february 2016 / accepted: 12 february 2016 abstract the study aims to determine the sensitivity levels of fourth-grade students to the natural environment, animals, social concerns and cultural heritage. besides, it has been investigated whether some personal characteristics of the students have differentiating effect on the views related to the sensitivity to the natural environment, animals, social concerns and cultural heritage. the participants of the study were a total of 447 fourth-grade students attending fifteen different public schools in afyonkarahisar province in the school year of 2014-2015. the data of the study were collected through the administration of the sensitivity value scale developed by the author of the current the study. the scale consisted of four dimensions and included fifty-eight items. in regard to content and face validity, the scale was reviewed by the field specialists. for construct validity first and second order confirmatory factor analysis was employed. in addition, the cronbach alpha coefficient was found for the reliability of the scale. the findings of the study showed that the participants had sensitivity to the natural environment, to animals, to social concerns and to cultural heritage. it was also found that the gender of the students, residence, the educational background and occupation of parents and the frequency of follow up news had statistically significant effects on the sensitivity levels of the participants. keywords: sensitivity value, confirmatory factor analysis, primary students, social studies. introduction doğanay (2006) argued that the course of social studies makes use of the content and methods of other disciplines about society and people to deal with the interaction of people with their physical and social environment in an interdisciplinary way and to produce individuals who are equipped with basic democratic values. one of the major goals of this course is to produce active citizens who can make informed decisions and solve problems in a changing world (öztürk, 2009). social studies is one of the main courses of the elementary and middle school curriculum in turkey. social studies took educators attention because it prepares students as active citizens (kılınç, 2014). active citizens are aware of the problems in society and attempt to eliminate these problems. they are also aware of their rights and responsibilities. they are expected to know and  nuray kurtdede fidan, afyon kocatepe university, faculty of education, afyonkarahisar/ turkey. phone: +90 272 2281418/334, e-mail: nurayfidan1@gmail.com international electronic journal of elementary education vol.8, issue 3, 403-424, 2016 404 make use of their rights, to fulfil their responsibilities and to involve societal activities (kuş, 2013). karatekin & sönmez (2014) argued that active citizens should not be insensitive about the problems they meet. instead, they should search for the reasons for these problems and attempt to solve. it is one of individuals’ responsibilities for themselves, other people and the world. in recent periods, the values education has become an effective method in producing active citizens. from 2005, the values education has been part of primary social studies programs. the values included in the social studies programs in turkey are as follows: importance of family unity and health, respect for the flag and national anthem, rights and freedoms, differences, fairness, independence, peace, freedom, scientific, industriousness, solidarity, sensitivity, integrity, aesthetics, tolerance, hospitality, cleanliness, nature, responsibility, patriotism and charity (mone, 2010). some of these values were included in the programs for different grades while others were grouped into sub-categories. for instance, the value of respect has five sub-categories and that of sensitivity has three sub-categories, which are related to historical heritage, natural environment and cultural heritage (keskin & öğretici, 2013). in the social studies program for grades of 4, 5, 6 and 7, which became effective in 2005, the most frequently stated value is that of sensitivity. sensitivity to the natural environment is one of the most frequent improvements targeted social studies (merey et al., 2012). in this course, there are certain values which are directly related to the natural environment and its protection. these values are that of love, respect, sensitivity, cleanliness, responsibility, fairness, solidarity, peace and aesthetics (karatekin & sönmez, 2014). one of the major goals of environmental education is to produce individuals who have environmental literacy which refers to cognitive and affective qualities about responsible acts towards the environment. sensitivity to the environment is one of the significant ingredients of environmental literacy (sivek, 2002). in producing responsible and sensitive individuals, informing them about social topics is a significant step. in addition, students should perceive social problems in a healthy way and have sensitivity to problems (johnson, 2005: cited in öcal et al., 2013). kıncal & işık (2003) analysed the democratic values and concluded that the values of sensitivity, responsibility and fairness are among the most included values in education worldwide. kıncal & işık (2003) found that basic democratic values include equality, respect for life, freedom, justice, honesty, quest for good, cooperation, self-confidence, tolerance, sensitivity and responsibility. individuals are expected to be sensitive to not only environmental and social problems, but also to cultural/historical heritage which consists of material and spiritual elements from the past. in a similar vein, the social studies program covers a learning domain called culture and heritage. this domain is explained as follows: “in this learning domain students generally become familiar with basic elements of turkish culture and develop an attitude towards the protection and improvement of it. students comprehend the fact that cultural elements in a society are distinctive features which make a distinction between their society and other societies and that cultural elements which are transferred from local to national and from national to international contribute to make culture much more varied.” (mone, 2010). in short, students are expected to be informed about and make evaluations concerning the cultural heritage of their society, about the continuity of culture and their own origins and to define their cultural and social identities (çulha özbaş, 2009). in social studies teachers play a significant role in protecting the cultural heritage and in developing awareness about its protection and significance of cultural heritage, and also in producing individuals who can internalize this awareness (selanik ay & kurtdede fidan, 2013). in teaching the topics related to cultural heritage, teachers may make use of several fields and materials such as historical places, cultural landscape, natural sites, sacred places, museums, festivals, sensitivity of fourth-grade students / fidan 405 traditional crafts, language, oral and written literature, religion and beliefs, rituals, music and dance, food culture, traditional children's games and sports (çulha özbaş, 2009). research suggests that sensitivity is among the values to be taught to students. susar kırmızı (2014) reviewed the textbook for the fourth-grade turkish language course in order to find which values were included in the texts. in the study of susar kırmızı (2014), it was found that the most frequently stated value in the texts was love of nature/sensitivity to natural environment (35% in 14 texts). ekinci et al. (2011) also reviewed the turkish language textbooks for the grades of 6, 7 and 8 in order to find which values are implied in the texts. they concluded that the most commonly emphasized values were patriotism, recognition of the national identity and sensitivity to cultural heritage and history. keskin (2008) examined the social studies programs which have been implemented since the establishment of the republic turkey. in the study of keskin (2008), it was found that all social studies programs included values and that the common values across the programs were cooperation/assistance, awareness, independence and responsibility. on the other hand, there are studies arguing that although values are covered in the educational programs and textbooks, students cannot acquire them sufficiently. in a study by yiğittir & öcal (2011), teachers reported that the acquisition of the values such as academic honesty, scientific diligence, sensitivity to the natural environment, aesthetics, respect for diversity, sensitivity and sensitivity to cultural heritage, awareness about historical heritage, self-confidence and responsibility cannot be efficiently transferred to students. elbir & bağcı (2013) reviewed 16 master’s theses and five phd theses in their study. they concluded that teachers did not have necessary information about the values education. although values are the center of education, the necessary importance has not been given to the values education. in education subjects and academic learning are emphasized, but there is a tendency to neglect the values (einarsdottira et al., 2015). sensitivity refers to developing relations with the world and the events and developing a responsibility about them. it is certain that individuals have ongoing relationships with the environment and the world. on the other hand, individuals live in an environment of which they are inseparable part. therefore, individuals should have relations to their environment and be sensitive to each ingredient of the environment. therefore, it is very significant that the awareness of students about sensitivity values should be improved (keskin & öğretici, 2013). there are numerous studies about the environmental sensitivity of students and also, of student teachers (çabuk & karacaoğlu 2003; makki et al., 2003; huang & yore, 2003; yilmaz et al., 2004; tuncer et al., 2005; erol & gezer 2006; uzun & sağlam, 2006; başal et al., 2007; chu et al., 2007; kaiser et al., 2007; tuncer, et al., 2009; gülay, 2011; tirri & nokelainen 2011; ozsoy et al.; yaşaroğlu, 2012; bilge, 2015; başal et al., 2015). however, there is no study specifically dealing with the sensitivity levels of primary students and secondary schools about the natural environment, animals, cultural heritage and social problems. therefore, the major aim of the study is to reveal sensitivity levels of fourth-grade students attending public schools concerning the natural environment, animals, social problems and cultural heritage. in parallel to this aim, the current study seeks to answer the following research questions: 1) at which level do the fourth-grade students have sensitivity to natural environment, animals, social problems and cultural heritage? 2) does sensitivity of the fourth-grade students significantly vary based on the following factors? a) gender b) residence c) educational background of parents international electronic journal of elementary education vol.8, issue 3, 403-424, 2016 406 d) occupations of parents e) frequency of following news method model of the study this study, which aims to reveal the sensitivity levels of fourth-grade students, attending public schools, concerning the natural environment, animals, social problems and cultural heritage based on some variables, was designed as having a scanning model. participants the participants of the study were a total of 447 fourth-grade students attending fifteen different public schools in afyonkarahisar province in the school year of 2014-2015. of them 238 were females (53.2%) and 209 (46.8%) were males. in regard to residence, 296 resided in the city centre, 123 in the villages and 28 in towns. table 1 presents demographic data about the participants. table 1. demographic data about the participants n % gender female 238 53.2 male 209 46.8 total 447 100 residence village 123 27.5 town 28 6.3 city 296 66.2 total 447 100 educational background of father illiterate 10 2.2 literate (without any formal education) 40 8.9 primary school 147 32.9 high school 153 34.2 undergraduate 81 18.1 graduate 16 3.6 total 447 100 educational background of mother illiterate 17 3.8 literate (without any formal education) 44 9.8 primary school 223 49.9 high school 100 22.4 undergraduate 53 11.9 graduate 10 2.2 total 447 100 occupation of father public servant 117 26.2 worker 139 31.1 private sector employer 48 10.7 tradesman 79 17.7 farmer 64 14.3 total 447 100 occupation of mother public servant 49 11 worker 32 7.2 housewife 359 80.3 other total 7 447 1.6 100 sensitivity of fourth-grade students / fidan 407 table 1. (cont.) demographic data about the participants how often you follow news? never 32 7.2 once a month 6 1.3 once a week 39 8.7 several days a week 186 41.6 everyday 184 41.2 total 447 100 data collection tools the data of the study were collected through the sensitivity value scale developed by the author. the scale also includes a demographic form which covers items related to the gender of the student, residence, educational background and occupation of parents and frequency of following news. the scale consists of four dimensions, namely sensitivity to natural environment, sensitivity to animals, sensitivity to social concerns and sensitivity to cultural heritage, and includes fifty-eight items. the scale is designed as a three-point likert-type scale and participants are asked to answer each item using one of the following options: “always (3), sometimes (2) and never (1)”. items in the scale are all positive statements. the codes assigned to each item range between 1.00 and 3.00. the minimum score from the scale is 58, while the maximum score is 174. the validity of the scale, its content validity, and its construct validity were analysed. for the reliability of the scale, the cronbach alpha coefficient was calculated. during the scale development, first, the studies concerning the values education and the sensitivity value as well as the similar scales were examined. then a total of fifty-five fourth-grade students were asked to write an essay about the definition of sensitivity. these essays were analysed by the author and another specialist in social studies using descriptive analysis technique. based on all findings the scale was designed as having four dimensions: sensitivity to natural environment, sensitivity to animals, sensitivity to social concerns and sensitivity to cultural heritage. of these dimensions, the sensitivity to environment and the sensitivity to cultural heritage were included in the educational program for the social studies course. the remaining two, namely the sensitivity to animals and the sensitivity to social problems, were added based on the review of literature. items were written down in a plain and understandable manner and each item expressed a single view or feeling. at the end, the scale was developed with sixty-four items. the item was three-point likert-type. in order to analyze the appropriateness of the items in the scale for identifying students sensitivity about the natural environment, animals, cultural heritage and social problems, the scale was reviewed by seven field specialists working at a public university. content validity refers to the sufficiency of the items about the qualities to be tested in terms of quality and quantity (büyüköztürk, 2007). two education specialists, one psychological counselling and guidance specialist, two social studies specialists and two science education specialists reviewed the scale in regard to the content validity and the face validity. the reviews showed that there were three inappropriate items and three overlapping items. therefore, these items were excluded from the scale. the final version of the scale included a total of fifty-eight items. the scale was used in a pilot study with twenty fourth-grade students to review its intelligibility. following the pilot study some of the items were redesigned. for construct validity of the scale first and second order confirmatory factor analysis was employed. confirmatory factor analysis (cfa) is a special form of factor analysis and is a statistical technique used to verify the factor structure of a set of observed variables. cfa allows the researcher to test the hypothesis that a relationship between observed variables and their underlying latent constructs exists. the factorial model to be international electronic journal of elementary education vol.8, issue 3, 403-424, 2016 408 tested may be based on the findings of an amprical study or based on a theory (sümer, 2000). there are numerous fit indices used to verify the validity of the model in the context of cfa. of them the most frequently used ones are as follows (cole, 1987; sümer, 2000): chi-square goodness (χ 2 ), root mean square error of approximation (rmsea), comparative fıt index (cfi), non-normed fit index (nnfi), normed fit index (nfi) and goodness of fit index (gfi). if the values observed are distributed in the following values χ2/d<3; 0<rmsea<0.05; 0.97≤nnfi≤1; 0.97≤cfi≤1; 0.95≤gfi≤1 and 0.95≤nfi≤1, it indicates that there is a perfect fit; the values of 4<χ2/d<5; 0.05<rmsea<0.08; 0.95≤nnfi≤0.97; 0.95≤cfi≤0.97; 0.90≤gfi≤0.95 and 0.90≤nfi≤0.95 indicate an acceptable fit (kline, 2005; sümer, 2000). in the first cfa, those items with statistically no significant t values were sought. it was found that there was no such item in the scale, and therefore, no item was not excluded from the scale. the path diagram resulted from the analyses is given in figure 1. in addition, cfa showed the following fit indices for the scale: χ 2 =5290.08, x2/sd= 3.33, rmsea=.072, cfi=.95, nfi=.92, nnfi=.95 and ifi=.95. the analysis indicated that all coefficients given above were at a sufficient level. the fit indices showed that there was a consistency between observed variables and their underlying latent constructs. table 2. regression and t values for the scale items regression values t values items regression values t values i1 0.25 9.10 i30 0.32 8.19 i2 0.31 9.66 i31 0.30 7.63 i3 0.34 9.87 i32 0.23 6.98 i4 0.21 9.08 i33 0.27 7.90 i5 0.28 9.77 i34 0.34 7.80 i6 0.14 7.63 i35 0.25 7.85 i7 0.36 8.60 i36 0.25 8.16 i8 0.28 8.29 i37 0.31 8.27 i9 0.40 10.57 i38 0.30 7.15 i10 0.34 9.71 i39 0.27 8.14 i11 0.24 8.80 i40 0.24 7.74 i12 0.22 9.11 i41 0.33 8.48 i13 0.29 8.92 i42 0.29 8.39 i14 0.39 10.02 i43 0.33 8.44 i15 0.36 10.09 i44 0.29 10.46 i16 0.24 8.66 i45 0.30 9.62 i17 0.25 9.09 i46 0.34 11.36 i18 0.30 9.29 i47 0.34 11.24 i19 0.34 9.24 i48 0.29 10.45 i20 0.23 7.79 i49 0.30 9.94 i21 0.29 8.45 i50 0.39 11.75 i22 0.28 8.46 i51 0.35 10.79 i23 0.26 8.48 i52 0.31 8.82 i24 0.52 9.85 i53 0.40 11.54 i25 0.51 9.31 i54 0.37 11.62 i26 0.51 9.58 i55 0.36 11.35 i27 0.29 8.45 i56 0.25 9.96 i28 0.24 8.24 i57 0.36 10.74 i29 0.21 7.71 i58 0.29 10.15 sensitivity of fourth-grade students / fidan 409 figure 1. shows the regression and t values of the four-factor model obtained from the cfa. international electronic journal of elementary education vol.8, issue 3, 403-424, 2016 410 table 2 indicates that both the regression values and t values obtained are significant and that the model is confirmed. table 2 also shows that t values range between 6.98 and 11.75. as stated earlier, the scale consisted of four dimensions. there are nineteen items under the dimension of the sensitivity to the natural environment, namely ı1, ı2, ı3, ı4, ı5, ı6, ı7, ı8, ı9, ı10, ı11, i12, 13, ı14, ı15, ı16, ı17, ı18 and ı19. the dimension of the sensitivity to animals includes seven items, namely ı20, ı21, ı22, ı23, ı24, ı25 and ı26. the third dimension, namely the dimension of the sensitivity to social problems, consisted of seventeen items, namely ı27, ı28, ı29, ı30, ı31, ı32, ı33, ı34, ı35, ı36, ı37, ı38, ı39, ı40, ı41, ı42 and ı43. there are a total of fifteen items in the dimension of the sensitivity to the cultural heritage, namely ı44, ı45, ı46, ı47, ı48, ı49, ı50, ı51, ı52, ı53, ı54, ı55, ı56, ı57 and ı58. in regard to the reliability of the scale, the cronbach alpha coefficient was found. the analysis showed that the cronbach alpha coefficient for the dimension of the sensitivity to the natural environment was .89. it was found to be .81 for the dimension of the sensitivity to animals. the cronbach alpha coefficient for the dimension of the sensitivity to social problems was found to be .89. it was found to be .90 for the dimension of the sensitivity to cultural heritage. the cronbach alpha coefficient was found to be .96 for the scale as a whole. data analysis the data collected were analysed using the spss 20. the normal distribution of the variables was analysed using the shapiro-wilk test. the significance level was set at .05. more specifically, if the significance level is p<.05, it refers to not normally distributed variables. however, if it is p>.05, it refers to normally distributed variables. the differences among the groups were analysed using the mann whitney u test and the kruskal wallis-h test when the variables were not normally distributed. given that the number of units is more than 20 the standardized z values for the mann whitney u test were given. when the kruskal wallis-h test produced significant differences, the groups causing the difference were identified using the multiple/post hoc comparison test. the significance level was set at .05. more specifically, the significance level of p<.05 indicated a significant difference while the significance level of p>.05 showed a non-significant difference. findings this section presents the findings of the study. table 3 shows the mean scores of the participants from the scale and from the dimensions of the scale together with arithmetic mean and standard deviation. table 3. mean scores and standard deviation related to the scale and the dimension scores n mean median minimum maximum sd sensitivity to natural environment 447 51.2 53.0 31.0 57.0 5.9 sensitivity to animals 447 18.4 19.0 7.0 21.0 2.8 sensitivity to social problems 447 46.6 49.0 29.0 51.0 5.1 sensitivity to cultural heritage 447 40.8 43.0 19.0 45.0 5.2 total sensitivity 447 157.1 162.0 110.0 174.0 16.6 as can be seen in table 3, the participants have an overall mean score of m= 157 from the sensitivity value scale. the minimum score from the scale is 58 and the maximum score is 174. the mean score of 157 suggests that their overall sensitivity is at a higher sensitivity of fourth-grade students / fidan 411 level. in terms of the dimensions of the scale, it is seen that the participants are sensitive to all dimensions, namely to the natural environment, animals, social problems and cultural heritage. as stated earlier, the effects of some variables on the sensitivity of the fourth-grade students were analysed. in regard to the effects of gender on their sensitivity levels the mann whitney u test was employed. the results of the test are given in table 4. table 4. the results of the mann whitney u test regarding the effects of gender on the mean scores for the dimensions of the scale gender mann whitney u test n mean min max sd mean rank z p effect size cohen’s d score for the dimension of the sensitivity to the natural environment female 238 51.8 35 57 5.7 236.83 -2.261 .024 0.20 male 209 50.6 31 57 6 209.39 total 447 51.2 31 57 5.9 score for the dimension of the sensitivity to animals female 238 18.5 7 21 3 231.08 -1.277 0.20 2 male 209 18.3 11 21 2.7 215.94 total 447 18.4 7 21 2.8 score for the dimension of the sensitivity to social problems female 238 47.1 33 51 4.7 232.77 -1.568 .117 male 209 46.2 29 51 5.4 214.01 total 447 46.6 29 51 5.1 score for the dimension of the sensitivity to cultural heritage female 238 41.1 24 45 5.1 233.74 -1.755 .079 male 209 40.4 19 45 5.3 212.9 total 447 40.8 19 45 5.2 total sensitivity score female 238 158.5 113 174 16.3 236.16 -2.128 .033 0.18 male 209 155.4 110 174 16.9 210.16 total 447 157.1 110 174 16.6 table 4 shows that the variable of gender has statistically significant effects on the scores of the participants in the dimension of the sensitivity to the natural environment (p<.05). more specifically, female students had significantly higher mean scores in the dimension of the sensitivity to the natural environment in contrast to male students. gender was also found to have statistically significant effects on the overall sensitivity score (p<.05) in that again female students had significantly much higher overall mean sensitivity scores in contrast to male students. effect size has been calculated to determine how much the gender variable has been effective on the sensitivity to the natural environment and the overall sensitivity. these scores (effect size= 0.20 and 0.18) that have been calculated related to the effect size show that the gender variable has little effects on sensitivity. based on the findings of both the effect sizes that have been calculated, it can be said that the gender variable has quite little (cohen, 1992) effect on the sensitivity towards the natural environment and the overall sensitivity. huang and yore (2003) explored the differences between two culturally distinct (canadian and taiwanese students) groups and also developed models of children's responsible environmental behavior. the differences in results for boys and girls, levels of nature participation, and nationalities were found to be significant but of small to moderate effect size. these effect sizes indicate that the comparison groups were more similar than different. (huang & yore, 2003). the potential effects of residence on the sensitivity levels of the participants were analysed using the kruskal wallis-h test. the results are given in table 5. international electronic journal of elementary education vol.8, issue 3, 403-424, 2016 412 table 5. the results of the kruskal wallis-h regarding the effects of residence on the mean scores for the dimensions of the scale residence kruskal wallis h test n mean min max sd mean rank h p score for the dimension of the sensitivity to the natural environment 1.village 123 49 32 57 6.4 177.69 50.366 .001 2.town 28 46.4 37 56 5.7 118.88 3.city 296 52.6 31 57 5.1 253.19 total 447 51.2 31 57 5.9 2-3 1-3 score for the dimension of the sensitivity to animals 1.village 123 17.8 11 21 2.7 190.02 39.254 .001 2.town 28 16.1 12 21 2.8 120.05 3.city 296 18.9 7 21 2.7 247.95 total 447 18.4 7 21 2.8 2-1 2-3 1-3 score for the dimension of the sensitivity to social problems 1.village 123 44.8 31 51 5.6 180.32 58.954 .001 2.town 28 41 30 51 5.7 98.88 3.city 296 47.9 29 51 4.2 253.99 total 447 46.6 29 51 5.1 2-1 2-3 1-3 score for the dimension of the sensitivity to cultural heritage 1.village 123 39.4 24 45 5.6 188.39 53.236 .001 2.town 28 35.3 26 45 5.1 95.25 3.city 296 41.9 19 45 4.5 250.98 total 447 40.8 19 45 5.2 2-1 2-3 1-3 total sensitivity score 1.village 123 151 110 174 17.9 179.64 59.67 .001 2.town 28 138.9 113 170 15.4 94.45 3.city 296 161.3 111 174 14.2 254.69 total 447 157.1 110 174 16.6 2-1 2-3 1-3 table 5 indicates that the variable of residence has a statistically significant effect on the mean scores of the participants both in overall scores and in the scores of the dimensions. therefore, it had significant effects in regard to the mean scores of the participants in the dimension of the sensitivity to the natural environment, the sensitivity to animals, the sensitivity to social problems, the sensitivity to the cultural heritage (p <.05). the effect of this variable on the overall sensitivity mean score was also found to be statistically significant (p <.05). more specifically, those participants living in villages and towns had lower overall mean sensitivity scores in contrast to those living in the city. in addition, those living in towns had lower overall mean sensitivity scores in contrast to those living in villages. the potential effects of the educational background of fathers on the sensitivity levels of the participants were analysed using the kruskal wallis-h test. the results are given in table 6. table 6. the results of the kruskal wallis-h regarding the effects of the educational background of fathers on the mean scores for the dimensions of the scale educational background of fathers kruskal wallis h test n min max ss mean rank h p score for the dimension of the sensitivity to the natural environment 1.illiterate 10 46 57 4.4 217.4 31.431 .001 2.literate 40 36 57 7 199.75 3.primary school 147 32 57 6.3 181.65 4.high school 153 31 57 5.2 247.05 5.undergraduate 81 39 57 4.7 260.28 6.graduate 16 39 57 4.6 273.75 total 447 31 57 5.9 3-4 3-5 sensitivity of fourth-grade students / fidan 413 table 6. (cont.) the results of the kruskal wallis-h regarding the effects of the educational background of fathers on the mean scores for the dimensions of the scale educational background of fathers kruskal wallis h test n min max sd mean rank h p score for the dimension of the sensitivity to animals 1.illiterate 10 14 21 2.4 204.75 19.238 .002 2.literate 40 12 21 2.6 226.69 3.primary school 147 7 21 3.1 190.17 4.high school 153 7 21 2.8 236.8 5.undergraduate 81 12 21 2.3 252.86 6.graduate 16 15 21 2.1 271.66 total 447 7 21 2.8 3-4 3-5 score for the dimension of the sensitivity to social problems 1.illiterate 10 35 51 5.4 182.85 38.721 .001 2.literate 40 29 51 5.8 199.6 3.primary school 147 31 51 5.5 180.56 4.high school 153 30 51 4.6 243.87 5.undergraduate 81 34 51 3.8 268.93 6.graduate 16 39 51 3.7 292.41 total 447 29 51 5.1 3-4 3-5 36 score for the dimension of the sensitivity to cultural heritage 1.illiterate 10 26 45 6.3 206.9 20.188 .001 2.literate 40 27 45 5.8 203.44 3.primary school 147 24 45 5.5 191.26 4.high school 153 24 45 4.3 245.35 5.undergraduate 81 19 45 5 247.65 6.graduate 16 24 45 6.2 263.03 total 447 19 45 5.2 3-4 3-5 total sensitivity score 1.illiterate 10 126 174 16.9 199.7 34.532 .001 2.literate 40 111 174 18.4 200.59 3.primary school 147 110 174 17.7 179.76 4.high school 153 115 174 14.9 247.82 5.undergraduate 81 118 174 13.7 263.22 6.graduate 16 119 174 14.6 277.81 total 447 110 174 16.6 3-4 3-5 as can be seen in table 6, the educational background of participants’ fathers had statistically significant effects on both overall sensitivity scores and the scores for the dimensions of the scale. the variable of fathers’ educational background had significant effects in regard to the mean scores of the participants in the dimension of the sensitivity to the natural environment, the sensitivity to animals, the sensitivity to social problems, the sensitivity to the cultural heritage (p<.05). the variable of fathers’ educational background had significant effects in regard to the overall mean sensitivity scores of the participants environment (p<.05). those participants whose fathers were the graduates of primary school had lower mean sensitivity scores than those participants whose fathers were the graduates of either high school or university. the potential effects of the educational background of mothers on the sensitivity levels of the participants were analysed using the kruskal wallis-h test. the results are given in table 7. international electronic journal of elementary education vol.8, issue 3, 403-424, 2016 414 table 7. the results of the kruskal wallis-h regarding the effects of the educational background of mothers on the mean scores for the dimensions of the scale educational background of mothers kruskal wallis h test n mean min max sd mean rank h p score for the dimension of the sensitivity to the natural environment 1.illiterate 17 50.5 40 57 6.2 215.53 21.145 0.001 2.literate 44 49.6 37 57 6 187.18 3.primary school 223 50.3 32 57 6.2 205.72 4.high school 100 53 36 57 4.7 260.37 5.undergraduate 53 52.9 31 57 5.4 261.34 6.graduate 10 53.1 49 57 2.8 246.45 total 447 51.2 31 57 5.9 2-4 3-4 score for the dimension of the sensitivity to animals 1.illiterate 17 18.5 14 21 2.5 225.76 11.571 0.041 2.literate 44 18.2 12 21 2.7 209.59 3.primary school 223 18.1 7 21 3.1 208.85 4.high school 100 18.7 7 21 2.7 240.55 5.undergraduate 53 19.2 13 21 2.1 256.32 6.graduate 10 19.8 16 21 1.9 285.6 total 447 18.4 7 21 2.8 3-4 3-5 score for the dimension of the sensitivity to social problems 1.illiterate 17 45.1 35 51 6.2 197.97 22.717 0.001 2.literate 44 45.2 29 51 5.4 183.23 3.primary school 223 46 30 51 5.3 207.91 4.high school 100 47.8 34 51 4.3 253.7 5.undergraduate 53 48.3 34 51 4.1 266.9 6.graduate 10 49 42 51 3.1 282.15 total 447 46.6 29 51 5.1 2-4 2-5 3-4 3-5 score for the dimension of the sensitivity to cultural heritage 1.illiterate 17 39.1 26 45 6.3 194.91 16.307 0.006 2.literate 44 39.5 27 45 5.3 188.64 3.primary school 223 40.4 24 45 5.2 212.21 4.high school 100 42.1 28 45 4 253.67 5.undergraduate 53 41.5 19 45 6.2 259.1 6.graduate 10 40.6 31 45 4.6 209.3 total 447 40.8 19 45 5.2 2-4 2-5 3-4 3-5 total sensitivity score 1.illiterate 17 153.2 126 174 19.9 208.56 20.476 0.001 2.literate 44 152.5 111 174 17.4 186.25 3.primary school 223 154.8 110 174 17.4 206.93 4.high school 100 161.7 122 174 13.2 255.52 5.undergraduate 53 162 115 174 15.8 268.11 6.graduate 10 162.5 149 174 9.4 248.05 total 447 157.1 110 174 16.6 2-4 2-5 3-4 3-5 table 7 shows that the educational background of participants’ mothers had statistically significant effects on both overall sensitivity scores and the scores for the dimensions of the scale. the variable of mothers’ educational background had significant effects in regard to the mean scores of the participants in the dimension of the sensitivity to the natural environment, sensitivity to animals, the sensitivity to social problems, the sensitivity to the cultural heritage (p<.05). the variable of mothers’ educational background had significant effects in regard to the overall mean sensitivity scores of the participants (p<.05). those participants whose mothers were literate without any formal education and those whose mothers were graduates of primary school had lower mean sensitivity scores than those participants whose mothers were the graduates of high school or university. the potential effects of the occupation of participants’ fathers on the sensitivity levels of the participants were analysed using the kruskal wallis-h test. the results are given in table 8. sensitivity of fourth-grade students / fidan 415 table 8. the results of the kruskal wallis-h regarding the effects of the occupation of fathers on the overall mean scores and mean scores for the dimensions of the scale occupation of fathers kruskal wallis h test n mean min max sd mean rank h p score for the dimension of the sensitivity to the natural environment 1.public servant 117 52.9 35 57 4.9 261.82 31.974 0.001 2.workers 139 50.5 37 57 5.8 205.17 3.private sector employer 48 53.1 41 57 4.5 259.6 4.tradesman 79 51.4 31 57 6 229.53 5.farmer 64 48.1 32 57 6.7 162.23 total 447 51.2 31 57 5.9 5-4 5-3 5-1 2-1 score for the dimension of the sensitivity to animals 1.public servant 117 19 7 21 2.6 251.08 23.509 0.001 2.workers 139 18.1 7 21 2.9 208.34 3.private sector employer 48 19.1 10 21 2.4 253.89 4.tradesman 79 18.6 7 21 2.9 237.52 5.farmer 64 17.3 11 21 2.9 169.41 total 447 18.4 7 21 2.8 5-4 5-1 5-3 score for the dimension of the sensitivity to social problems 1.public servant 117 48.2 34 51 3.8 259.83 30.29 0.001 2.workers 139 45.6 29 51 5.6 199.96 3.private sector employer 48 48.4 33 51 3.7 267.43 4.tradesman 79 46.7 31 51 5.4 227.42 5.farmer 64 44.7 34 51 5.2 173.91 total 447 46.6 29 51 5.1 5-1 5-3 2-1 2-3 score for the dimension of the sensitivity to cultural heritage 1.public servant 117 41.7 19 45 4.7 249.41 20.712 0.001 2.workers 139 40.1 27 45 5.2 203.87 3.private sector employer 48 42.4 24 45 3.9 260.48 4.tradesman 79 41.2 20 45 5.2 234.72 5.farmer 64 38.7 24 45 6.1 180.66 total 447 40.8 19 45 5.2 5-1 5-3 2-1 total sensitivity score 1.public servant 117 161.9 119 174 13.7 260.89 31.669 0.001 2.workers 139 154.3 111 174 17.6 203.02 3.private sector employer 48 163 125 174 12 261.85 4.tradesman 79 157.9 110 174 16.9 232.01 5.farmer 64 148.7 117 174 17.9 163.85 total 447 157.1 110 174 16.6 5-4 5-1 5-3 2-1 table 8 shows that the occupation of participants’ fathers had statistically significant effects on both overall sensitivity scores and the mean scores for the dimensions of the scale. this variable is found to have significant effects on the mean scores for the dimension of the sensitivity to the natural environment, sensitivity to animals, the sensitivity to social problems, the sensitivity to the cultural heritage (p<.05). the overall mean sensitivity score of the participants was also significantly affected by the occupation of fathers (p<.05). more specifically, those participants whose fathers were farmers had lower overall mean sensitivity scores than those whose fathers were private sector employers, or public servants or tradesmen. in addition, those participants whose fathers were workers were found to have lower overall mean sensitivity scores than those whose fathers were public servants. the potential effects of the occupation of participants’ mothers on the sensitivity levels of the participants were analysed using the kruskal wallis-h test. the results are given in table 9. international electronic journal of elementary education vol.8, issue 3, 403-424, 2016 416 table 9. the results of the kruskal wallis-h regarding the effects of the occupation of mothers on the overall mean scores and mean scores for the dimensions of the scale occupation of mothers kruskal wallis h test n mean min max sd mean rank h p score for the dimension of the sensitivity to the natural environment 1.public servant 49 53.4 39 57 4.1 261.3 5.881 0.118 2.worker 32 50.5 38 57 6.6 216.53 3.housewife 359 50.9 31 57 6 218.64 4.other 7 54 49 57 3.5 272 total 447 51.2 31 57 5.9 score for the dimension of the sensitivity to animals 1.public servant 49 19 13 21 2.3 246.36 4.827 0.185 2.worker 32 18.5 12 21 2.7 226.52 3.housewife 359 18.3 7 21 2.9 219.2 4.other 7 20 17 21 1.7 302 total 447 18.4 7 21 2.8 score for the dimension of the sensitivity to social problems 1.public servant 49 48.5 34 51 4 273.81 12.234 0.007 2.worker 32 45.8 29 51 5.9 206.55 3.housewife 359 46.4 30 51 5.1 217.16 4.other 7 49.4 44 51 2.8 305.79 total 447 46.6 29 51 5.1 3-1 score for the dimension of the sensitivity to cultural heritage 1.public servant 49 41.8 24 45 4.7 249.96 6.779 0.079 2.worker 32 40.7 30 45 5.2 228.48 3.housewife 359 40.6 19 45 5.3 218.24 4.other 7 44.1 40 45 1.9 317.21 total 447 40.8 19 45 5.2 total sensitivity score 1.public servant 49 162.7 119 174 12.7 262.84 8.459 0.037 2.worker 32 155.5 111 174 18.6 219.3 3.housewife 359 156.2 110 174 16.9 217.46 4.other 7 167.6 154 174 9.3 308.93 total 447 157.1 110 174 16.6 3-1 table 9 indicates that the occupation of participants’ mothers had statistically significant effects on both overall mean sensitivity scores and the score for the dimensions of the sensitivity to social problems (p<.05). it was found that those participants whose mothers were housewives had lower mean score for the dimensions of the sensitivity to social problems than those whose mothers were public servant. this variable is also found to have significant effects on the overall mean sensitivity scores (p<.05). more specifically, those participants whose mothers were housewives had lower overall mean sensitivity score than those whose mothers were public servant. the potential effects of participants’ habits of reading or listening to news on the sensitivity levels of the participants were analysed using the kruskal wallis-h test. the results are given in table 10. sensitivity of fourth-grade students / fidan 417 table 10. the results of the kruskal wallis-h regarding the effects of participants’ habits of reading or listening to news on the overall mean scores and mean scores for the dimensions of the scale frequency of following news kruskal wallis h testi n mean min max sd mean rank h p score for the dimension of the sensitivity to the natural environment 1.never 32 44.7 32 57 7.1 108.95 47.435 0.001 2.once a month 6 47.8 42 54 3.9 126.5 3.once a week 39 49.6 39 57 5.7 184.09 4.several days a week 186 51 35 57 5.8 220.18 5.everyday 184 53 31 57 4.8 259.51 total 447 51.2 31 57 5.9 1-4 1-5 3-5 4-5 score for the dimension of the sensitivity to animals 1.never 32 15.7 7 21 3.6 125.08 33.674 0.001 2.once a month 6 17.2 15 21 2.4 153.08 3.once a week 39 17.6 11 21 3.1 185.41 4.several days a week 186 18.6 12 21 2.5 225.6 5.everyday 184 19 7 21 2.7 250.08 total 447 18.4 7 21 2.8 1-4 1-5 3-5 score for the dimension of the sensitivity to social problems 1.never 32 42.1 29 51 6.5 132.58 44.35 0.001 2.once a month 6 41.7 35 47 4 90.33 3.once a week 39 45.6 32 51 5.2 194.9 4.several days a week 186 46.3 30 51 5 212.08 5.everyday 184 48.2 33 51 4.1 262.48 total 447 46.6 29 51 5.1 2-5 1-4 1-5 3-5 4-5 score for the dimension of the sensitivity to cultural heritage 1.never 32 37.1 27 45 5.8 143.86 22.043 0.001 2.once a month 6 39.8 29 45 5.9 200.25 3.once a week 39 40.1 24 45 5 199.04 4.several days a week 186 40.6 20 45 5.2 219.68 5.everyday 184 41.7 19 45 4.8 248.37 total 447 40.8 19 45 5.2 1-4 1-5 total sensitivity score 1.never 32 139.7 110 174 19.7 114.78 43.391 0.001 2.once a month 6 146.5 125 162 12.6 123.25 3.once a week 39 152.9 119 174 16 184.5 4.several days a week 186 156.5 117 174 16.4 220.53 5.everyday 184 161.9 113 174 14 258.16 total 447 157.1 110 174 16.6 1-4 1-5 3-5 4-5 table 10 shows that participants’ habits of following news had statistically significant effects on both overall sensitivity scores and the scores for the dimensions of the scale. this variable is found to have significant effects on the mean scores for the dimension of the sensitivity to the natural environment, of the sensitivity to the natural environment, the sensitivity to animals, the sensitivity to social problems, the sensitivity to the cultural heritage (p<.05). the overall mean sensitivity score was also found to be significantly affected by participants’ habits of following news (p<.05). those participants who never read or listened to news had significantly lower overall mean sensitivity scores than those who read or listened to news once a week or several days a week and those who read or listened to news several days a week had significantly lower overall mean sensitivity scores than those who read or listened to news every day. international electronic journal of elementary education vol.8, issue 3, 403-424, 2016 418 discussion and conclusions in the study, it was found that the mean scores of the fourth-grade students were high for both the scale as a whole and the dimensions of the scale. therefore, it can be argued that the participants were sensitive to the natural environment, animals, social problems and cultural heritage. keskin & öğretici (2013) examined the effects of the activities used in the social studies course to improve the student awareness about sensitivity. they found that these activities specifically designed to improve sensitivity were successful in achieving the goal. however, it was also found that the knowledge base and awareness of the sixth-grade students regarding sensitivity were very limited, although they studied topics about the sensitivity to the natural environment and the sensitivity to historical heritage in the fifth-grade. on the other hand, they suggested that activity-based teaching and learning should be employed in the social studies courses and that the textbooks should contain much more references to the values education. erdoğan (2009) found that the fifth-grade students had higher levels of environmental attitudes and the sensitivity to environment. in the current study it was found that the variables of gender, residence, educational background and occupation of parents significantly affected the sensitivity levels of the participants concerning the natural environment, animals, social problems and cultural heritage. the gender of the participants had statistically significant effects on both their overall mean sensitivity scores and their mean scores for the dimensions of the scale. it was seen that female students had much higher levels of sensitivity in contrast to the male students sampled. research findings suggest that gender is a significant factor concerning sensitive behaviour of individuals. the inferential statistics showed that female students had higher levels of environmental sensitivity, environment-related information and positive attitudes towards environment. in the literature, gender difference in favour of females is reported in many of research studies (huang & yore, 2003; chu et. al. 2007). similar results are also reported in studies conducted with turkish samples (by çabuk & karacaoğlu, 2003; yılmaz et. al. 2004;, tuncer et. al. 2005; atasoy & ertürk 2008; keskin 2008; tuncer et. al. 2009; ozsoy et al.). the findings of the current study are consistent with these previous findings. on the other hand, there are studies suggesting that female students’ levels of environmental sensitivity, environment-related information and attitudes were lower than those of male students (o’brein, 2007). it was also found in the current study that the place of living had statistically significant effects on both overall mean sensitivity scores and the mean scores for the dimensions of the scale. those participants living either in villages or in towns had lower mean scores for the dimension of the sensitivity to the natural environment in contrast to those living in the city. it was also found for the dimensions of the sensitivity to animals, social problems and cultural heritage. in addition, those living in towns had lower mean scores in the dimensions of the sensitivity to animals, social problems and cultural heritage in contrast to those living in villages. therefore, those students living in villages and towns have lower levels of sensitivity to the topics examined in the study. it was seen that residence plays a significant role in shaping sensitivity levels. it is argued that people living in cities are much more anxious about environment than those living in villages (fransson & gärling, 1999 cited in erdoğan 2009). there are studies suggesting that students living in cities have much higher awareness about the environmental problems and are much opitimistic about the solutions for such problems (tuncer et. al, 2005; yılmaz et. al; 2004; yaşaroğlu, 2012). it is natural that students living in cities much more frequently come across environmental problems resulting from rapid urbanization and fast population growth. therefore, they are very eager to exhibit actions to protect the natural environment (erdoğan, 2009). the findings of the present study concerning the sensitivity of fourth-grade students / fidan 419 higher sensitivity levels of students’ living in cities about animals, social problems and cultural heritage can also be accounted for using the same factors. in the current study, it was found that the educational background of parents had a statistically significant effect on the mean scores of the participants both in overall scores and in the scores of the dimensions. those participants whose fathers were the graduates of primary school had significantly lower mean scores for the dimensions of the sensitivity to the natural environment, animals, social problems and cultural heritage as well as significantly lower overall mean sensitivity score in contrast to those participants whose fathers were the graduates of either high school or university. in addition, those participants whose mothers were literate without any formal education or the graduate of primary school had significantly lower mean scores for the dimensions of the sensitivity to the natural environment, animals, social problems and cultural heritage as well as significantly lower overall mean sensitivity score in contrast those participants whose mothers were the graduates of either high school or university. therefore, it is possible to argue that those students whose parents have higher levels of education have much higher levels of sensitivity. in other words, educated parents seem to educate their children more sensitively and more consciously. such parents encourage their children to read books and to play games and are the models for their children in regard to sensitive acts and behaviours. varışlı (2009) and chu et. al (2007) also found that educational background of parents had a significant effect on their children’s environmental literacy. carlisle (2007) argued that the educational background of parents has positive effects on their children’s knowledge about environment and their attitudes towards environment. because educated parents share their knowledge and awareness about environment with their children through different activities (cited in varışlı; 2009; erdoğan, 2009; yaşaroğlu, 2012). makki et. al. (2003) also concluded that the educational background of parents has positive effects on their children’s knowledge about environment and that those students whose parents have graduate education had higher levels of environment-related information and much more positive attitudes towards environment in contrast to those whose parents were the graduates of high school or whose parent had lower educational levels. keskin (2008) stated that the higher educational levels of parents higher scores for the sensitivity to historical heritage and to the natural environment. all these findings are consistent with the present findings and therefore, it can be argued that the educational levels of parents plays a significant role in their children’s sensitivity to the natural environment, animals, social problems and cultural heritage. in the current study, it was found that the occupation of participants’ fathers had statistically significant effects on both overall sensitivity scores and the mean scores for the dimensions of the scale. more specifically, the participants whose fathers were either farmers or workers had significantly lower mean scores for the dimensions of the sensitivity to the natural environment, animals, social problems and cultural heritage than those whose fathers were private sector employers, or public servants or tradesmen. yaşaroğlu, (2012) concluded that those students whose fathers were public servants exhibited much more environmentally sensitive behaviour in terms of interest in environment, cleaning-saving and love for animals in contrast to those students whose fathers were tradesmen or whose parents were unemployed. keskin (2008) also found that there was a statistically significant correlation between the sensitivity of the fifthgrade students to the natural environment and historical heritage and the occupation of their fathers. this correlation was in favor of those students whose fathers were either public servant, or tradesman or self-employed. in the study, it was also concluded that the occupation of participants’ mothers had statistically significant effects on their mean scores. more specifically, those participants international electronic journal of elementary education vol.8, issue 3, 403-424, 2016 420 whose mothers were housewives had significantly lower mean scores for the dimension of the sensitivity to social problems as well as significantly lower overall mean sensitivity scores in contrast to those whose mothers were public servants. therefore, it can be argued that children whose mothers are employed have much more sensitivity to social problems. staub (1979) stated that social sensitivity refers to behaviour related to positive social acts and related to the needs of other people. people have an instinct of considering the outcomes of their behaviour. if this inherent tendency is improved during the childhood and adolescence periods, individuals become responsible for their acts and sensitive to the society. however, if it is not improved or not encouraged, they become both irresponsible and insensitive. societal sensitivity is one of the basic life skills and it first emerges in family context and improves in the peer circles and in school (cited in akman . et. al., 2006). varışlı (2009) concluded that environmental knowledge which is a significant ingredient of environmental literacy was much higher in the students whose mothers were employed. those students whose mothers are employed better know the concepts related to the environmental sensitivity. for instance, keskin (2008) found that there was a significant correlation between the sensitivity of the fifth-grade students for the natural environment and historical heritage and the occupation of their mothers. this correlation was positive for the students whose mothers were public servants. this finding is consistent with the current finding. therefore, it is possible to suggest that both educational background of parents and their employment status have significant effects in children’s acquisition of knowledge, values and skills. in the study, it was also found that participants’ habits of following news had statistically significant effects on both overall sensitivity scores and the scores for the dimensions of the scale. those participants who never read or listened to news or those who read or listened to news several days a week, once a week or once a month had lower mean scores for the dimension of the sensitivity to the natural environment, animals, social problems and cultural heritage than those who read or listened to news every day. this finding clearly shows that frequent follow up news has positive effects on the sensitivity levels of people. through follow up actual events the awareness of students about the world improves and they can relate their learning with real events and apply their learning to the actual problems or situations. it is argued that interest in actual events improves interest in the world (moffatt, 1957; ord, 1972 cited in gedik, 2010). based on the findings of the study, the following suggestions were developed:  given that both education and socio-economic status are significant in producing sensitive individuals, parents living in villages or unemployed parents can be trained about sensitivity.  during the pre-school education behaviour related to the sensitivity to environment, animals, cultural heritage and social problems can be emphasized to begin to educate children.  projects can be developed to make sensitivity common in family, school and society; such projects may target younger children. in order to create awareness on sensitivity training can be offered to people in public education centres and in workplace and media may also support for similar attempts and activities. • • • sensitivity of fourth-grade students / fidan 421 references akman, b., yangın b., usluel-koçak y., kargı e., baş t., çelen n. & şanlı m. (2006). toplumsal duyarlılık ve sosyal hizmet i. uluslararası sosyal hizmetler sempozyumu, antalya-türkiye (9-12 nisan 2006) atasoy e. & ertürk h. (2008). i̇lköğretim öğrencilerinin çevresel tutum ve çevre bilgisi üzerine bir alan araştirmasi. erzincan eğitim fakültesi dergisi, 10(1), 105-122. başal, h. a., doğan, y. & atasoy, e. (2007). developing environmental sensivity for children scale: realibility and validity studies. journal of biological& environmental sciences, 1(2), 99-104. başal, h.a, özen, r. & bağçeli kahraman, p. (2015). adaptation study of environmental sensitivity scale for university students. the journal of academic social science studies, 35, 13-23. bilge, h. (2015). öğrencilerin çevresel duyarlılığının çeşitli değişkenler açısından incelenmesi. yayımlanmamış yüksek lisans tezi, [unpublished master thesis,] atatürk üniversitesi, eğitim bilimleri enstitüsü, erzurum. büyüköztürk, ş. (2007). sosyal bilimler için veri analizi el kitabi (8.baskı). ankara: pegem yayıncılık. chu, h., lee, e. a., ko, h. r., shin, d. h., lee, m. n., min, b. m., & kang, k. h. (2007). korean year 3 children’s environmental literacy: a prerequisite for a korean environmental education curriculum. international journal of science education, 29(6), 731–746. cohen j. (1992). statistical power analysis. current directions in psychological science, 1(3), 98-101. cole, d. a. (1987). utility of confirmatory factor analysis in test validation research. journal of consulting and clinical psychology, 55, 584-594. çabuk, b. & karacaoğlu, c. (2003). üniversite öğrencilerinin çevre duyarlıklarının incelenmesi. ankara üniversitesi eğitim bilimleri fakültesi dergisi, 36(1-2), 190-198 çulha özbaş b. (2009). i̇lköğretim sosyal bilgiler derslerinde kültürel miras eğitimi. mustafa safran (ed.). sosyal bilgiler öğretimi, (ss. 701-720), ankara: pegema yayıncılık. doğanay, a. (2006). değerler eğitimi. c. öztürk (ed.), hayat bilgisi ve sosyal bilgiler öğretimi (ss. 255–286). ankara: pegem yayıncılık ekinci çelikpazu, e. & aktaş, e. (2011). meb 6, 7 ve 8. sınıf türkçe ders kitaplarinda yer alan metinlerin değer iletimi açisindan incelenmesi. turkish studies international periodical for the languages, 6(2), 413-424. elbir b. & bağcı c. (2013). değerler eğitimi üzerine yapılmış lisansüstü düzeyindeki çalışmaların değerlendirilmesi, turkish studies international periodical for the languages, literature and history of turkish or turkic 8(1), 1321-1333. einarsdottira, j., purolab a.m., johanssonc, e.m., broströmd s. & emilson, a. (2015). democracy, caring and competence: values perspectives in ecec curricula in the nordic countries. international journal of early years education, 23(1), 97–114. erdoğan m. (2009). fifth grade students’ environmental literacy and the factors affecting students’ environmentally responsible behaviors, thesis doctor middle east technical university, ankara. erol, g. h. & gezer, k. (2006). sınıf öğretmenliği öğretmen adaylarının çevreye ve çevre s orunlarına yönelik tutumları. international journal of environmental and science education, 1(1), 65 – 77. gedik, h. (2010). güncel olaylarin ilköğretim sosyal bilgiler derslerinde kullanimi ve öğrenci görüşleri, ahi evran üniversitesi, kırşehir eğitim fakültesi dergisi (kefad) 11(2), 97-118. gülay, h. (2011). reliability and validity studies of the turkish version of the children’s attitudes toward the environment scale-preschool version (cates-pv) and the analysis of children’s proenvironmental behaviors according to different variables. asian social science. 7(10), 229-240. international electronic journal of elementary education vol.8, issue 3, 403-424, 2016 422 huang, h. & yore, l. (2003). a comparative study of canadian and taiwanese grade 5 children’s environmental behaviors, attitudes, concerns, emotional dispositions, and knowledge. international journal of science and mathematics education, 1, 419–448. kaiser, f., oerke, b. & bogner, f. (2007). behavior-based environmental attitude: development of an instrument for adolescents. journal of environmental psychology, 27, 242–251. karatekin k., & sönmez, ö.f. (2014). çevresel değerler eğitimi. refik turan ve kadir ulusoy (ed). farklı yönleriyle değerler eğitimi. (ss.46-73), ankara: pegem akademi. keskin, y. (2008). türkiye’de sosyal bilgiler öğretim programlarında değerler eğitimi: tarihsel gelişim 1998 ve 2004 programlarının etkililiğinin araştırılması. yayınlanmamış doktora tezi. [doctoral thesis], marmara üniversitesi, eğitim bilimleri enstitüsü, i̇stanbul keskin, y., & öğretici, b. (2013). sosyal bilgiler dersinde “duyarlılık” değerinin etkinlikler yoluyla kazandırılması: nitel bir araştırma. değerler eğitimi dergisi, 11(25), 143-181. kline, r. b. (2005). principles and practice of structural equation modeling (2nd edition ed.). new york: the guilford press. kılınç, e. (2014). pre-service social studies teachers’ understandings about the nature of the social studies, international electronic journal of elementary education, 6(3), 415-426. kıncal, r. & işık, h. (2003). demokratik eğitim ve demokratik değerler. eğitim araştırmaları, 3(11), 54-58. kuş, z. (2013). politik okuryazarlık ve aktif vatandaşlık, ebru gençtürk ve kadir karatekin (ed.). sosyal bilgiler i̇çin çoklu okuryazarlıklar, (ss.208-227), ankara: pegema yayıncılık. makki, m., abd-el-khalick, f., & boujaoude, s. (2003). lebanese secondary school students’ environmental knowledge and attitudes. environmental education research, 9(1), 21–34. merey z. , kuş, z. & karatekin k. (2012). türkiye ve abd ilköğretim sosyal bilgiler öğretim programlarının değerler eğitimi açısından karşılaştırılması. kuram ve uygulamada eğitim bilimleri, 12(2), 1613-1632. milli eğitim bakanlığı (mone) (2010). i̇lköğretim sosyal bilgiler dersi 4-5 sınıflar öğretim programı ve kılavuzu anı yayıncılık: ankara. o’brien, s. r. m. (2007). indications of environmental literacy: using a new survey instrument to measure awareness, knowledge, and attitudes of university-aged students. master’s thesis, iowa state university. iowa ozsoy s., ozsoy g. & kuruyer h. g.(2011).turkish pre-service primary school teachers' environmental attitudes: effects of gender and grade level, asia-pacific forum on science learning and teaching, 12(2), 1-21. öcal, a., demirkaya, h. & altınok, a. (2013). i̇lköğretim öğrencilerine yönelik sosyal duyarlılık ölçeğinin geliştirilmesi. pegem eğitim ve öğretim dergisi, 3(1), 67-76. öztürk, c. (2009). sosyal bilgiler: toplumsal yaşama disiplinlerarası bir bakış. c. öztürk (ed.). sosyal bilgiler öğretimi (ss. 1-31). ankara: pegem a yayıncılık. selanik ay, t. & kurtdede fidan, n. (2013). öğretmen adaylarının "kültürel miras" kavramına ilişkin metaforları, turkish studies international periodical for the languages, literature and history of turkish or turkic, 8(12), 1135-1152. sivek, d. (2002). environmental sensitivity among wisconsin high school student. environmental education research, 8(2), 155-170 susar kırmızı, f. (2014). 4. sınıf türkçe ders kitabı metinlerinde yer alan değerler, değerler eğitimi dergisi, 12(27), 217-259. sümer, n. (2000). yapısal eşitlik modelleri: temel kavramlar ve örnek uygulamalar. türk psikoloji yazıları, 3(6), 49-74. sensitivity of fourth-grade students / fidan 423 tirri, k. & nokelainen, p. (2011). environmental sensitivity scale. kirsi tirri & petri nokelainen (editors). measuring multiple intelligences and moral sensitivities in education. (pp.37-57) finland: sense publishers. tuncer, g., ertepinar, h., tekkaya, c., & sungur, s. (2005). environmental attitudes of young people in turkey: effects of school type and gender, environmental education research, 11, 215–233 tuncer, g., tekkaya, c., sungur, s., cakiroglu, j., ertepinar, h., & kaplowitz, m. (2009). assessing preservice teachers’ environmental literacy in turkey as a mean to develop teacher education programs, international journal of educational development, 426-436. uzun, n. & sağlam, n. (2006). orta öğretim öğrencileri için çevresel tutum ölçeği geliştirme ve geçerliliği. hacettepe üniversitesi eğitim fakültesi dergisi, 30, 240-250. varışlı, t. (2009). evaluating eighth grade students’ environmental literacy: the role of sociodemographic variables. [master thesis] middle east technical university, ankara. yaşaoğlu, c. (2012). i̇lköğretim birinci kademe öğrencilerinin çevreye yönelik tutum ve davranışlarının değerlendirilmesi. doktora tezi, i̇nönü üniversitesi eğitim bilimleri enstitüsü, malatya yiğittir, s. & öcal, a. (2011). lise tarih öğretmenlerinin değerler ve değerler eğitimi konusundaki görüşleri, kmü sosyal ve ekonomik araştırmalar dergisi, 13(20), 117-124. yilmaz, o., boone, w. j., & andersen, h. o. (2004). views of elementary and middle school turkish students toward environmental issues, international journal of science education, 26, 1527– 1546. international electronic journal of elementary education vol.8, issue 3, 403-424, 2016 424 www.iejee.com this page is intentionally left blank http://www.iejee.com/ © 2022 published by kura education & publishing. this is an open access article under the cc by ncnd license. (https://creativecommons.org/ licenses/by/4.0/) 123 copyright © www.iejee.com issn: 1307-9298 international electronic journal of elementary education december 2022, volume 15, issue 2, 123-136 handwriting speeds of 4th–8th grade students* ayşe dilek yekeler gökmena,**, mustafa yıldızb, nurhan aktaşc, merve ataşd abstract introduction for the effective expression of feelings and thoughts, handwriting should be produced fluently at a certain speed. the aim of this study was to examine the development of handwriting speeds in 4th–8th grade primary and secondary school students in terms of somevariables. in line with this aim, answers were sought to the question, “do 4th–8th grade primary and secondary school students’ handwriting speeds differ significantly depending on their grade, gender, hand preference, and handwriting style?” the study was designed inthe survey method, and the study group consisted of 322 students attending the 4th, 5th, 6th, 7th, and 8th grades of primary and secondary school. a form developed by the researchers for all grade levels was usedas the data collection tool in the study. in the data collection process, the students were required to copy a given text within one minute. the research results revealed that while the primary and secondary school students’ handwriting speeds differed significantly according to their grade level, gender, and handwriting style, they did not differ significantly according to their hand preferences. as the grade level increased, mean handwriting speeds also increased. female students wrote faster than male students. students using the manuscript handwriting style wrote faster than students who used the cursive handwriting style. handwriting is a language skill that begins with the ability to produce letters, and it is necessary for students to be able to express themselves, their feelings, and their thoughts. as temur, aksoy, and tabak maintains, “writing begins with various scribbling and drawing activities and requires coordinated movements of the elbow, shoulder, and trunk muscles with the hand and wrist muscles” (2012, p.311). güneş (2016) defines handwriting as a skill that requires various processes such as “holding a pencil, writing the letters, activating prior knowledge, organising, reviewing, and textualizing” (p.20). as the definitions suggest, handwriting skills entailspecific skills and stages. tompkins et al. (2014) discuss this process in three stages: preparation for writing, starting writing, and gaining fluency. at the preparation for writing stage, children need to recognise the keywords: handwriting speed, primary and secondary school students, handwriting style, hand preference received : 5 may 2022 revised : 21 december 2022 accepted : 29 december 2022 doi : 10.26822/iejee.2023.284 * the data of this research were collected simultaneously from the sample of the study named “development of story writing skills in primary school (1st-8th grades)” presented by the researchers as an oral presentation at the 15th international primary teacher education symposium. a,** corresponding author: ayşe dilek yekeler gökmen, department of elementary education, faculty of education, giresun university, giresun, turkey. e-mail: dilekyekeler@gmail.com orcid: https://orcid.org/0000-0002-7084-1986 b mustafa yıldız, gazi university, department of elementary education, gazi faculty of education, ankara, turkey. e-mail: mustafa@gazi.edu.tr orcid: https://orcid.org/0000-0003-3885-5322 c nurhan aktaş, selcuk university, department of elementary education, gazi faculty of education, konya, turkey. e-mail: nurhanakts@gmail.com orcid: https://orcid.org/0000-0003-0264-5120 d merve ataş, trabzon university, department of elementary education, institute of educational sciences, trabzon, turkey. e-mail: merveatas90@hotmail.com orcid: https://orcid.org/0000-0003-2312-6418 124 december 2022, volume 15, issue 2, 123-136 units (letters, words, sentences, and texts) that make up writing (güneş, 2007). subsequently, an attempt is made to foster handwriting skills in students from the 1stgrade of the primary school and onwards (ziviani & watson-will, 1998). according to bara and morin (2013), in the early stages, students must learn the shape, connections, and the direction of movement of letters. therefore, handwriting performance requires the careful and simultaneous use of a number of cognitive (e.g., intellect, attention), sensory, and psychomotor (e.g., motor development, muscle development) behaviours (dodd & carr, 2003, p.128). alongwith these behaviours, handwriting is affected by social factors such as constraints to use the right hand and environmental factors such as the literacy curriculum (ziviani & wallen, 2006). handwriting quality increases more rapidly during the first years of learning and develops more slowly at later ages (karsldottir & stefansson, 2003). handwriting requires cognitive effort not only for children at the beginning of the literacy process, but also for students at all grade levels (morin, morin, lavoie, & montésinos-gelet, 2012) since writing needs to be produced fluently at a certain speed so that feelings and thoughts can be expressed effectively at all stages of learning. a high handwriting speed is essential in terms of written communication as well as academic success (phelps, stembel, & speck, 1985; tseng & hsueh, 1997). young children reserve most of their cognitive energies for the motor aspect of writing (morin, lavoie, & montésinosgelet, 2012). therefore, in this process, for students to write accurately and fluently, teachers should pay attention not only to what students write, but also to how they write (taylor, 2010, as cited in başaran, 2020). when handwriting skills become more automatic, the attention and cognitive resources that are used to carry out other learning processes will mostly be used to develop the content of writing (mccutchen, 2011). graham and weintraub (1996) expressed this situation as children’sforgetting andnot recalling all their ideas while putting them on paperif they write slowly. in a study by medwell, strand, and wray (2009) in britain with children who finished primary school, it was revealed that handwriting, especially the ability to produce letters automatically, has an important role in text quality. if children do not produce letters at a certain speed and with a certain legibility, they cannot convert their ideas into written texts. the aim of teaching handwriting is to enable children to produce rapid and legible handwriting (galanis, 2008). failure in this learning process may affect school success (vinter & chartrel, 2010) and may cause writing problems such as inconsistency and slowness in writing speed (akyol, 2014).since copying, note-taking, composition writing, and written exams are based on their handwriting ability (bara & morin, 2013),students with high handwriting speeds are at an advantage compared to other students when putting their ideas in writing. particularly in the 1st grades, children’s acquisition of the skill of orderly and legible handwriting at a suitable speed is important in both educational and social sense (yıldız & ateş, 2010; ziviani & watson-will, 1998). therefore, legibility and writing speed are the two most important criteria in the development of handwriting (akyol, 2008; galanis, 2008; graham et al.,1998). sincethe handwriting speed is known to be directly related to students’ ability to express their feelings and thoughts, it has been the subject of scientific research studies for many years. it is generally understood that letters written within a certain time period are taken into consideration for determining handwriting speed (akyol, 2008). freeman (1954) was the first researcher to suggest the number of letters written per minute to measure handwriting speed and create a norm (as cited in tseng & hsueh, 1997). various studies were conducted with different grade levels to assess the handwriting speeds of students. basic findings regarding mean handwriting speeds determined in some studies withdifferent grade levels are presentedin table 1. in the table, the fact that the research findings show differences in mean handwriting speeds may be due table 1 mean handwriting speeds by copying according to different grade levels researcher(s) country number of students duration (minutes) grade level and words per minute 1 2 3 4 5 6 7 8 9 ziviani& elkins (1984) australia 575 2 32.6 34.2 38.4 46.1 52.1 ziviani (1984) australia 575 2 33 34 38 46 52 phelps, stempel, & speck (1985) usa 1365 2 35 46 54 66 hamstra-beltz & blote (1990) germany 127 5 24 35 46 54 66 66 graham et al. (1998) usa 900 1.5 19 34 47 63 73 85 100 115 118 temur (2012) turkey 75 2.5 115 jimenez & hernandez-cabrera (2019) spain 1124 1 5 6 türker & tunç (2020) turkey 95 1 51 55 skar et al. (2021) norway 4950 1.5 16 29 39 125 handwriting speeds of 4th–8th grade students / yekeler, yıldız, aktaş & ataş to the effect of variables such as students’ individual differences, the number of students included in the sample, the country’s language characteristics, and the use of different words or texts (temur, 2012). regarding language features, turkish is a transparent language. since the letter-sound relationship in transparent languages is almost standard (one sound corresponds to one letter), it facilitates the learner’s acquisition and development of literacy skills. for this reason, turkish literacy is taught and learned in a relatively short time. moreover, it is stated that the time required to write a letter varies according to itsshape and size, and since the shapes of some letters are simple, they are easily produced,and thus, the time taken to write each letter is not equal (güneş, 2016). besides these, it can be said that the method used to measure handwriting speed (dictation or copying) is one of the factors causing differences in handwriting speed. in addition to the studies above, there are also studies in which the effect of handwriting style on handwriting speed is examined. some of these state that cursive handwriting is written faster than manuscript handwriting (foster, 1957; graham & miller, 1980; kazu & ersözlü, 2006). in manuscript handwriting, since there is a pause after each letter, the handwriting process is interrupted and slowed. on the other hand, in cursive handwriting, since the student does not have to continue writing by frequently lifting his/her hand and finding the writing point again, his/her handwriting speed increases (akyol, 2008; güneş, 2006; güneş, 2007; meb, 2009). however, in some studies, it is revealed that while writing cursive letters, the student has to change the movement of his/her hand more (graham,1992), and it is stated that manuscript letters are learnt more easily and written more rapidly than cursive handwriting (bara & morin, 2013; berninger et al., 2006; gates & brown, 1929; houston, 1938; turan, 2010; yıldız et al., 2016; yıldız, 2019). arslan and ilgın (2010) maintained that teaching different handwriting styles could create problems for students and lead to students writing with a mixed style, which could have a negative effect on the legibility and the speed of handwriting in later grades. in the study that they conducted with 600 students in 4th–9th grades, graham, berninger and weintraub (1998) determined that there was no significant difference between beginning literacy with manuscript or cursive letters in terms of speed and legibility. morin, lavoie, and montésinos-gelet (2012) examined letter writing, word copying, and text creation skills over a 45-minute period in 715 canadian 2nd grade students who learned literacy with different handwriting styles. it was concluded that students who learned literacy with cursive handwriting made more progress in the areas of word production and syntax than students who learned the other handwriting styles. they explained this situation by stating that in cursive writing, in contrast tomanuscript letters, all the letters of the word are interconnected, which enables students to memorise word units more easily and facilitates their recall. in a study by bara and morin (2013), in which they examined the handwriting speed, legibility of written texts, and handwriting styles in 4th and 5th grade students according to the handwriting style they had learnt in the 1stgrade, significant differences were found between grade level, handwriting style, handwriting teaching method, and countries. it was concluded that canadian students wrote faster than french students, but their handwriting was less legible, the cursive handwriting speed was slower than a mixed handwriting style, and the handwriting speed and legibility improved as the grade level increased. in a study conducted with 9th grade students who had begun their first literacy education with cursive writing, aydın (2016) determined that seeing two different handwriting styles used in class created problems for students, and this situation had a negative effect on the handwriting speed and legibility in later grades. when other studies related to handwriting speed are examined, it is seen that variables such as gender, grade level, and hand preference are generally taken as the basis for comparisons. in their study, graham and weintraub (1996) reported that the relationship between handwriting speed and grade level might not always be linear. as a result of the study by tseng and hsueh (1997), which aimed to examine the handwriting speeds of 1525 chinese children in 2nd– 6th grades, it was reported that the handwriting speed increased with age, the rate of increase was greater among the 2nd, 3rd, and 4th grades, and girls wrote faster than boys in the 3rd, 4th, and 5th grades. in the study carried out with a total of 900 students between the 1st and 9th grades by graham et al. (1998), it was revealed that female students wrote faster than males at the beginning and end of primary school and at the beginning of secondary school, that right-handed students wrote faster than left-handed students, and that the handwriting speed showed a tendency to increase from one grade level to another. as a result of the study conducted by ziviani and watson & will (1998), in which they examined the handwriting speeds and text legibility of 372 students aged between 7 –14, it was stated that there was no significant difference between the mean scores for handwriting speed in male and female students, but girls were better than boys in terms of legibility, and there was a significant difference in text legibility according to gender, while a weak correlation between speed and legibility was found. o’mahony, dempsey, and killeen (2008) concluded that hand preferences of students aged 8 –18 had no effect on their handwriting speed, but the handwriting speeds of students attending schools that were named as disadvantaged were below average. as a result of his study, bay (2010) determined 126 december 2022, volume 15, issue 2, 123-136 that there was no significant difference between female and male students attending the 1stgrade in terms of handwriting speed. in their study examining the effect of 1st grade students’ pencil holding styles, pencil gripping, and compressive strengths on their handwriting speeds and errors,temur, aksoy, and tabak (2012) reported that students’ pencil gripping points affected their handwriting speeds, and girls’ handwriting speeds were faster than those of boys. temur (2012) aimed to evaluate turkish and american 3rd grade students’ handwriting speeds in terms of the variables of degree of flexion of the index finger and the position of the thumb and the forearm. as a result of the study, it was reported that the degree of flexion of the index finger did not lead to a significant change in students’ mean handwriting speeds, whereas the forearm and thumb position had an effect on mean handwriting speeds. moreover, it was determined that turkish students’ handwriting speeds were higher than those of american students, and in students of both countries, girls were more successful than boys in writing skills. aydın (2016) determined that 9th grade female students who had learnt literacy first with cursive handwriting had higher mean handwriting speeds than male students in both cursive and manuscript types of handwriting. in anotherstudy conducted with 95 students attending 3rdand 4thgrades of primary school, türker and tunç (2020) examined students’ manuscript handwriting speeds with regard to the variables of gender, age, grade level, hand preference, statusof receiving preschool education, class type (independent or combined class), and class size. the results of the research revealed that students attending independent classes wrote faster than students attending combined classes, but no significant differenceswere foundin terms of the other variables. the studies conducted offer different perspectives related to primary and secondary school students’ handwriting speeds. o’mahony, dempsey, and killeen (2008) state that handwriting speed is clearly an important factor worthy of examination on its ownbecause copying a text, taking notes, and freewriting tasks are among the most common writing tasks in school (fogel, rosenblum, & barnett, 2022). moreover, writing as a skill takes time for students. spending too much or little time on typing tasks depends on writing speed.hammerschmidt and sudsawad (2004) indicated that while evaluating students’ handwriting, teachers judge whether the writing is quick by comparing students with their peers. this situation makes it important to carry out this study and supports the idea that writing speed should be examined according to grade levels or age because there is no standard or norm for writing speed in terms of class levels. the statement that writing speed increases as grade level increases remainsvalidonly in theory or expectation. when studies conducted on this subject in turkey are examined, it is seen that they have generally been made with a single class level, e.g., with 1st grades (bay, 2010; duran & akyol, 2010; erdoğan, 2012; kadıoğlu, 2012; temur, aksoy, & tabak, 2012) and 3rd grades (temur, 2012; türker & tunç, 2020). therefore, there is a need for further research aimed at determining students’ handwriting speeds comparatively. some of the main writing tasks of students at school is answering questions in examsand classwork, such as copying from the board to notebooks or doing homework (barnett, prunty, & rosenblum, 2018). studies have stated that the inability to produce legible letters automatically and effortlessly at an appropriate speed may be an indicator that a child is at risk for developing inadequate composition skills (berninger & amtmann, 2003), low achievement, and low selfperception (feder & majnemer, 2007). research about writing speed can inform countries about determining and controlling the variables that may affect the speed of writing (e.g., letters are produced more effortlessly in cursive handwriting, right-handed people write faster) whiledesigning curricula about language and writing skills.this study has been designed with the aim of filling the gap that exists due to this deficiency by examining the development of students’ handwriting speeds with regard to severalvariables. within the scope of the research, answers were sought to the question, “do 4th–8th grade primary and secondary school students’ handwriting speeds differ significantly depending on their grade, gender, hand preference, and handwriting style?” method this study was designed as a survey method toexamine whether primary and secondary school students’ writing speed is affected by variables such as grade level, gender, hand preference, and writing style. "survey research enables to reveal what is experienced or what already exists. the researcher does not intervene in these phenomena or situations, he takes the phenomenon as it is, how it works, and examines it" (sönmez & alacapınar, 2017, pp.47-48).the study intended to describe the writing skills of a large sample of participants. study group table 2 distribution of participating students according to their gender and grade level gender grade level total 4 5 6 7 8 female 41 30 44 29 26 170 male 33 35 27 30 27 152 total 74 65 71 59 53 322 127 handwriting speeds of 4th–8th grade students / yekeler, yıldız, aktaş & ataş the study group of the research consisted of 322 students attending 4th, 5th, 6th, 7th, and 8th grades of primary and secondary schools in three state schools located in the city centres of ankara, trabzon, and giresun provinces (table 2). in order to determine the study group, the convenience sampling method was used since voluntary participants were selected in schools that could be easily and quickly accessed in the provinces where the researchers were employed. the aim of this sampling technique is to select cases where participants who are willing to take part can be included in the research according to the principle of convenience (patton, 2014). for this reason, an effort was made to reach a certain number of students (at least 30) from each class in each grade. the reason for starting the assessment of primary school students’handwriting speeds from the 4thgrade of primary school was that the handwriting skills of students in the 1st, 2nd, and 3rd grades are still in the process of motor development. moreover, they do not usually gain the fluencyneeded in handwriting skillsto express themselves in these grade levels yet. students begin to organise their ideas and put them in writing more efficientlyonce they have learnt to write letters, syllables, and words legibly and correctly (tompkins et al., 2014). they generate a writing character specific to themselves by beginning written homework, tests, and longer written exercises from the 4thgrade and onwards, whereas especially in the first three years of primary school, they are expected to acquire an adequate level of handwriting skill as a means by which they can conduct their schoolwork (yıldız &ateş, 2010, p.13). data collection tool in order to determine the handwriting speeds of the students in the study, a form that was developed by the researchers was used. the form consisted of a single text for all grade levels (a paragraph taken from an informative textfor determining students’ handwriting speeds), and questions related to students’ grade levels, gender, and hand preferences. tocreate the form, first,the literature was reviewed, and the criteria based on the assessment of handwriting speed were examined. drawing upon these criteria, an informative text which was included in the primary school turkish textbook andwhich was considered suitable to be read in schools by the board of instruction and discipline of the ministry ofnational education was selected. while choosing the text, care was taken to ensure that the text contained a topic on which all students could meet on common ground by obtaining the views of three fieldexperts. the informative text consisted of 49 words and 314 letters which included all the lowercase letters of the alphabet at least once, with the exception of j, p, and u. students come across informative texts more frequently in textbooks and in their daily lives. the selected text from the turkish textbook contains a suitable topic for all grade levels.the students were asked to copy and write the following text in the data collection form in one minute using the handwriting style they wished (seeappendix for its translation): “türkiye iki kıta üzerinde yer alan, dört mevsimin doyasıya yaşandığı cennet gibi bir ülkedir. ülkemiz üç tarafı denizlerle çevrili, bol güneşli, birbirinden güzel sahillerini size cömertçe sunar. bitki örtüsü ve hayvan türleri bakımından son derece zengindir. bu topraklar camileri, kiliseleri, heykelleri, sarayları, medreseleri ile geçmişi her an yaşar gibidir.” data collection process after obtaining the necessary permission from the schools to collect the data forthe study,students in the classrooms of teachers who were able to voluntarily spare time for the research were requested to copy and write the text on the form within one minuteto determine their handwriting speeds. before the text was written down by the students, information was given about the activity they would perform as follows:“the purpose of this study is to determine your writing speed in one minute. you can use any handwritingstyle (cursive or manuscript) you want.” students were given some time to prepare for writing, such as taking appropriate sitting positions for writing and preparing theirwriting tools.next, with the instruction “you can start writing,” the one-minute period began. at the end of one minute, with the instruction “you can stop writing,” the students were asked to circle the last letter they had written and put down their pencils. in granting a period of one minute, the study by graham et al. (1998) was taken as reference. data analysis for the analysis of the research data, descriptive statistics (percentage, frequency, arithmetic mean, standard deviation, and kurtosis and skewness values) were used to evaluate the writing speed of the students by analysingthe number of letters written per minute according to the grade levels, and twoway analysis of variance was used to determine whether there was a difference between the groups. while investigating the effect of more than one independent variable on the dependent variable, the examination of two different effects at the same time by taking into account of the interrelated interaction of independent variables and the case of interaction between variables is known as “interactive analysis of variance” (karagöz, 2019, p.445). accordingly, the handwriting speed of primary and secondary school students (4th–8th grades) wasthe dependent variable, and with grade levels as covariate, students’ gender, hand preferences, and handwriting styles were dealt with as independent variables. 128 december 2022, volume 15, issue 2, 123-136 firstly, among the total 341 student forms that were collected, 19 illegible oneswere removed from the data set. therefore,the analyses were carried out with 322 students. the spss software was used in the data analysis. in the analyses made, the cronbach alpha values of .05 and .01 were taken as the levels of significance. the suitability of the data forparametric testswas examined in terms of normal distribution (skewness and kurtosis values should be between -1 and +1). skewness and kurtosis values between -1 and +1 are the most important indicators of mutually overlapping (equal or nearly equal) normal distribution of the arithmetic mean, mode, and the median (can, 2013, pp.82-89). accordingly, the arithmetic mean and the standard deviation values for the data were examined, and the skewness and kurtosis coefficients were taken into account. the measures of central tendency and normality values related to the data are presented in table 3. table 3 measures of central tendency and normality values ofthedata grade n lowest highest m sd skewness kurtosis 4 84 33 99 59.50 14.81 .389 -.279 5 65 25 132 76.08 25.5 .352 -.458 6 71 28 144 94.46 23.30 -.162 .121 7 59 41 159 104.63 26.78 -.007 -.479 8 53 49 161 116.79 30.11 .541 .567 asdemonstrated in table 3, the skewness and kurtosis values for the handwriting speeds according to grade level are between -1 and +1, and thus, thescores in the data set for the students show normal distribution (tabacnick & fidel, 2015). further, the test results for the homogeneity of variance are presented in table 4. table 4 results of levene’s test for homogeneity of variance dependent variable sd1/sd2 f p handwriting speed 33/288 1.22 .20 table 4 reveals that according to the results of levene’s test for determining homogeneity of error variances, there is no significant difference between the groups in the distribution of error variances of the dependent variables (f = 1.22, p >.05), and the variances are homogeneous. in this case, to ascertain whether there is a significant difference between the mean values of handwriting speed (the dependent variable) according to the factors of the independent variables, two-way anova can be performed. findings in this section, the descriptive findings related to the independent variables of grade level, gender, hand preference, and handwriting style, and the two-way anovafindings for mean handwriting speeds are presented by taking the research question as the basis. before the text was copied, it was stated that students could use any writing style (cursive or manuscript) they wanted and that they had one minute to write. in table 5, the arithmetic mean and standard deviation values for the grade level, gender, hand preference, and handwriting style variables for handwriting speed are presented. as table 5 shows, the general findings for handwriting speed reveals thatthe mean values related to handwriting speed increase as the grade level increases, mean values related to handwriting speed also increase. moreover, although the speedvaries according to grade level, it can be said that girls write faster than boys, and right-handed students write faster than left-handed students. it is also striking that apart from 4thgrades, students who write with manuscript handwriting have higher mean handwriting speeds than those who write with cursive handwriting. following the descriptive findings, the results of the two-way anova, which was performed to holistically evaluate the significance and size of the effects of the independent variables on the dependent variable, are given in table 6. an examination of table 6 shows that the effect of the grade level variable on handwriting speed is significant (f(4, 314) = 35.38, p < .01) and that the effect on handwriting speed is h2=.31. it can be said that grade level is the most effective variable, explaining 31% of the variance in handwriting speed. according to the results of tukey’s multiple comparison test, which was performed to determine in which grades this effect achieved was significant, it was determined that the difference in students’ handwriting speed in the 4th grade (m = 59.50) and 5th grade (m = 76.07) was significant both between themselves and amongthe other grades. however, it was revealed that the difference between the 6th grade (m = 94.46) and7th grade (m = 104.62) and the difference between the 7th grade (m = 104.62) and 8th grade (m = 116.79) was not significant. this situation shows that handwriting speed increases from the 4th grade onwards, but the difference between some grades is not statistically significant. furthermore, it can be seen in table 6 that the effect of gender on handwriting speed is significant (f (1, 314) =39.29, p < .01), and the effect on handwriting speed is h2=.11. when the mean values included in table 5 are examined, it is seen that female students’ mean handwriting speeds are higher than male students’ at all grade levels. this situation shows that gender is an effective variable that explains 11% of the variance in handwriting speed. it can be understood from table 6 that another variable having an effect on handwriting speed is the handwriting 129 handwriting speeds of 4th–8th grade students / yekeler, yıldız, aktaş & ataş style, with an effect of f (1, 314) = 16.25, p < .01. it is seen that handwriting style explains 5% (h2=.05) of the variance in handwriting speed. when the mean values included in table 5 are considered, students who write with a manuscript handwriting style write faster than those who write with a cursive handwriting style (at all grade levels except for the 4thgrade). however, the effect of hand preference,another independent variable of the study, on handwriting speed is not significant (f (1, 314) = 3.78, p >.01). even though it is seen that right-handed students write faster than left-handed students according to the mean values included in table 5, this difference is not statistically significant. based on these findings, the independent variables of grade level, gender, and handwriting style have separate, significant effects on mean handwriting speeds. when grade level is assigned as a covariate, however, it is seen that grade level and gender jointly have a significant effect on mean handwriting speeds (f (4, 314) = 3.53, p =< .05), whereas grade level with hand preference (f (4, 314) = 1.21, p => .05) and grade level with handwriting style (f (4, 314) =1.68, p => .05) do not have a significant joint effect on mean handwriting speeds. table 5 students’ mean handwriting speeds according to grade level, gender, hand preference, and handwriting style grade gender hand preference handwriting style total female male right left cursive manuscript 4 n 41 33 64 10 61 13 74 m 61.41 57.12 59.51 59.40 59.50 59.46 59.50 sd 13.87 15.78 15.23 12.40 14.66 16.09 14.80 5 n 30 35 52 13 33 32 65 m 81.50 71.42 76.69 73.61 65.42 87.06 76.07 sd 22.97 27.05 26.06 24.18 21.84 24.72 25.56 6 n 44 27 62 9 13 58 71 m 102.40 81.51 95.98 84.00 81.15 97.44 94.46 sd 19.85 22.98 23.16 22.71 28.13 21.22 23.29 7 n 29 30 53 6 13 46 59 m 120.06 89.70 106.41 88.83 91.76 108.26 104.62 sd 19.89 24.15 26.16 29.47 24.67 26.47 26.78 8 n 26 27 50 3 6 47 53 m 128.23 105.77 118.50 88.33 95.16 119.55 116.79 sd 29.95 26.33 29.88 19.85 15.01 30.51 30.10 to ta l n 170 152 281 41 126 196 322 m 95.79 79.82 90.08 75.73 68.32 101.07 88.25 sd 32.13 28.45 32.03 23.72 22.76 29.54 31.43 table 6 two-way anova related to the effects of grade level, gender, hand preference, and handwriting style on handwriting speed variable sum of squares df mean square f p h2 grade level 68235.16 4 17058.79 35.38 .00 .31 gender 1822.54 1 18944.61 39.29 .00 .11 hand preference 1822.54 1 1822.54 3.78 .06 .05 handwriting style 7835.69 1 7835.69 16.25 .00 .05 grade level x gender 7008.92 4 1752.23 3.53 .01 .04 grade level x hand preference 2780.22 4 695.05 1.21 .31 .02 grade level x handwriting style 3639.05 4 909.76 1.68 .15 .02 error 151370.80 314 482.07 total 2825142.00 322 *p>.05 130 december 2022, volume 15, issue 2, 123-136 discussion as a result of this study, which was carried out with the aim of examining primary school students’ handwriting speeds according to somevariables, it was concluded that 4th–8th grade students’ mean handwriting speeds differed significantly depending on grade level, gender, and handwriting style, whereas they did not differ significantly according to hand preference. the research findings reveal that primary school (4th–8th grade) students’ handwriting speeds differed significantly depending on their grade level. with respect to the multiple comparison test, the difference between all the other grades was significant except forthe difference between the 6th and 7th grades, and the 7th and 8th grades, whichdid not differ significantly regarding their handwriting speed. these differences according to grade levels can be interpreted as an indication that students’ writing speed development is linear as well as the fact that writing tasks at secondary school level are mostly being replaced by test and exam mindsets with less focus on writing.this also emphasises the importance of the attention paid by primary school and classroom teachers on the development of writing skills because when children reach the age of 8–10, their handwriting skills usually become automatic, and they can be used as a tool to organise their ideas well and facilitate the development of their thoughts (julius et al., 2016). the primary school level is critical to achieve this and to acquire the ability to read and write at an appropriate writing speed. the significant difference in students’ handwriting speeds in terms of grade level shows similarity with some previous studies (e.g., graham et al., 1998; hamstra-bletz&blote, 1990; phelps, stempel, & speck, 1985; skar et al., 2021; ziviani,1984). in the study by graham et al. (1998), in which they examined the mean handwriting speeds of 900 students between the 1st and 9th grades, it was reported that the handwriting speed showed a tendency to increase from one grade to the next. in a longitudinal study by hamstra-bletz and blote (1990), in which they conducted five copying activities with 127 dutch children in three groups, it was reported that between the2ndgrade and 6thgrade, students’ mean handwriting speeds increased from 24 letters to 66 letters per minute. in a cross-sectional study conducted by phelps, stempel, and speck (1985) in the usa, 1365 children were required to copy a paragraph in their own handwriting with a pencil onto unlined paper for a period of 2 minutes, and it was seen that there were significant differences in mean handwriting speeds from the 3rdgrade (25 letters per minute) up to the 8thgrade (72 letters per minute). in australia, ziviani (1984) conducted a cross-sectional study with 575 children who copied expressions such as “cat and dog” for a period of 2 minutes “as quickly as possible,” and it was determined that between the3rdgrade and the 7thgrade, the mean handwriting speeds increased from 33 letters to 52 letters per minute and that this increase was also linear. on the other hand, the finding in the present study that handwriting continued to increase between the 6th, 7th, and 8th grades whereas the difference was not significant corresponds with the results of the study by graham and weintraub (1996), in which it was stated that the relationship between handwriting speed and grade level might not always be linear. similarly, as a result of the study carried out by türker and tunç (2020), it was reported that the mean handwriting speeds of 3rdand 4thgrade primary school students did not differ according to the grade level variable. on the other hand in tseng and hsueh (1997) research that's sample group consists 1525 chinese student between 2nd to 6th grade, pointed that speed of handwriting increase with age and reach its highest level especially at 2nd, 3rd and 4th grades. this may be due to morphological features of chinese. according to akyol (2008) the impact of the first years of writing learning on handwriting speed is great.although languages such as italian, spanish, and turkish are transparent, the fact that the spelling rules are easier in turkish can provide more advantages to students in the writing process in the coming years (jimenez & hernandez-cabrera, 2019). according to the findings in the current study, primary and secondary school (4th– 8th grade) students’ mean handwriting speeds showed significant differences in terms of gender. this result is in favour of female students. this situation suggests that male students should be supported more in order to be able to write at the appropriate speed in writing tasks starting from the pre-school period because unless they can master basic handwriting acquisition, they may be inadequate in writing processes that require high-level cognitive processes, such as planning what to say and how to say it, translating ideas into written text, and reviewing what has been written (fogel, rosenblum, & barnett, 2022).although the differences regarding gender that were found in this present study are supported by some previous studies in the literature (e.g., aydın, 2016; graham, et al., 1998; skar et al., 2021; temur, aksoy, & tabak, 2012), they differ from the findings of other studies (e.g., bay, 2010; türker & tunç, 2020; ziviani & watson-will, 1998). in his study related to the handwriting speeds of 9th grade students who had learnt first literacy with cursive writing, aydın (2016) concluded that female students’ mean handwriting speeds were higher than those of male students in both handwriting styles (manuscript and cursive). moreover, in their study in which they examined the mean handwriting speeds of 900 students from the 1stgrade up to the 9thgrade, graham et al.(1998) revealed that there was a significant difference between male students’ and female students’ mean handwriting speeds and that 131 handwriting speeds of 4th–8th grade students / yekeler, yıldız, aktaş & ataş female students in the 1st, 6th, and 7th grades wrote faster than male students. similarly, in their study, temur, aksoy, and tabak (2012) examined 1st grade students’ ergonomic factors in handwriting together with the handwriting speed, and as a result, it was stated that girls’ handwriting speed was greater than that of boys. in a study examining the effect of the direction of drawing letter lines on handwriting speed and the quality of letter drawing in 1st, 2nd, 3rd, and 4th grade primary school students, başaran (2020) stated that female students drew letter lines faster than male students. on the other hand, in a study carried out with students attending the 1st grade, bay (2010) determined that there was no significant difference in mean handwriting speeds between female students and male students. as a result of the study by ziviani and watson-will (1998), in which they examined the mean handwriting speeds and text legibility of 372 children aged 7–14, it was reported that there was no significant difference between genders with respect to mean handwriting speeds. based on the formation of handwriting skills specific to each gender, it can be said that female students are developmentally better in fine motor skills than male students (unutkan, 2006; başaran, 2020), and this also has an effect on handwriting speed. the findings in the study also revealed that primary and secondary school (4th–8th grade) students’ mean handwriting speeds differed significantly in terms of handwriting styles. among the reasons for this situation, it can be thought that students in turkey do not want to continue with cursive handwriting after primary school, that cursive handwriting is challenging/hard for them, and that they prefer manuscript handwriting with the influence of branch teachers. on the other hand, when grade level was treated as a covariate, whether students used cursive or manuscript handwriting did not have an effect on their mean handwriting speeds. studies conducted about the effect of handwriting style on handwriting speed can be discussed as studies in which the results were in favour of manuscript letters, such as the present study, and studies in which they were not. for example, in his study examining the handwriting speeds of 9th grade students who had learnt first literacy with cursive writing, aydın (2016) revealed that students who wrote with manuscript letters had higher handwriting speeds over a one-minute period than students who used cursive handwriting. in the study by morin, lavoie, and montésinos-gelet (2012), in which they examined letterwriting, wordcopying, and textcreation skills over a 45-minute period in 715 canadian 2nd grade primary school students who learned literacy with different handwriting styles, it was reported that the mean handwriting speeds of students who used mixed and manuscript handwriting styles had greater mean handwriting speeds than those who used cursive handwriting. in their study, bara and morin (2013) examined 4thand 5thgrade students’ handwriting speed, text legibility, and handwriting style according to the handwriting style they had learnt in primary school, and they stated that canadian students wrote faster than french students, and students who used a cursive handwriting style had slower mean handwriting speeds than students who used a mixed handwriting style. on the other hand, in the present study, when the joint interaction of grade level was considered, handwriting style did not have a significant effect on mean handwriting speeds. this finding corresponds with the results of the study carried out with 600 students attending 4th–9th grades by graham, weintraub, and berninger (1998), in which the use of cursive and manuscript letters did not make a difference in terms of handwriting speeds. the findings obtained in this study also revealed that there was no significant difference in primary and secondary school (4th–8th grade) students’ mean handwriting speeds with respect to their hand preferences. it did not make a significant difference to mean handwriting speeds if students wrote by using their right or left hand. in studies carried outin relation to the effect of hand preference on handwriting speed, it is striking that there are both similar and different findings. for example, in their study, in which they examined the mean handwriting speeds of 900 students between the 1st and 9th grades, graham et al. (1998) stated that students who wrote with their right hand wrote faster than students who wrote lefthanded, whereas o’mahony, dempsey, and killeen (2008) reported that hand preferences (right or left) of 607 male and 617 female students, with ages ranging between 8 and 18, had no effect on handwriting speed. similarly, when the joint interaction of grade level was taken into consideration in this current study,the fact that there was no significant difference in mean handwriting speeds between students who used their right or left hands is confirmed withthe findings of some experimental studies (benik, 2018; bonoti et al., 2005; kadıoğlu, 2012; o’mahony, dempsey, & killeen, 2008; türker&tunç,2020; wallen& mackay, 1999). this situation reveals that in the primary school education process in turkey, students should not be forced to use their right hand in the first literacy process sinceusing their right hand or left hand does not affect their handwriting speed. when the studies are evaluated in general, it is seen that there are differences between the effects of the variables of grade level, gender, hand preference, and handwriting style on mean handwriting speedsamong countries. on the other hand, the findings of this study clearly show that mean the handwriting speeds of turkish students increase according to grade level and vary according to gender as well ashandwriting style. this situation reveals that handwriting speed is a skill that changes and continually develops 132 december 2022, volume 15, issue 2, 123-136 together with age, the handwriting style used in the first literacy process should be chosen carefully, and possible effects of differences between genders need to be monitored.in the learning and teaching process, students are given dictation exercises, copying tasks, or writing tasks aimed at producing content-based text. this study of writing speed may contribute to teachers in having more realistic expectations from students about the time required to complete writing tasks. thus, teachers can have an idea about planning more effective writing activities at more appropriate times. in addition, the areas reserved for writing activities in coursebooks and workbooks can be arranged more appropriately by considering the results for writing speed according to grade level. limitations of the research and future directions this study is limited to 322 students attending the 4th– 8th grades of primary and secondary schools in the provinces of ankara, trabzon, and giresun and to the variables of grade level, gender, hand preferences, and handwriting styles for students’mean handwriting speeds. moreover, to seta common task standard for all students in the study, an informative text was used. concerningthe differences in the findings for writing speed,variables such as the individual differences of the students, the number of students participating in the sample, the time, the language characteristics of the country, and the use of different words or texts may be effective (temur, 2012). in addition, it is stated that the time required to write a letter varies according to the shape and size of the letter, and since the shapes of some letters are easier to produce because they are simple, the writing time of each letter is not equal (güneş, 2016). writing is a skill that requires gross and fine motor skills and should be supported in early childhood through play-based learning environments and lessons such as physical education (patiño et al. 2020).in addition to these, it can be said that the method used to measure the writing speed (dictation or copying) is one of the factors that causes differences in writing speed. because in a longitudinal study conducted by gosse, parmentier, & van reybroeck, (2021) with primary school 2nd,3rd,4th and 3rd,4th,5th graders, each child was given a digital tablet and a digital pen to complete a writing task (dictation). as a result of the study, they reported that handwriting speed was associated with high spelling ability and that fast handwriting was detrimental to handwriting quality. thus, while a positive relationship was found between spelling and speed, a negative relationship was found between handwriting speed and handwriting quality. moreover, it is seen that the studies conducted in turkey are with the participation of one or two grade levels. however, since writingspeed is an important factor worth examining on its own, longitudinal studies with different grade levels are needed. in this research, students’ mean handwriting speeds have been examined according to differentvariables. in future studies to be conducted, legibility skills, students’ socio-economic levels, and cognitive and motor factors that are directly and indirectly related to handwriting speed, such as attention deficit and muscle fatigue, can be discussed together with handwriting speed. in this way, whether factors specific to handwriting ability or whether external factors that remain beyond students’ control have a greater effect on handwriting speed can be investigatedin these future studies. furthermore, letter production alongwith motor skills in the development of handwriting skills, the role of primary school, and the effect of teaching students a digital handwriting style on handwriting speed by taking technological developments into consideration can be investigated. conflicts of interest: there are no conflicts of interest. author contributions: all authors contributed equally to this study. references akyol, h. (2008). türkçe i̇lk okuma ve yazma öğretimi [teaching reading and writing in turkish]. ankara: pegema yayıncılık. akyol, h. (2014). programa uygun türkçe öğretim yöntemleri [turkish teaching methods according to the program]. ankara: pegema yayıncılık. arslan, d., & ilgın, h. (2010). öğretmen ve öğrencilerin bitişik eğik yazı ile ilgili görüşleri [teachers’ and students’opinions about cursive handwriting]. i̇nönü üniversitesi eğitim fakültesi dergisi, 11(2), 19-92. https://dergipark.org.tr/en/download/ article-file/92274. aydın, s. (2016). i̇lk okuma yazma öğrenimine doğrudan bitişik eğik yazı ile başlayan 9. sınıf öğrencilerinin bitişik eğik yazıyı kullanma durumlarının incelenmesi [examination of the use of cursive handwriting by 9th grade students who started their first literacy education directly with cursive writing] [master’s thesis, karadeniz teknik üniversitesi, eğitim bilimleri enstitüsü, trabzon]. bara, f., & morin, m. f. (2013). does the handwriting style learned in first grade determine the style used in the fourth and fifth grades and influence handwriting speed and quality? a comparison between french and quebec children. psychology in the schools,50(6),601-617.https:// onlinelibrary.wiley.com/ doi/epdf/10.1002/ pits.21691. 133 handwriting speeds of 4th–8th grade students / yekeler, yıldız, aktaş & ataş barnett, a. l., prunty, m., & rosenblum, s. (2018). development of the handwriting legibility scale (hls): a preliminary examination of reliability and validity. research in developmental disabilities, 72, 240–247. https://doi.org/10.1016/j. ridd.2017.11.013. başaran, m. (2020). the relationship between the direction of drawing the letter lines and the drawing speed and the line quality. pamukkale university journal of education, 52, 86-118. doi:10.9779.pauefd.682929. accessed from https://dergipark.org.tr/ en/download/articlefile/1367279 on 02.01.2021. bay, y. (2010). ses temelli cümle yöntemiyle ilk okuma yazma öğrenen ilköğretim birinci sınıf öğrencilerinin okuma yazma hızları ve okuduğunu anlama düzeyleri [reading and writing speeds and reading comprehension levels of primary school first grade students who learned first reading and writing with the sound-based sentence method]. journal of kirsehir education faculty, 11(1), 257-277. https://dergipark.org.tr/en/download/articlefile/1493954. benik, t. (2018). i̇lkokul birinci sınıf öğrencilerinin yazma becerilerinin ve yazma hızlarının incelenmesi [study of writing skills and writing speed of primary school fırst grade students] [unpublished master’s thesis, gaziantep university. gaziantep]. berninger, v. w., abbott. r. d., jones. j., wolf, b. j., gould, l., anderson, m., shimada, s., & apel, k. (2006). early development of language by hand: composing, reading, listening and speaking connections; three letterwriting modes; and fast mapping in spelling. developmental neuropsychology, 29, 61-92. https://www.tandfonline.com/doi/abs/10.1207/ s15326942dn2901_5. berninger, v. w., & amtmann, d. (2003). preventing written expression disabilities through early and continuing assessment and intervention for handwriting and/or spelling problems: research into practice. in h. l. swanson, k. r. harris, & s. graham (eds.), handbook of learning disabilities (pp. 345-363). guilford press. blote. a & hamstra-bletz. l. (1991). a longitudinal study on the structure of handwriting. perceptual and motor skills, 72, 983-994. https://journals. sagepub.com/doi/10.2466/pms.1991.72.3.983. bonoti, f., vlachos, f., & metallidou, p. (2005). writing and drawing performance of school age children: is there any relationship? school psychology international, 26 (2), 243– 255. https://journals.sagepub.com/doi/ abs/10.1177/0143034305052916. can, a. (2013). spss ile bilimsel araştırma sürecinde nicel veri analizi [quantitative data analysis in the scientific research process with spss]. ankara: pegem akademi. cohen, m. r. (1997). individual and sex differences in speed of handwriting among high schoolstudents. perceptualand motor skills, 84, 1428-1430. https://journals.sagepub.com/doi/ abs/10.2466/pms.1997.84.3c.1428. çelenk, s. (ed.). (2013). i̇lk okuma yazma programı ve öğretimi [first literacy program and its teaching]. ankara: eğiten kitap. dodd, b., & carr, a. (2003). young children’s letter-sound knowledge. language, speech and hearing services in schools, 34, 128-137. https://pubs.asha. org/doi/abs/10.1044/0161-1461%282003/011%29. duran, e., & akyol, h. (2010). bitişik eğik yazı öğretimi çalışmalarının çeşitli değişkenler açısından incelenmesi [examination of cursive handwriting studies in terms of various variables.]. türk eğitim bilimleri dergisi, 8(4), 817-838. https://dergipark. org.tr/en/download/article-file/256228. ediger, m. (2001). assessing handwriting achievement. eric document reproduction service no. ed 346 082. 1-11. accessed from https://files.eric. ed.gov/fulltext/ed447508.pdf on 30 october 2020. erdoğan, t. (2012). i̇lköğretim birinci sınıf öğrencilerinin bitişik eğik yazı yazma gelişimlerinin incelenmesi [an investigation on the development of cursive handwriting of primary school first year students]. eğitim ve bilim, 37(165), 93-103. http://eb.ted.o rg.tr/ index . php/e b/ar ticle/ view/1009/415. feder, k. p., & majnemer, a. (2007). handwriting development, competency, and intervention. developmental medicine & child neurology, 49(4), 312–317. https://doi.org/10.1111/j. 14698749.2007.00312.x fogel, y., rosenblum, s., & barnett, a. l. (2022). handwriting legibility across different writing tasks in school-aged children. hong kong journal of occupational therapy, 35(1), 1–8. https://journals.sagepub.com/doi/ pdf/10.1177/15691861221075709. 134 december 2022, volume 15, issue 2, 123-136 foster, e. m. (1957). a comparison of intermediate grade manuscript and cursive handwriting in two typical elementary school programs [doctoral dissertation, state university of iowa, iowa]. galanis, j. a. (2008). the effect of the handwriting without tears program on student cursive writing achievement at central institute for the deaf (cid)[master’s thesis in science in deaf education, washington university school of medicine program in audiology and communication sciences]. gates, a. i. & brown, h. (1929). experimental comparisons of print-script and cursive writing. the journal of educational research, 20(1), 1-14.https://www. tandfonline.com/doi/abs /10.1080/0022067 1.1929.10879960?journalcode=vjer20. gosse, c., parmentier, m., & van reybroeck, m. (2021). how do spelling, handwriting speed, and handwriting quality develop during primary school? cross-classified growth curve analysis of children's writing development. frontiers inpsychology, 12, 1-19. https://www.frontiersin. org/articles/10.3389/fpsyg.2021.685681/full. graham, s., & miller, l.(1980). handwriting research and practice: a unified approach. focus on exceptional children, 13, 1-16. graham, s. (1992). issues in handwriting instruction. focus on exceptional children, 25(2), 1-16. graham, s., & weintraub, n. (1996). a review of handwriting research: progress and prospects from 1980 to 1994. educational psychology review, 8(1), 7-87. https://link.springer.com/ article/10.1007%2fbf01761831. graham, s., weintraub. n., & berninger. v. w. (1998). the relationship between handwriting style and speed and legibility. the journal of educational research, 91(5), 290296. https://www.tandfonline.com/doi/ abs/10.1080/00220679809597556 graham, s., berninger, v., weintraub, n., & schafer, w. (1998).development of handwriting speed and legibility in grades 1-9. the journal of educational research, 92(1), 42-52. https://www.tandfonline. com/doi/abs/10.1080/00220679809597574. güneş, f. (2006). niçin bitişik eğik yazı [why cursive handwriting?]. meb bilim ve aklın aydınlığında eğitim dergisi, 71(1), 17-19. güneş, f. (2007). ses temelli cümle yöntemi ve zihinsel yapılandırma [sound-based sentence method and mental structuring]. ankara: nobel yayınları. güneş, f. (2016). eğitimde kalem ve tuşlarla yazma tartışmaları [the discussion on writing with pencil and typing on keyboard in education]. bartın üniversitesi eğitim fakültesi dergisi, 5(1), 19-33. güneş, f. (2019). i̇lk okuma ve yazma öğretimi yaklaşımlar ve modeller [first reading and writing teaching approaches and models]. ankara: pegem. hammerschmidt, s. l., & sudsawad, p. (2004). teachers’ survey on problems with handwriting: referral, evaluation, and outcomes. american journal of occupational therapy, 58(2), 185–192. https:// doi.org/10.5014/ajot.58.2.185. hamstra-bletz, l., & blote, a. w. (1990). development of handwriting in primary school: a longitudinal study. perceptual and motor skills, 70, 759-770. https://journals.sagepub.com/doi/abs/10.2466/ pms.1990.70.3.759. houston, h. (1938). manuscript writing and progress in reading. the elementary school journal,39(2),116-118.https://www. j o u r n a l s . u c h i c a g o . e d u /d o i /a b s / 1 0.1 0 8 6 / 457577?journalcode=esj. julius, m. s., meir, r., shechter-nissim, z., & adi-japha, e. (2016). children’s ability to learn a motor skill is related to handwriting and reading proficiency. learning and individual differences, 51, 265–272. https://doi.org/10.1016/j.lindif. 2016.08.034. jimenez, j. e., & hernandez-cabrera, j. a. (2018). transcription skills and written composition in spanish beginning writers: pen and keyboard modes. reading and writing: an interdisciplinary journal, 32(7), 1847–1879. https:// doi. org/ 10. 1007/ s111450189928-4. kadıoğlu, h. (2012). i̇lköğretim birinci sınıf öğrencilerinin bitişik eğik yazma becerisi hızı ve tutumlarının incelenmesi [examining the cursive handwriting speed and attitudes of first grade primary school students] [doctoral thesis, marmara university, i̇stanbul]. karagöz. y. (2019). spss, amos, meta uygulamalı istatistiksel analizler [spss, amos, meta applied statistical analysis] (2nded.). ankara: nobel akademi. karlsdottir, r., & stefansson, t. (2003). predicting performance in primary school subjects. perceptual and motor skills, 97,1058 1060. https://journals.sagepub.com/doi/abs/10.2466/ pms.2003.97.3f.1058. 135 handwriting speeds of 4th–8th grade students / yekeler, yıldız, aktaş & ataş kazu, h., & ersözlü, n. z. (2006). yeni müfredatla gelen bitişik eğik yazıya i̇lişkin öğretmen görüşleri [teachers’ views regarding the cursive handwriting coming with the new curriculum], in ulusal sınıf öğretmenliği kongresi bildiriler kitabı (pp. 238-248). ankara: kök yayıncılık. koenke, k. (1986). handwriting instruction: what do we know? eric digest. https://files.eric.ed.gov/ fulltext/ed272923.pdf. meb, (2009). i̇lköğretim türkçe dersi (1-5.sınıflar) öğretim programı ve kılavuzu [primary turkish course (grades 1-5) curriculum and guide.]. ankara: devlet kitapları müdürlüğü basım evi. medwell, j.,strand. s., & wray, d. (2009). the links between handwriting and composing for y6 children. cambridge journal of education, 39, 329 344. https://www.tandfonline.com/doi/ abs/10.1080/03057640903103728 meulenbroek, r., & van galen, g. (1990). perceptualmotor complexity of printed and cursive letters. the journal of exrperimentu1 education, 58, 95110. https://www.tandfonline.com/doi/abs/10.1 080/00220973.1990.10806527 mccutchen, d. (2011). from novice to expert: implications of language skills and writingrelevant knowledge for memory during the development of writing skill. journal of writing research, 3, 51 -68. https://www. jowr.org/articles/vol3_1/jowr_2011_vol3_nr1_ mccutchen.pdf morin, m. f., lavoie. n., & montésinos-gelet, i. (2012). the effects of manuscript-cursive or manuscript/ cursive styles on writing development in grade 2. language and literacy, 14(1), 110-124. https:// journals.library.ualberta.ca/langandlit/index. php/langandlit/article/view/11028/13067 o’mahony, p., dempsey, m., & killeen, h. (2008). handwriting speed: duration of testing period and relation to socio-economic disadvantage and handedness. occupational therapy international, 15(3), 165-177. https://onlinelibrary. wiley.com/doi/epdf/10.1002/oti.255 patiño, j. f., calixto, a. l., chiappe, a., & almenarez, f. t. (2020). ict-driven writing and motor skills: a review. international electronic journal of elementary education, 12(5), 489–498. retrieved from https://www.iejee.com/index.php/iejee/ article/view/1074. patton, m. q. (2014). qualitative research & evaluation methods: integrating theory and practice (4thed.). thousand oaks, ca:sage publications. phelps, j., stempel, l., & speck, g. (1985). the children’s handwriting scale: a new diagnostic tool. the journal of educational research, 79(1), 46-50. https://www.tandfonline.com/doi/abs/10.1080/ 00220671.1985.10885646. skar, g. b., lei, p. w., graham, s., aasen, a. j., johansen, m. b., & kvistad, a. h. (2021). handwriting fluency and the quality of primary grade students’ writing. reading and writing, 35(2), 509-538. sönmez, v. &alacapınar, f. g. (2017). scientific research methods (5th edition). ankara: anı. tabachnick, b. g., & fidell, l. s. (2015). çok değişkenli istatistiklerin kullanımı [using multivariate statistics] (m. baloğlu, trans.). ankara: nobel. temur, t. (2012). türk ve amerikan 3. sınıf öğrencilerinin yazma hızlarının bazı değişkenler açısından karşılaştırılması [comparison of writing speed of turkish and american 3rd grade students in terms of certain variables]. mustafa kemal üniversitesi sosyal bilimler enstitüsü dergisi, 9(17), 207-218. https://dergipark.org.tr/en/download/ article-file/182965. temur, t., aksoy, c. c., & tabak, h. (2012). i̇lköğretim birinci sınıf öğrencilerinin kalem tutma biçimleri ve kavrama-sıkıştırma kuvvetlerinin yazma hızı ve hatalarına etkisi [the impact of holding and gripping a pencil and compressive strength on writing speed and mistakes of first grade students]. eğitim ve bilim,37(165), 307319. http://eb.ted.org.tr/index.php/eb/article/ view/1318/430. tseng, m. h., & hsueh, i. p. (1997). performance of schoolaged children on a chinese handwriting speed test. occupational therapy international, 4 (4), 294-303. https://onlinelibrary.wiley.com/doi/ epdf/10.1002/oti.61. tompkins, g., campbell, r., green, d., & smith, c. (2014). literacy for the 21st century. australia: pearson. turan, m. (2010). analogy of lower and capital letters of vertical and italic handwriting in first grade literacy. educational research and review, 5 (10), 593-603. https://academicjournals.org/ journal/err/article-full-text-pdf/46a38454194. unutkan, ö. (2006). okul öncesi dönem çocuklarının ince motor becerileri açısından ilköğretime hazır bulunuşluğunun incelenmesi [a study of preschool children’s readiness for primary education in terms of fine motor skills]. sakarya üniversitesi eğitim fakültesi dergisi, 11, 92-99. https://dergipark.org.tr/en/download/articlefile/115724. 136 december 2022, volume 15, issue 2, 123-136 vinter, a., & chartrel, e. (2010). effects of different types of learning on handwriting movements in young children.learning and instruction, 20(6), 476-486. https://www.sciencedirect.com/ science/article/abs/pii/s0959475209000681. yıldız, m., & ateş, s. (2010). i̇lk okuma yazmayı farklı yöntemlerle öğrenen ilköğretim 3. sınıf öğrencilerinin yazılarının okunaklılık ve yazım hataları bakımından karşılaştırılması [comparison of the writings of 3rd grade students who acquired first reading and writing in different ways, in respect to their legibility and writing errors]. türkiye sosyal araştırmalar dergisi, 14(1), 11-30. https://www.acarindex.com/ dosyalar/makale/acarindex-1423935145.pdf. yıldız, m., ataş, m., yekeler, a. d., & aktaş, n. (2016). dik temel yazıyı nasıl öğrendiler? nasıl yazıyorlar? [how did they learn the manuscript writing? how do they write?]. turkish studies international periodical for the languages, literature and history of turkish or turkic, 11(3), 2461-2480. https:// w w w. re s e a rc h g a te. n et/ p rof i l e/ m u sta fa yildiz-17/publication/301343295_dik_temel_ ya z i y i _ n a s i l _o g re n d i l e r_ n a s i l _ya z i yo r l a r/ links/5773dd1808aead7ba06e595e/dik-temelyaziyi-nasil-oegrendiler-nasil-yaziyorlar.pdf. yıldız, m. (2019). türkiye’de bitişik eğik yazı reformu ve sonrası: i̇lköğretim öğrencileri (4-8. sınıf) hangi el yazısı stiliyleyazıyor? [cursive handwriting reform and its aftermath in turkey: which handwriting style do primary school (4th-8th grades) students use?]. eğitim ve bilim, 44(197), 209-222. http://egitimvebilim.ted.org.tr/index. php/eb/article/view/7931/2872 wallen, m., & mackay, s. (1999). test-retest, interrater, and intrarater reliability, and construct validity of the handwriting speed test in year 3 and year 6 students. physical and occupational therapy in pediatrics, 19(1), 29–42. https://www.tandfonline. com/doi/abs/10.1080/j006v19n01_03. ziviani, j. (1984). some elaborations on handwriting speed in 7-14 year olds. perceptual and motor skills, 58, 535-539. https://journals.sagepub.com/ doi/abs/10.2466/pms.1984.58.2.535. ziviani, j. & elkins, j. (1984). an evaluation of handwriting performance. educational review, 36, 251–261. https://www.tandfonline.com/doi/ abs/10.1080/0013191840360304. ziviani, j., & elkins, j. (1986). effect of pencil grip on handwriting speed and legibility. educational review, 38(3), 247-257. https://www.tandfonline. com/doi/abs/10.1080/0013191860380305. ziviani, j. (1996). use of modern cursive handwriting and handwriting speed for children aged 7 to 14 years. perceptual and motor skills, 82(1), 282. https://journals.sagepub.com/doi/abs/10.2466/ pms.1996.82.1.282?journalcode=pmsb. ziviani, j., & watson-will, a. (1998).writing speed and legibility of 7–14-year-old school students using modern cursive script. australian occupational therapy journal, 45(2), 5964. https://onlinelibrary.wiley.com/doi/ abs/10.1111/j.1440-1630.1998.tb00783.x. ziviani, j. m., & wallen, m. (2006). the development of graphomotor skills. a. hendersonand c. pehoski (eds.), hand function in the child: foundations for remediation (pp.217-236). philadelphia, usa: mosby elsevier. appendix “located on two continents,turkey is an idyllic country in which the four seasons are experienced in full. our country is surrounded by seas on three sides, and generously offers you lovely, sundrenched beaches. it is extremely rich in terms of its vegetationcover and animal species. these lands seem always to re-live the past with their mosques, churches, statues, palaces, and madrasas.” international electronic journal of elementary education, 2015, 8(2), 267-286 issn:1307-9298 copyright © iejee www.iejee.com the arts in and out of school: educational policy, provision and practice in ireland today siobhán dowling long  university college cork, ireland. abstract the debate relating to the place and value of the arts in irish education is one that has dominated educational policy, provision, and practice down through the history of irish educational policy from the late nineteenth century to the present day. indeed, interest in this topic has been reignited with the recent publication of two educational policy documents, one based on the arts-ineducation in and out of school the arts in education charter (2013), and the other on the development of children and young people’s literacy and numeracy literacy and numeracy for learning and life: the national strategy to improve literacy and numeracy among children and young people 2011––2020 (2011). despite the irish government’s commitments to promote the arts in and out of school, this paper draws attention to the lack of any real investment in the arts in education charter by the irish government, and the neglect of policymakers to include references to national and international educational research on the value of the arts for enhancing children’s life-long learning. noting the pressures on primary teachers to allocate more time to the literacy and numeracy strategy, it highlights the potential threat of this initiative to the primary school arts education programme. finally, it draws attention to the notable absence of an arts education programme for the majority of senior post-primary pupils who leave school without any in-depth knowledge and appreciation of their rich cultural heritage. this is an area of grave concern, and one that has received very little, if any, attention to date. keywords: arts in education charter, the national strategy to improve literacy and numeracy, arts education, arts-in-education. introduction the arts in education charter was launched in january 2013 by minister for arts, heritage and the gaeltacht, jimmy denihan td and minister for education and skills, mr ruairi quinn td. this landmark document is a joint commitment by the two government departments “to promote both arts education and the arts-in-education among children and young people through the alignment of a joined up, integrated and collaborative approach across government departments, education agencies, and arts organizations”  siobhán dowling long, school of education, university college cork, ireland. phone: +353 21 4903232, e-mail: s.dowling@ucc.ie http://www.iejee.com/ international electronic journal of elementary education vol.8, issue2, 267-286, 2015 268 (cf. arts in education charter website). much of the charter’s content derives from the more extensive report points of alignment (2007), which was produced by the special committee on the arts and education. this report came about after the then minister for the arts, sports and tourism, now arts, heritage and the gaeltacht, with the cooperation of the minister for education and skills, established a special committee on the arts and education to “advise the arts council on how best to align the council’s strategies for the promotion and encouragement of the arts with the priorities of the formal education system” (points of alignment, p. 8). the process of collaboration fulfilled a “long-standing wish” by the arts council to work more formally with the department of education and skills (points of alignment, p. 4). arts council the arts council (an chomhairle ealaíon) is the government’s advisory body on the arts in the republic of ireland. following the findings of the bodkin report on the arts in ireland (1949), it was established in 1951 as a new statutory body under the terms of the first arts act (1951). its function, as broadly defined in each of the three arts acts (1951, 1973, 2003), is to stimulate interest in the arts; to promote knowledge, appreciation and practice of the arts; to assist in improving standards in the arts; and to advise the minister and other public bodies on the arts through various types of funding initiatives, publications of research on the arts, and by undertaking a range of projects to promote and develop the arts, in partnership with others (cf. arts council website). as noted in both points of alignment and the arts in education charter, the arts council has been a primary agent of both policy and practice in the arts-in-education. but while it has supported key arts-inschools resources and programmes, and acted as advocate for arts education, the primary responsibility for “arts education” remains both with the department of education and skills and education providers (points of alignment, p. 3; arts in education charter, p. 3). the arts council published two landmark reports following the second arts act (1973), the first, provision for the arts (1976) known as the “richards report”, and the second, the place of the arts in irish education (1979), known more generally as the “benson report”. the richards report surveyed the state of the arts in ireland up to and including the time of its publication, and concluded with a list of recommendations to enable the irish government to best support the arts in irish society. recommendations relating to arts education included the need to further develop arts education in schools, to include more art subjects to examination level, especially in boys’ schools, and to promote not only the training of practitioners but an understanding and enjoyment of the arts among all irish people (richards report, pp. 115–19, 99). this report was followed three years later by the benson report, a highly significant report that was regarded for many years as the “cornerstone for policy and action by the arts council in the field of education” (points of alignment, p. 3). it highlighted the place as well as the grave neglect of the arts in irish education. the distinction between “arts education” and the “arts-in-education” that is found both in points of alignment (p. 3) and the arts in education charter (pp. 3, 10) only emerged in irish educational policy documents dating from the late 1980s through to the 1990s. the two areas are thus defined: while these are two areas of co-dependency, arts education refers usually to mainstream teaching and learning of the arts as part of mainstream education and learning of the arts as part of general education, while arts-in-education refers mostly to interventions from the realm of the arts into education system, by means of artists of all disciplines visiting schools or by schools engaging with professional arts practice in the public arena. (points of alignment, p.3 /arts in education charter, pp. 3, 10) the arts in and out of school / long 269 recent arts-in-education publications and schemes since the publication of the richards and benson reports, the arts council has published numerous reports on the arts-in-education. most notable among them are arts, education and other learning settings: a research digest (2007) containing information on seventytwo research reports based on the arts-in-education in ireland between 1979-mid 2007; the arts in education directory (2004, updated 2007) a directory of arts organizations working with primary and secondary schools; artist-schools guidelines (2006) a practical set of guidelines produced by the arts council and the department of education and skills to encourage partnerships between artists and teachers; and more recently, a report on early childhood arts, early childhood arts: three perspectives (2013). two very successful schemes funded by the arts council under its arts in schools programmes include the artist in schools scheme run in conjunction with various local authorities, and the writers in schools scheme run by poetry ireland, with the latter featuring as one of the longest running art-in-education programmes in ireland. the national architects in school initiative is another popular scheme available to transition year pupils and their teachers in conjunction with the irish architecture foundation. it is co-funded by the arts council, the department of education and skills and the department of arts, heritage and the gaeltacht in partnership with the six ateci (association of teacher education centres of ireland) regions across ireland. the recent partnership between music generation and the arts council, mentioned in the arts in education charter (arts in education charter, pp. 14, 18, 19), “is aimed at creating better access to high quality performance music education for children and young people” (cf. link to music generation website). initiated by music network and co-funded by u2, the ireland funds, the department of education and skills, and local education partnerships, the new partnership has led to an investment of e450,000 in music generation by the arts council. to date over 18,500 children and young people have accessed quality performance music education with job opportunities created for more than 220 musicians. new programmes of tuition are currently being developed and delivered by local music education partnerships nationwide in carlow, clare, cork city, donegal, laois, louth, limerick city, offaly/westmeath, mayo, sligo, south dublin and wicklow (cf. music generation website). while there are many other exciting schemes available to children and young people, too numerous to mention here, one in particular deserves particular mention, namely fighting words: the write to right (sic), a creative writing centre established by author roddy doyle and seán love (cf. fighting words on tedx dublin). based in dublin’s north inner city since january 2009, it provides free tutoring and mentoring in creative writing and related arts events (e.g. music, song-writing, play-writing, writing for tv, media) to children and young people aged between 6–17 years, and adults with special needs. it fosters creative collaborations with other arts organizations, such as the abbey theatre in dublin, the irish times, the national gallery, the print museum, the science gallery, and brown bag film animation among others, and functions on the goodwill of over 500 volunteers. funded mainly from private individuals and institutions, it receives grant aid for publications, written by its participants, from the arts council and dublin city council arts office. a recent collaboration resulted in an animated film for 9–11 year olds entitled “the weirdest thing in toast and all of mankind”, written, designed and performed by children in collaboration with brown bag films. premiered at galway film fleadh in july 2012, it has been shown at international film festivals to critical acclaim. in the first four years of opening (2009–2013), fighting words hosted over 40,000 children and young people in numerous creative writing workshops, courses, and programmes a testament, no doubt, to the scheme’s success in teaching real skills for life, among them literacy skills. international electronic journal of elementary education vol.8, issue2, 267-286, 2015 270 another very popular scheme that deserves mention is the creative engagement scheme run by the national association of principals and deputy principals (napd). this is an annual arts-in-education scheme, which seeks to foster collaboration between student, teacher and artists as set out in the artist-schools guidelines (arts council, 2006). although not supported by funding from the arts council, the scheme is co-funded by the department of education and skills, the department of arts, heritage and gaeltacht and the heritage council. the projects, which span a variety of art forms, are produced in schools with the assistance of an artist/composer, and later showcased at the collins barracks creative engagement exhibition (cf. creative engagements website). national networks for the arts-in education in ireland, the association for creativity and arts-in-education (acae) is a national network of educators and artists who are concerned with the promotion of creativity and the development of the imagination through the arts. under the patronage of michael d. higgins, president of ireland, this organization celebrates creative teaching and learning with the creative schools award “an evidence-based award that promotes process rather than product and looks to foster and reward creativity, the imagination and the arts.” (cf. acae website). the award is distributed annually to successful primary schools at a presentation ceremony in the ark, dublin. the ark is “a unique, purpose-built cultural centre in the heart of dublin's temple bar, where children aged 2–12 can explore theatre, music, literature, art, film, and more” (cf. the ark website). the new national scheme “arts rich schools” (aris) (cf. arts in education charter, pp. 5, 17), that promises to “incentivise and recognize those schools which makes the arts a key part of school life” bears a close resemblance to the initiative developed by the acae. both the napd and the acae, mentioned above, are members of the umbrella body encountering the arts ireland (etai). launched by the ministers for arts, heritage and the gaeltacht and the department of education and skills on the 27th november 2013, this organization includes representatives from over thirty arts, education, and cultural institutions. as a direct response to the arts in education charter, it represents the colorful array of collaborative partnerships that have been created amongst irish artists, cultural institutions, and educational establishments today. two significant events, which led to the formation of this body, lighting a fire conference (national gallery of ireland, 19th april, 2013), and schools: access to culture conference (national gallery, september, 2013), were initiated to open the space for dialogue on the theme of the arts-in-education. attended by pre-school, primary and post-primary teachers, the aim of the sessions was to discuss the integration of culture and creativity into pre-school, primary and post-primary schoolwork. the discussion also focused on the reality of cutbacks and the lack of resources, which militate against implementing an arts-in-education program for many schools today. although not discussed at the conferences, one wonders why there are so many arts-in-education schemes, many of which are not financed by the department of education and skills? do they illustrate commitment to the arts by the irish government or the reverse? the irish government’s commitments to the arts-in education as noted above, the new collaboration between the department of education and skills and the arts council of ireland has focused attention on the arts-in-education in the educational sector in the republic of ireland. despite the lack of government funding, the arts council’s sterling work over the last number of years will enable the government to promote the “arts-in-education” with relative ease and on-going continued success. as outlined in the arts in education charter, the arts council will assist the government in a variety of ways such as promoting aris i.e. arts rich schools, (pp. 5, 17), and schemes led the arts in and out of school / long 271 by music generation through its newly formed partnership programme (p. 18). it will encourage arts organizations in receipt of government funding to develop policies and programmes in the arts-in-education as part of the new public service education dividend (psed) (pp. 5, 12-13, 17-18). it will foster new provision “especially in parts of the country with little or no arts-in-education service and in art forms or practices where current artsin-education work is under-developed” (p. 18), and review the professional development needs of artists and other art professionals engaged in the arts-in-education. it will also engage with the national council for curriculum and assessment (ncca), as well as acting as a representative on relevant ncca committees (pp. 6, 15, 18). in addition, and as outlined in the government’s commitment to the arts-in-education, the national cultural institutions have ensured that every student will visit a national cultural institution at least once in their second level school career (pp. 5, 13). their commitment to the arts in education charter is outlined in the policy document a fresh view for the 21st century: education, community, outreach policy framework 2014 developed by the education, community, outreach working group of the council of national cultural institutions in association with anne gallagher. the government has also promised that discounted tickets to national cultural institutions will be available to primary, post-primary and third level students (arts in education charter, pp. 5, 13), and that artists’ residencies in colleges of education will increase (pp. 5, 17). there is also an expectation that second level schools will develop school policies and plans that reflect their commitment to the arts-in-education (pp. 6, 14), and a promise that provision for the arts-in-education will be reflected in the design and equipping of schools (p. 6, 15). according to the commitments in the charter, provision for out-of-hours use of school facilities will be available for children and young people accessing arts related activities (pp. 16-17). a national mapping exercise is also underway to give an up-to-date account of arts-in-education activities currently on offer in the republic of ireland. the government has committed to the development of a national programme of local arts education partnerships (p. 16), and the procurement, development and establishment of a new dedicated arts-in-education portal for the purpose of providing a platform for collaborative practice for those working in the arts-in-education and arts education (pp. 5, 15, 19). since the publication of the charter, the minister for education and skills, jan o’sullivan td and minister for arts heritage and the gaeltacht, heather humphreys td launched the new online portal at dublin castle printworks on the 19th may, 2015. supported by the department of education and skills and the department of arts, heritage and gaeltacht, it was developed by kids’ own publishing partnership, in consultation with an editorial committee, as a national digital resource of arts and education practice for teachers, artists, anyone working in the field of arts-in-education. the site provides space for the sharing of good practice, research, resources and details of case studies, along with calls for applications for initiatives involving various partners (cf. arts in education portal). despite the many initiatives to promote the arts-in-education, and the charter’s frequent references to arts education (pp. 3, 4, 7-9, 11, 14, 16, 18, 20), arts education in schools has received very little attention to date. factors such as limited investment in arts education, the emphasis on literacy and numeracy, standardized testing, and the failure to fully integrate arts education into the post-primary curricula have reduced arts education to the periphery of irish education. at a european level, it has been noted in the compendium of cultural policies and trends in europe (council of europe and the ericarts institute) that the arts in education charter “has occasioned mixed responses” in ireland …. and that it suffers from a “lack of commitment to real investment”: international electronic journal of elementary education vol.8, issue2, 267-286, 2015 272 the arts in ireland were and still are the domain of the privileged or the lucky. the new arts in education charter, which might have the potential to achieve greater spread for the arts and to improve access, has occasioned mixed responses, some commentators feeling that it lacks commitment to real investment and relies overly on publicly funded artists and organisations, requiring them to donate time to education projects (cf. link to website of the council of europe and the ericarts institute). at the launch of the arts in education charter, prof. john coolahan, chairman of the high-level implementation committee, also noted the worrying lack of investment in the arts in education charter by the irish government (cf. link to the irish times). by comparison, the national strategy to improve literacy and numeracy among children and young people has received considerable financial support from the irish government probably to the detriment of arts education programmes in some schools today. the irish government’s commitments to arts education? as witness to this, a key pillar of the programme for government, and a national priority for the minister for education and skills, the national strategy has called for the improvement of children’s literacy and numeracy through “a strong focus on literacy and numeracy skills within a broad and balanced curriculum” (national strategy, p. 10). as a vital part of ireland’s labour market policy, the national strategy aims to achieve smart growth by 2020 as part of a wider european strategy europe 2020, i.e. a ten year jobs and growth strategy for europe, which set ambitious objectives for smart, inclusive and sustainable growth for the european union. key concerns of this international strategy include literacy, numeracy, science and technology (cf. european commission, “education and training”). the national strategy to improve children and young people’s literacy and numeracy has been supported by generous funding of €9 million from budget 2014 and €13.8 from budget 2015. the ambitious targets are set to be achieved through the administration of annual standardized tests of reading and mathematics for primary school children in 2nd, 4th and 6th classes and for young people in 2nd year post-primary school. the results of the assessments will be reported to parents, boards of management, to other schools on the transfer of pupils to another primary school, and on transition of every pupil to postprimary school. in addition, schools are required to report the aggregate standardized test results to the des once annually. despite the intention of the des not to publish data for the compilation of league tables, there is some anxiety among teachers on the ground that government officials could use data from the standardized test results to assess teacher performance, especially those with poor records of standardized test results. an american study into the perception of teaching in an environment of standardized testing by franklin and snow-gerona (2007) indicated that “increased emphasis on preparing students to test well had occurred in 93% of the classrooms” and “a lack of student choice in terms of curricular decisions had increased in more than 50% of the classrooms” (franklin and snow-gerona, p. 12). in addition, a segment from an interview with one teacher revealed how some teachers had cut back on science, health, social studies and art to focus on the content that was tested. the teacher expressed a fear that pupils would end up only knowing curricular content that was the subject of the test, and remain ignorant of other curricular content necessary for the development of important life-skills (franklin and snow-gerona, p. 12). in a background report on cultural education: the promotion of cultural knowledge, creativity and inter-cultural understanding through education (parliamentary assembly council of europe, 2008), barbara putz-plecko has discussed the importance of children and young people’s aesthetic education and education in the arts. she noted that one potential consequence of an education that reduces or neglects the aesthetic dimension is the arts in and out of school / long 273 the non-development of important competences and qualities necessary for society as a whole (putz-plecko, p. 5). among a long list of outcomes provided by an aesthetic education she highlights the importance of “the faculty of discriminative observation and perception; imagination, inventiveness and vision; creativity; emotional intelligence; a capacity for critical reflection; and the capacity to make decisions independently” (putzplecko, p. 5). putz-plecko attributes the clear hierarchisation of school subjects, exacerbated by the oecd instrument of the pisa study, as inhibiting the aims of cultural education which values creativity, and which can neither be “measured”, “produced according to scale” nor “categorized” (putz-plecko, p. 7). the three rs vs arts education the current focus on standardized tests and teachers possibly teaching-to-the-test call to mind nineteenth century irish educational policy by the board of education in which the promotion of the three rs through standardized tests and payment-by-results contributed to the marginalization of the arts in irish education. while the arts in school were included as part of the curriculum, most notably music and drawing, the narrow approach, which was enacted by teachers under pressure to reach the required standards for fear of losing payment for poor results, contributed to more time dedicated to the teaching of the three rs and less time to teaching and learning in the arts. this outcome had a devastating effect on arts education in ireland resulting in limited government funding and resources for the arts, inadequate teacher training and eventual poor take-up by pupils. in the twenty-first century, the primary school curriculum (1999), lauded for its holistic approach, integrated curriculum, and excellent arts education, now faces a similar threat to its arts programme (cf. hyperlink to irish primary school mathematics curriculum for examples of an integrated curriculum). if “elementary schools have traditionally been places where more creative action and thinking have occurred than in further stages of education” (sahlberg, p. 338), then the increasing focus on standardized testing in irish primary schools will most likely result in teaching and learning that is “less creative and less imaginative, with less focus on collaborative and cooperative learning, less space for teachers to take worthwhile risks, and more intolerance for learning that is wrong” (sahlberg, p. 338). sahlberg points out that the most important enabling factors for schools to be creative are “collaboration, risk-taking, and learning to be wrong” (sahlberg pp. 342–44) enabling factors, i might add, which are not achieved through an emphasis on teaching-to-the-test. the threat to primary school arts education before the introduction of the national strategy to improve literacy and numeracy, an into discussion document on creativity on the arts in the primary school in 2009 noted that the arts were “alive and well in irish schools” (creativity on the arts in the primary school, p. 31). this positive state of health was due in no uncertain terms to the greater emphasis on arts education as a compulsory component of the primary school curriculum a revision of the primary school curriculum that replaced curaclam na bunscoile (1971) in 1999 (cf. fig. 1). international electronic journal of elementary education vol.8, issue2, 267-286, 2015 274 figure 1. organizational framework of the primary curriculum primary school curriculum (p. 40). the irish primary school arts education programme, unlike most primary arts curricula in european schools, includes drama, namely process drama alongside the visual arts and music. the importance of educational drama in the lives of children, and in their learning, is lauded in the primary school curriculum introduction as follows: it involves children’s experiences, concerns and needs, things they imagine or read about, aspects of life from the past, present or possible future that arouse their curiosity, and particular issues the teacher may wish children to explore from any of the other curriculum areas. what is special about children’s learning through drama is that it harnesses children’s imaginative potential and leads them to new knowledge and perspectives not available to them in any other activity. primary school curriculum introduction (p. 54). european experts anne bamford and michael wimmer have noted in their paper, the role of arts education in enhancing school attractiveness: a literature review (2012) that “drama, dance, media arts and architecture are rarely if ever taught in their own right” (p. 5). within a european context then, the emphasis on drama in the irish primary school arts curriculum is a unique feature, and its inclusion, along with music and the visual arts, fosters the aesthetic dimension of every child’s learning: the curriculum enables the child to perceive the aesthetic dimension in every area. this enriches the learning experiences for the child and the different aspects of conceptual development. the uniqueness of the child is perhaps most apparent in the innate creativity of each individual, while valuing the child’s creative response and expression of perceptions, insights, interpretations and knowledge is an important principle of the curriculum (primary school curriculum, p. 15). circular letter (0056/2011) a circular letter (0056/2011) issued to irish primary schools set out the requirements for schools to increase the time allocation for literacy and numeracy within the primary curriculum (cf. circular letter 0056/2011). it mandated an increase of 60 minutes per the arts in and out of school / long 275 week for literacy [i.e. from 5.5 to 6.5 hours per week for infants with a shorter day, and to 7.5 to 8.5 hours per week for students with a full day], and 70 minutes per week for numeracy [i.e. from 2.15 to 3.25 hours per week for infants with a shorter day, and from 3 to 4.10 hours per week for students with a full day]. with effect from 2012, all primary schools are requested to make provision for these arrangements through a variety of approaches that include “integrating literacy and numeracy with other curriculum areas, using some or all of the discretionary curriculum time for literacy and numeracy, reallocating time spent on other subjects to literacy and numeracy, and/or prioritising curriculum objectives which are considered most valuable in supporting children’s learning and delaying the introduction of elements of some subjects” (cf. circular letter 0056/2011). while a more integrated approach to literacy and numeracy would foster the inclusion of other curricular subjects such as music, drama, visual arts and dance and avoid “a narrowing of the curriculum”, the reality for some teachers finding themselves under pressure to achieve “good test results” is likely to result in decisions to teach-to-thetest, using non-creative pedagogical approaches. franklin and snow-gerona referencing amrein & berliner, 2002; birkmire, 1993; darling-hammond, 1991; madaus, 1988; pedulla, 2003 have indicated how the trend toward teaching-to-the-test leads to an emphasis on content that is tested and an increasingly narrow curriculum with less focus on higher-order thinking skills (snow-gerona, p. 3). in ireland, at present, there is a danger that teachers would take time away from the teaching and learning of compulsory core areas, such as music, the visual arts and drama, to focus solely on the task of children’s attainment of literacy and numeracy skills. in the light of the possible reduction in time from the teaching of the various strands of the arts programme from the primary school curriculum, it is worrying then that more attention is not given to the value of arts education in the national strategy. with the exception of two passing references to the arts (national strategy, pp. 44, 56) one is immediately struck by a notable absence of references to the value of learning in and through the arts, which as i point out later are not only proven modes of learning that develop mathematical and linguistic abilities, skills and knowledge but the means to developing and enhancing the capacity to foster creativity, imagination, critical and divergent thinking including intra and interpersonal, spatial and kinesthetic abilities. it is a well-known fact among educational researchers that the arts embrace pupils’ humanity, support social and emotional learning and, depending on the level of teaching and expertise of individual teachers, contribute to the overall happiness of pupils in particular and to the school and wider community in general. lessons from finland: the integration of the arts across the curriculum in finland, arts education is prioritized in children and young people’s education up to the age of 15 or 16 years of age. music, the visual arts, and crafts are included in the compulsory national core curriculum as stipulated by the basic education act (628/1998) (cf. syllabus of the national core curriculum). students also have an opportunity to study dance and drama as part of the respective pe and mother tongue language and literature curricula. the chart below, (cf. fig. 2 and hyperlink to the website of the finnish ministry of education and culture), illustrates the distribution of lesson hours in basic education, and highlights the prominence given to arts education in finland over the first ten years of compulsory education. international electronic journal of elementary education vol.8, issue2, 267-286, 2015 276 figure 2. finnish basic education students also have the option of participating in a non-compulsory basic education curriculum in the arts for a small fee after school. the finnish national board of education decides upon the objectives and core curricular content in music, the literary arts, dance, performing arts [circus and theatre], and visual arts [architecture, audio visual art, visual arts, and craft]. this course of study is designed to develop students’ hobbies in the arts, to support their personal development, and to provide them with knowledge and skills to enable them to apply, if desired, for professional studies within a given art form. the arts are also woven through the national core curriculum for upper secondary school 2003 (cf. national core curriculum for upper secondary school), and feature in an array of compulsory courses, such as finnish culture and literature (fina6) (cf. fig. 3, as found in national core curriculum for upper secondary school, p. 93). this course supports not only the development of literacy, but highlights finnish national identity, integrated with a study of and acquaintance with the work of internationally renowned finnish artists, authors, and composers. finnish culture and literature (fina6) the course will acquaint students with key areas of finnish culture and arts and will also deal with internationally renowned finnish artists and authors through basic texts, works of art or study visits. during the course, students will read at least one free-choice finnish language work and present it either orally or in writing and discuss its content. upon completion of the course, students will be capable of naming a few well-known finnish artists, composers, and authors, describing them briefly and mentioning something about their major works. in terms of grammar, the course will deal with attributes and involve reflection on issues of style and variation relating to texts. (national core curriculum for upper secondary school, p. 93). figure 3. compulsory course on finnish culture and literature (fina6) the arts in and out of school / long 277 unlike the majority of irish post-primary pupils, finnish pupils leave school with an indepth knowledge of their cultural heritage, along with an appreciation of key compositions by finnish composers and works of art by finnish artists. in upper secondary school, students also take compulsory and specialization courses in music and the visual arts. for the latter, the objectives include, among others, “supporting the development of students’ imagination, creative thinking and associative skills”, and “encouraging reflection and justification for pupils’ own aesthetic and ethical value choices in their own lives, the visual arts, the media and the cultural environment” (national core curriculum for upper secondary school 2003, p. 204). by comparison, in ireland at present, the arts (i.e. music and art) are not a compulsory component of the post-primary curriculum, but are selected as optional subjects by a minority cohort of students. as in the past, the poor take-up of arts subjects by irish students has contributed to the marginalization of the arts at postprimary level. professor john coolahan noted in an overview essay “the changing context of the arts in irish education” (appendix. points of alignment) “that there is evidence that a great deal remains to be done so that the arts are genuinely embedded as part of holistic education” (coolahan, p. 38). he pointed out that “of the 171, 313 pupils who sat the junior certificate programme in 2005, only 44% participated in the art and craft programme and only 23% took the music course.” at the established leaving certificate examination, “20% took art and only 4% studied music” (coolahan, p. 38). despite the fact that the statistics are ten years old, they highlight the notable absence of an arts education programme in the final two years of post-primary school for the majority of irish pupils. and while it might be argued that there is a very good arts education programme as part of the leaving certificate applied programme i.e. a two-year programme that prepares students for adult and working life, only 2,902 students sat this examination in 2015 compared to 55,963 students who sat the established leaving certificate (cf. state examinations commission "press release 2015 state examinations). the trend away from arts education in post-primary education was observed by bamford and wimmer who, citing evidence from eurydice 2009 on the arts and cultural education at school in europe, noted how the focus on the arts diminishes as the child moves from primary to post-primary school, how the arts are relegated to the margins of the post-primary curriculum, and how some art forms are not taught at all (bamford and wimmer, p. 8). the present non-compulsory dimension to ireland’s post-primary junior curriculum explains ireland’s low ranking, i.e. 26th place out of 28 countries surveyed in 1996, 2002 and 2009 by the oecd for instruction-time of arts subjects as a percentage of total compulsory instruction time for 12-14 year-olds (cf. oecd, art for art’s sake? the impact of arts education, p. 28). this ranking is set to change however with the introduction of a revised curriculum at junior post-primary level a framework for junior cycle (2012) (cf. department of education and skills, a framework for junior cycle, 2012). while priority will be given to literacy and numeracy, six other key skills for learning, living and working in the 21st century will be named and embedded in every subject, among them “being creative” (cf. key skills of junior cycle and towards a framework for junior cycle). students will choose a maximum of eight to ten full subjects out of a total of twenty-one subjects, and a maximum of four short courses. the ncca is presently developing a number of short courses, one entitled “artistic performance: engagement with the arts” (cf. draft specification for junior cycle short course). this course aims to engage students “in creative activity and artistic practices, by offering a structure that can be used to facilitate imaginative initiatives both individual and collaborative, culminating in a single group production” (ncca, p. 6). non-certified involvement in “other learning experiences and achievements” such as the staging of a drama, musical or show, will also be documented on the school’s “junior cycle achievement profile” (cf. des “briefing note” 4th october, 2012, p. 11). curriculum reform of the traditional established leaving international electronic journal of elementary education vol.8, issue2, 267-286, 2015 278 certificate programme, however, is a long way off, and any discussion of an integrated curriculum involving arts education remains, for the time being, “the stuff of dreams” (shakespeare, the tempest, act 4, scene 1, 156-57). the right of children and young people to a cultural education as outlined in unesco’s road map for arts education: building creative capacities for the 21st century (2007), the basic rationale for making arts education an important and compulsory part of the educational programme in any country emerges from articles 22, 26, and 27 of the international declaration of human rights (1948), and articles 29 and 31 of the united nations convention on the rights of the child (1989). within ireland, the government’s ten year plan for impoving the lives of children, the national children’s strategy: our children their lives (2000) has also highlighted the importance of the arts in children’s lives under the third of three national goals. in this document, objective d (the national children’s strategy, p. 58) points to the rights and entitlements of children to participate in arts and cultural activities with a commitment by the government to increase children’s engagement in the arts through various plans of the arts council. but is this happening? words are certainly happening. for example, the education act (1998), which highlights the arts in and out of education, states that one of the functions of a recognized school is “to promote the development of the irish language and tradition, irish literature, the arts, and other cultural events” (cf. education act, section 9, f ). similarly, in the white paper on education, charting our education future (1995), the government affirmed the centrality of the arts within educational policy and provision, particularly during compulsory schooling (cf. white paper, pp. 22-23). noting the positive benefits of artistic and aesthetic education, it regards the arts as “key elements within the school experience of young people” because an education which includes the arts fosters “a nurturing of creativity” and “assists the young person to become a tolerant, critically aware and socially committed citizen who can live with confidence in the world” (white paper, p. 22). a good arts education, according to the white paper, “develops the imagination as a central source of human creativity, and fosters important kinds of thinking and problem solving, as well as offering opportunities to symbolise, to play and to celebrate” (white paper, p. 22). as noted already, while efforts are being made to increase the provision of arts education at post-primary school through the reform of the junior certificate curriculum, more attention needs to focus on the integration of the arts at leaving certificate level. in reality, however, the rigidity of the leaving certificate programme, and the pressure of the points race have resulted to date in a high percentage of young people leaving school with limited knowledge of their rich cultural heritage. actions not words numerous reports and studies have provided evidence demonstrating the contribution of the arts for learning across other subject areas, most notably for the attainment of literacy and numeracy skills. for example, a well-known report from the united states champions of change: the impact of the arts on learning (1999) edited by edward b. fiske demonstrates how involvement in the arts enables young people to attain higher levels of academic achievement. the study, conducted by seven teams of notable researchers from the united states, examines why and how young people were changed, academically and behaviourally, through their arts experiences. similarly, a report by the centre for arts education staying in school: arts education and new york city high school graduation rates (october, 2009) noted the correlation between exposure to the arts and improved skills in cognition and attention for learning” (cf. hyperlink staying in school: arts education and new york city high school graduation rates, p. 6). furthermore, the dana the arts in and out of school / long 279 consortium report on arts and cognition learning, arts and the brain, conducted by leading cognitive neuroscientists from seven universities across the united states, noted among its many findings correlations between music training and reading acquisition and sequence learning, links between high levels of training in music and acting on memory skills (cf. arts and cognition learning, arts and the brain, pp.v-vi, 11-16), and links between music and mathematics. commenting on this report, mariale hardiman, professor of clinical education and co-founder and director of the school of education’s neuro-education initiative (nei) at john hopkins university stated that the time had come for educational policymakers to pay attention to studies highlighting the influence of the arts on cognition, thinking and learning. given the wealth of neuro-scientific evidence, she has called for such findings to be a part of the research agenda and a central focus in educational policymaking (cf. “commentary: the arts will help school accountability”, mariale hardiman). a more recent report published by the oecd art for art’s sake? the impact of arts education (2013) has demonstrated not only the positive impact of the arts on students’ social and behavioural skills but also the impact of arts education on academic achievement in subjects measured by scores on verbal and mathematical standardized tests. it also pointed to numerous studies which have investigated the impact of students’ participation in theatre education as a means to strengthening students’ verbal skills and improving reading and writing, oral language, vocabulary, and reading readiness. the importance of educational drama in education has also been highlighted in the two-year international eu-supported cross-cultural quantitative and qualitative research project “drama improves lisbon key competencies in education” (dice). this study, which involved 5000 young people aged between 13-16 years of age, demonstrated the positive effects of educational theatre and drama on five of the eight lisbon key competencies, namely, “communication in the mother tongue, learning to learn, interpersonal intercultural and social competences – civic competences, entrepreneurship, and cultural expression.” it mentions an additional competency, “all this and more” not mentioned among the key lisbon competences, and which explores what it means to be human. the project concluded that the six competences are life-learning skills necessary for the personal development of young people, their future employment, and active citizenship (cf. dice consortium, pp. 49–52). the above named international studies and reports illustrate the transferability of learning in and through the arts to other subject areas, most notably for the attainment of literacy and numeracy skills. but despite the insights from educational research, and the government’s acknowledgement of the need to draw upon such research (arts in education charter, p. 15), it is regrettable, then, that there are no references to national and international research findings or to neuro-scientific evidence on the influence of the arts on cognition, thinking and learning either in the arts in education charter or the national strategy for literacy and numeracy. young people’s voices: the need for more arts-based subjects the importance of listening to children’s voices and involving them in national decisionmaking is a topic that is found in numerous irish policy documents, including the national children’s strategy, our children, their lives (2000) (the department of health and children, pp. 29-36), better outcomes, brighter futures: the national policy framework for children and young people, 2014–2020 (pp. vi, vii, 2, 4, 8, 17, 20, 22, 31, 100, 112, 141, 150), and more recently in the action plan for the national strategy on children and young people’s participation in decision-making 2015–20 (department of children and youth affairs, 2015). the latter, which includes over a hundred references to the importance of children’s voices, sets out the commitments and actions of various irish government international electronic journal of elementary education vol.8, issue2, 267-286, 2015 280 departments and agencies to include children’s voices in decision-making. all three documents acknowledge the importance of the arts and culture in children and young peoples’ lives. for example, the national policy framework better outcomes, brighter futures notes that the arts and culture, along with play, recreation, sport and the natural environment, are “essential to the health and wellbeing of children and young people, and promote the development of creativity, imagination self-confidence and self-efficacy, as well as physical, social, cognitive and emotional strength and skills” (better outcomes, brighter futures, p, 55). within education, the report of a consultation with young people on reform of the junior cycle (2011) includes a record of young people’s voices on the reform of the junior cycle. compiled by sandra roe after a consultation with 88 young people, aged 12–18 years, from various comhairlí na nóg around ireland, it includes the findings of students who were currently at junior cycle level and those who had progressed through the junior cycle. key findings revealed that young people enjoyed learning “sports, arts-based activities, and life-skills” because they were “fun, interesting and practical”, and encouraged “creativity and self-expression” (roe, p. 1). the most enjoyed learning in primary school included arts-based activities, followed by sport and playing, and subjects such as irish, history, maths, english, science, religion, french and geography; social skills, such as how to make friends and be part of a team; and cooking” (roe, p. 7). the most enjoyed subjects at junior cert level included history, maths, music and english (roe, p. 7). commenting on the most enjoyed learning in their whole lives, senior cycle students mentioned arts-based activities, such as “art, music, drama and dancing”, followed by “sport, life skills, self-discovery and learning about others, learning a language, creative writing, public speaking and woodwork” (roe, p. 14-15). overall, the young people involved in the day-long consultation called for a wider range of subjects to be available to students at junior cycle level and highlighted “more arts-based subjects and cultural studies” (roe, p. 29). given the prominence to the importance of children’s voices in the above mentioned national policy documents, along with the documentation of children’s voices in roe’s report, it is regrettable then that no references to the children and young people’s voices are included in the arts in education charter or the national strategy to improve literacy and numeracy. recommendations for the future the learning metrics task force (lmtf), which was convened by unesco’s institute for statistics and the centre for universal education at the brookings institution, released recommendations for measuring and improving the learning experiences of children and youth worldwide in a report entitled toward universal learning: recommendations from the learning metrics task force (2013), based on the findings of a series of three reports what every child should learn (no. 1), a global framework for measuring learning (no. 2), and implementing assessment to improve learning (no. 3). the task force called upon all stakeholders working in the field of education, national governments, civil society groups, assessment institutions and universities to carry the recommendations forward into action, and to endorse a broad and holistic definition of learning across the seven domains of learning across three levels of education from early childhood to post-primary school. it is envisioned that the seven learning domains (cf. fig. 4) will be acquired over the course of a child and young person’s education as the means to enabling a person to lead a productive and healthy life, and to acquire a sustainable livelihood. the arts in and out of school / long 281 figure 4. the global framework of learning domains taken from toward universal learning: recommendations from the learning metrics task force, p. 20. with regard to the third learning domain “culture and the arts” there is an expectation that children and young people will have opportunities for creative expression in a wide variety of art forms and cultural experiences to foster knowledge and skills in the area of the creative arts and culture, personal and communal identity, and awareness of and respect for diversity. while it is gratifying to see a reference to culture and the arts alongside numeracy and mathematics, literacy and communication, and science and technology one wonders when the rhetoric of educational policy, both national and international, will become a reality for irish children and young people in and out of school. conclusion despite the government’s lack of funding for targeted investment in the arts in education charter, and the lack of attention given to arts education at senior post-primary school, one must acknowledge the pioneering work of the arts council in promoting the arts-ineducation in ireland today. at a recent plenary meeting of the arts council on the 25th march 2015, the council discussed “the integration of the arts into the education experience of children and young people”. professor john coolahan, who was at this meeting, reported that the high-level implementation group for the arts in education charter was impressed with the range of work relating to the arts-in-education across the republic of ireland. however, alluding to provision for the integration of arts education into the experience of every child, he re-iterated that “greater targeted investment in this area is necessary in order to ensure substantive progress” (cf. arts council website “ireland needs to integrate the arts into the education experience of children and young people”). arts council chair, sheila pratschke, concluded the meeting saying that the arts council is committed to ensuring that opportunities to participate in the arts are central to children’s experience growing up in ireland. given the art council’s new partnership with the department of education and skills, there is hope for the arts in and out of school, and that some day soon, the rhetoric of national and international policy will become a reality in the educational experiences of young people who: have a right to develop their imaginations, to participate in the arts, and to develop their own interests and skills across different art forms …. this should be a central aspect of their experience in and out of school (sheila pratschke, meeting of the arts council 25th march 2015.). international electronic journal of elementary education vol.8, issue2, 267-286, 2015 282 • • • siobhán dowling long is a lecturer in the school of education at university college cork. a contributor to scholarly journals and edited collections, she is the author of the sacrifice of isaac: the reception of a biblical story in music (2013), and with prof. john f.a. sawyer, the bible in music: a dictionary of songs and works and more (2015). references arts council of ireland, the & department of education & skills, the. (2006). artist~schools guidelines: towards best practice in ireland. dublin: the arts council. available at: http://www.artscouncil.ie/uploadedfiles/wwwartscouncilie/content/arts_in_ireland/young_p eople,_children_and_education/artistsschools06_english.pdf arts council of ireland, the. (2007). arts in schools directory. dublin: the arts council. available at: http://www.artscouncil.ie/uploadedfiles/wwwartscouncilie/content/arts_in_ireland/ young_people,_children_and_education/artsineducationdirectory07_eng.pdf arts council of ireland, the. (2008). points of alignment: the report of the special committee on the arts and education. dublin: the arts council. available at: http://www.artscouncil.ie/uploadedfiles/wwwartscouncilie/content/arts_in_ireland/young_p eople,_children_and_education/arts_education_en_08.pdf arts council of ireland, the. (2013). early childhood arts: three perspectives. dublin: arts council. available at: http://www.artscouncil.ie/uploadedfiles/wwwartscouncilie/content /research_and_publications/young_people,_children,_and_education/earlychildhoodarts.pdf bamford, a. (2006). the wow factor: global research compendium on the impact of the arts in education. munster: waxmann. bamford, a. & wimmer.m. (2012). the role of arts education in enhancing school attractiveness: a literature review. eenc (european expert network on culture). available at: http://www.kulturnibazar.si/data/upload/school_attractiveness_paper_final_website.pdf benson, c. & ó tuama, s. (1979). the place of the arts in irish education. dublin: the arts council. bodkin, thomas. (1949). report on the arts in ireland. dublin: stationary office. colgan, anne. young voices: guidelines on how to involve children young people in your work. (2005). dublin: the national children’s office/the children’s rights alliance/the national youth council of ireland. available at: http://www.dcya.gov.ie/documents/publications/31267_young_voices_.pdf coolahan, john. (2015). ‘a portal to the future of the arts in education’ in the irish times 19th may, 2015. available at: http://www.irishtimes.com/news/education/a-portal-to-the-futureof-the-arts-in-education-1.2212746 council of europe & the ericarts institute. compendium of cultural policies and trends in europe (council of europe and the ericarts institute). [online] available at: http://www.culturalpolicies.net/web/ireland.php?aid=33&cid=1240&curl=43 council of national cultural institutions in association with anne gallagher. a fresh view for the 21st century: education, community, outreach policy framework 2014. available at: http://artsineducation.ie/wp-content/uploads/freshviewframeworkdoc2014.pdf dooley, t. et al. (2014). mathematics in early childhood and primary education: teaching and learning (3-8 years). research report no. 18. dublin: ncca. available at: http://ncca.ie/en/publications/reports/ncca_research_report_18.pdf the arts in and out of school / long 283 department of arts, heritage and the gaeltacht, the & department of education and skills, the. (2013). arts in education charter. dublin: government publications. available at: https://www.education.ie/en/publications/policy-reports/arts-in-education-charter.pdf department of children & youth affairs, the. better outcomes brighter futures: the national policy framework for children and young people, 2014-2020. dublin: stationary office. available at: http://dcya.gov.ie/documents/cypp_framework/betteroutcomesbetter futurereport.pdf department of children & youth affairs, the. the national children’s strategy, our children, their lives. (2000). dublin: stationary office. available at: http://www.dcya.gov.ie/documents/aboutus/stratfullenglishversion.pdf department of children & youth affairs, the. national strategy on children and young people’s participation in decision-making, 2015-20. (2015). dublin: stationary office. available at: http://www.dcya.gov.ie/documents/playandrec/20150617natstratparticipationreport.pdf department of education, the. (1995). white paper: charting our education future. dublin: stationery office. available at: https://www.education.ie/en/publications/policyreports/charting-our-education-future-white-paper-on-education-launch-copy-1995-.pdf department of education & skills, the. (2011). literacy and numeracy for learning and life: the national strategy to improve literacy and numeracy among children and young people 2011–– 2020. (dublin: the department of education & skills. https://www.education.ie/en/publications/policy-reports/lit_num_strategy_full.pdf department of education and skills, the. (2012). a framework for junior cycle. dublin: department of education and skills. available at: https://www.education.ie/en/publications/policy-reports/a-framework-for-junior-cyclefull-report.pdf department of education and skills, the. (2011). circular letter (0056/2011). available at: https://www.education.ie/en/circulars-and-forms/active-circulars/cl0056_2011.pdf dice consortium (drama improves lisbon key competencies in education). the dice has been cast: a dice resource research findings and recommendations on educational theatre and drama. available at: http://www.dramanetwork.eu/file/policy%20paper%20long.pdf douglas, i. (2009). staying in school: arts education and new york city high school graduation rates. new york: the centre for arts education (cae). available at: http://www.creativenz.govt.nz/assets/ckeditor/attachments/1029/staying_in_school.pdf?141 0234540 eurydice. (2009). arts and cultural education at school in europe. brussels: eurydice/ education, audiovisual and culture executive agency (eace). available at: http://eacea.ec.europa.eu/education/eurydice/documents/thematic_reports/113en.pdf eurydice. (2013). recommended annual taught time in full-time compulsory education in europe 2012/13. brussels: eurydice/audiovisual and cultural executive agency (eace). available at: http://www.indire.it/lucabas/lkmw_file/eurydice/taught_time_en.pdf finnish national board of education. national core curriculum for upper secondary schools 2003. (helsinki: ministry of education). available at: http://www.oph.fi/download/47678_core_curricula_upper_secondary_education.pdf fiske, e.b (ed.). (1999). champions of change: the impact of the arts on learning (washington d.c., arts education partnership. available at: http://artsedge.ken nedycenter.org/champions/pdfs/champsreport.pdf european commission. (2010). europe 2020: a strategy for smart, sustainable and inclusive growth. communication from the commission. european commission: com document. available at: http://eurlex.europa.eu/lexuriserv/lexuriserv.do?uri=com:2010:2020 :fin:en:pdf international electronic journal of elementary education vol.8, issue2, 267-286, 2015 284 franklin, c. a., & snow-gerono, j. l. (2007). “perceptions of teaching in an environment of standardized testing: voices from the field”. the researcher, 21(1), 2-21 (33). available at: http://nrmera.org/pdf/researcher/researcherv21n1franklin.pdf government of ireland. (1999). primary school curriculum introduction. dublin: stationary office. available at: https://curriculumonline.ie/getmedia/93de2707-f25e-4bee-903592b00613492e/introduction-to-primary-curriculum.pdf government of ireland. education act (1998). available at: http://www.oireachtas.ie/documents/bills28/acts/1998/a5198.pdf hibernian consulting & martin drury. (2007). arts, education and other learning settings: a research digest. dublin: arts council. available at: http://www.artscouncil.ie/uploadedfiles/wwwartscouncilie/content/arts_in_ireland/young_p eople,_children_and_education/research_digest.pdf into (irish national teachers’ organisation). (2009). creativity and the arts in the primary school. discussion document and proceedings of the consultative conference on education. dublin: into. available at: https://www.into.ie/roi/publications/creativityartsintheps.pdf kay, m. (2013). sound before symbol: developing literacy through music. thousand oaks, ca: sage publications. kennedy, e. et al. (2012). literacy in early childhood and primary education (3-8 years). report no. 15. dublin: ncca. available at: http://www.ncca.ie/en/publications/reports/literacy_in_early_childhood_and_primary_educa tion_3-8_years.pdf lmtf (learning metrics task force). 2013. toward universal learning: recommendations from the learning metrics task force. montreal and washington, d. c.: unesco institute for statistics and center for universal education at the brookings institution. available at: http://www.brookings.edu/~/media/research/files/reports/2013/09/learning-metricstask-force-universal-learning/ltmf-recommendationsreportfinalweb.pdf?la=en lmtf (learning metrics task force). 2013a. toward universal learning: what every child should learn. report no. 1 of the learning metrics task force. montreal and washington, d.c.: unesco institute for statistics and center for universal education at the brookings institution. available at: http://www.uis.unesco.org/education/documents/lmtf-rpt1-toward-universal-learningexecsum.pdf lmtf (learning metrics task force). 2013b. toward universal learning: a global framework for measuring learning. report no. 2 of the learning metrics task force. montreal and washington, d.c.: unesco institute for statistics and center for universal education at the brookings institution. available at: http://www.uis.unesco.org/education/documents/towards-universallearning-a-global-framework-for-measuring-learning-metrics-task-force-education-2013-en.pdf lmtf (learning metrics task force). 2014. toward universal learning:implementing assessment to improve learning. report no. 3 of the learning metrics task force. montreal and washington, d. c.: unesco institute for statistics and center for universal education at the brookings institution. available at: http://www.brookings.edu /~/media/research/files/reports/2014/07/implementing-assessment-improvelearning/unescolmtfreport3web3.pdf ministry of education & culture. basic education in finland. available at: http://www.minedu.fi/opm/koulutus/perusopetus/?lang=enhttp://www.minedu.fi/opm/kou lutus/perusopetus/?lang=en national gallery of ireland. (2013). schools access to culture: report of the art, education and schools conference 2013 at the national gallery ireland. dublin: national gallery of ireland. available at: the arts in and out of school / long 285 http://www.nationalgallery.ie/en/learning/schools/~/media/files/education/schoolsreport13. ashx ncca (national council for curriculum and assessment). (2013). artistic performance: engagement with the arts. draft specification for junior cycle short course. (dublin: ncca). available at: http://www.juniorcycle.ie/ncca_juniorcycle/media/ncca/documents/curriculum/short%20 courses/artistic-performance-oct-13-final.pdf ncca (national council for curriculum and assessment). (2014). key skills of junior cycle. dublin: ncca. available at: http://www.curriculumonline.ie/getmedia/def48e3f-68f9-42e4-95def30086321fd0/jsec_key_skills_of-jc_english).pdf ncca (national council for curriculum and assessment). (2011). towards a framework for junior cycle. innovation and identity: schools developing junior cycle. dublin, ncca. available at: http://ncca.ie/framework/doc/ncca-junior-cycle.pdf oecd (organization for economic cooperation and development). pisa 2012 results in focus: what 15 year olds know and what they can do with what they know (2014). available at: http://www.oecd.org/pisa/keyfindings/pisa-2012-results-overview.pdf putz-plecko, barbara. background report on cultural education: the promotion of cultural knowledge, creativity and inter-cultural understanding through education. (2008). parliamentary assembly council of europe: paris. available at: http://www.kulturskoleradet.no/upload/bruker/dokumenter/dokumentarkiv/11_internasjon alt_og_nordisk/2008_12_cultural_education_-_europaradet.pdf richards, j.m. (1976). provision for the arts. dublin: arts council. roe, sandra. the report of a consultation with young people on reform of the junior cycle. (2011). dublin: government publications. available at: http://www.ncca.ie/en/curriculum_and_assessment/postprimary_education/junior_cycle/junior_cycle_developments/dáil_na_nóg_report_.pdf sahlberg, p. “the role of education in promoting creativity: potential barriers and enabling factors”. (337-44). available at: http://0nulled.com/doc/pdf/download/www__greenschool__org--wp-content--uploads--2012-03--pasi-sahlberg.pdf the dana foundation. (2008). learning, arts, and the brain: the dana consortium report on arts and cognition. new york; washington, d.c.: dana press. available at: http://www.dana.org/uploadedfiles/news_and_publications/special_publications/learning,% 20arts%20and%20the%20brain_artsandcognition_compl.pdf un (united nations) general assembly. (1992). the united nations convention on the rights of the child (uncrc). available at: http://www.childrensrights.ie/sites/default/files/submissions_reports/files/uncrcenglish_0.pdf un (united nations) general assembly. (1948). universal declaration of human rights. available at: http://www.un.org/en/documents/udhr/ unesco (united nations educational, scientific and cultural organization). (2006). road map for arts education: building creative capacities for the 21st century. unesco: paris. available at: http://www.unesco.org/new/fileadmin/multimedia/hq/clt/clt/pdf/arts_edu_roadmap_ en.pdf wallace, arminta. ‘the seeds sown are rarely wasted: a charter for change in arts education’ in irish times april 23, 2013. available at: http://www.irishtimes.com/news/education/the-seeds-sown-are-rarely-wasted-a-charter-forchange-in-arts-education-1.1368792 international electronic journal of elementary education vol.8, issue2, 267-286, 2015 286 winner, e., goldstein, t. r. & vincent-lancrin, s. (2013). art for art’s sake? the impact of arts education. educational research and innovation, oecd publishing. websites ark, the. http://ark.ie/ arts council. http://www.artscouncil.ie/about/our-role/ arts council news: http://www.artscouncil.ie/news/ireland-needs-to-integrate-the-arts-into-theeducation-experience-of-children-and-young-people/ arts in education charter website http://artsineducation.ateci.ie arts in education portal. http://artsineducation.ie/en/watch-listen-read/ ateci (association of teachers’/education centres in ireland). http://www.ateci.ie/component/seminar/?task=3&cid=1420 commentary: the arts will help school accountability by mariale hardiman. http://www.dana.org/news/details.aspx?id=42952 creative engagement. http://www.creativeengagement.ie/index.php/about-us/ des, “briefing note” 4th october, 2012. https://www.education.ie/en/schoolscolleges/information/curriculum-and-syllabus/junior-cycle-/a-framework-for-junior-cyclebriefing-note.pdf encountering the arts ireland (etai). http://www.lec.ie/media/docs/ etaipressrelease131113.pdf european commission. “education and training”. http://ec.europa.eu/education/ policy/school/math_en.htm fighting words. http://www.fightingwords.ie/about-us irish architecture foundation (ifa). http://www.architecturefoundation.ie irish primary school mathematics curriculum. http://www.curriculumonline.ie/getmedia/9df5f3c5-257b-471e-8d0ff2cf059af941/psec02_mathematics_curriculum.pdf irish times, the. http://www.irishtimes.com/news/education/a-portal-to-the-future-of-the-artsin-education-1.2212746 music generation. http://www.musicgeneration.ie/about/ state examinations commission "press release 2015 state examinations. https://www.examinations.ie/misc-doc/pr_examstart_15.pdf tedx dublin. you tube. http://www.tedxdublin.com/portfolio-item/fighting-words-sean-love/ writers in schools scheme. http://www.artscouncil.ie/funds/writers-in-schools-scheme/ https://www.examinations.ie/misc-doc/pr_examstart_15.pdf http://www.tedxdublin.com/portfolio-item/fighting-words-sean-love/ microsoft word 6iejee_6_1_reath_warren international electronic journal of elementary education, 2013, 6(1), 95-116. issn:1307-9298 copyright © iejee www.iejee.com mother tongue tuition in sweden curriculum analysis and classroom experience anne reath warren ∗∗∗∗ stockholm university, sweden received: 19 april 2013 / revised: 11 june 2013 / accepted: 21 october 2013 abstract the model of mother tongue tuition (mtt) which has developed in sweden since the 1970’s offers speakers of languages other than swedish the opportunity to request tuition in their mother tongue, from kindergarten through to year 12. it is unique among the major immigrantreceiving countries of the world yet little is known about mtt and its syllabus outside of its nordic context. this article examines the syllabus for mtt from two perspectives; firstly using the framework of constructive alignment, secondly from the perspective of what is hidden. the intended syllabus is revealed as well-aligned, but the hidden curriculum impedes successful enactment in many contexts. examples from case studies in a larger on-going research project offer an alternate approach to syllabus implementation when the negative effects of the hidden curriculum are challenged. while highly context-specific, this model may represent a step in the right direction for implementation of the syllabus. keywords: mother tongue tuition, syllabus, constructive alignment, hidden curriculum. introduction the swedish model of mother tongue tuition, (hereafter mtt) whereby students from kindergarten through to year 12 can request tuition in their mother tongue at school, is unique among the major immigrant-receiving countries, yet little is known about it outside of scandinavia. teachers of mtt are also often involved in study guidance in the mother tongue (sgmt), which is available to students who need help in understanding the school subject matter (which is in swedish) in their mother tongue. while research and evaluations (skolverket, 2008; taguma et al,. 2010, skolinspektionen, 2012) have indicated that changes in the way mtt is organized and implemented in sweden need to occur if the educational needs of multilingual students in sweden are to be better met, the syllabus and content of mtt have not been the subject of much academic research (bunar, 2010, p. 115). this article aims to ∗ anne reath warren, department of language education, stockholm university. se10691, sweden. e-mail: anne.reath.warren@isd.su.se international electronic journal of elementary education vol.6, issue 1, 95-116, 2013 96 contribute to that field through critical examination and evaluation of its syllabus for years 7-9 (hereafter mtt 7-9). this article addresses the questions firstly of how well-designed the syllabus for mtt 7-9 is, secondly how enactment is influenced by factors external to the syllabus itself, and finally how impediments to successful implementation may be challenged. curriculum documents, the literature on and evaluations of the subject and examples from case studies in a larger, on-going, qualitative study on mother tongue tuition comprise the data. after this introduction, a brief overview of the history of the mtt will be given, followed by a discussion of the theoretical and analytical frameworks used in this study. the methods, materials and participants involved are introduced and then the results. finally the results of the evaluation are summarised and implications discussed. the first research question which explores the design of the intended syllabus for mtt will be addressed through examination and evaluation of the written curricula documents. firstly, embedded in its macro-context the curriculum for the swedish compulsory school, pre-school class and leisure time centre (lgr11, 2011), the syllabus for mtt 7-9 will be described and evaluated using the framework of constructive alignment (biggs and tang, 2011). this framework is an approach to curricula development and evaluation in which clear connections, or consistency, between curricula aims, learning activities and assessment criteria is a central feature. as structured learning objectives are central to the syllabus for mtt 7-9, as well as the starting point for curriculum development in the framework of constructive alignment, evaluation of the syllabus in this framework is appropriate. the internal consistency of the syllabus for mtt7-9 will be evaluated by exploring whether its learning aims can be traced through both learning and teaching activities and assessment criteria. the wording of the syllabus aims, learning activities and assessment tasks will also be measured against those principles characteristic of constructive alignment, e.g. the use of appropriate verbs from the solo taxonomy in describing course aims (see section on constructive alingment below for further detail). the second dimension of analysis in this article draws on traditions of critical pedagogy (freire, 1970/2000; apple 1990; mclaren, 2007) and addresses the second research question: how is enactment of the mtt syllabus influenced by factors external to the syllabus itself? it is proposed that a hidden curriculum for mtt impedes successful enactment of the syllabus in many cases. critical theorists regard the hidden curriculum as the “unintended outcomes of the schooling process” which “can often displace the professional educational ideals and goals of the classroom teacher or school” (mclaren, 2007, p. 212). it is that part of the curriculum which is not written in documents, but is known about, learnt, expected and experienced by students and teachers alike. evaluations of the subject of mtt (skolverket, 2008; taguma et al 2010, skolinspektionen, 2010), for example, report that it is usually held after-hours, in borrowed classrooms, when all other students have gone home. this sends signals to students and staff that mtt is something which is not included in the daily work of the school, that the environment in which it takes place is not a priority, and that it takes place when other students or teachers have finished their daily work and have moved on to other activities. other results from these evaluations and responses from semistructured interviews with mtt teachers conducted in a larger on-going research project have been coded, analysed thematically, and categories, including challenges faced by mtt teacher, established. these challenges, it will be argued, form the basis of the hidden curriculum of mtt, which hinders its successful enactment in many environments. mother tongue tuition in sweden / reath warren 97 the final research question, which explores how impediments to successful implementation of the syllabus might be challenged, will be addressed through analysis of examples of the enacted syllabus for mtt 7-9 collected in case studies of a northern kurdish mtt class, part of an on-going ph.d. research project. classroom documents describing procedures and assessment criteria, and classroom activities recorded through photographs, field notes and recordings from lesson observations comprise this data, which will be evaluated in the framework of constructive alignment, in terms of their internal consistency and their alignment with the intended syllabus. while this analysis is highly context specific, it will be suggested that awareness of and directly challenging the conditions which create the hidden curriculum of mtt 7-9, may be a step in the direction of improved implementation, enactment and outcomes. mother tongue tuition in the swedish compulsory school history. while students with finnish as a mother tongue were able to study finnish as an elective subject in grades 7 and 8 at school from 1962, it was not until 1977, as a result of the home language reform that mother tongue tuition in other languages was introduced into the swedish compulsory school (hyltenstam & milani, 2012). this government reform meant that students at swedish compulsory schools who spoke a language other than swedish at home would, under certain conditions, be able to study and develop knowledge of and competency in their mother tongue, in a subject offered through the state education system. the conditions at the time of the reform included that the language should be a “living element” in the home life of the student, that a group of at least four students could be formed and that a suitable teacher could be employed (p. 57). being an elective subject, students and parents were required to request it, rather than it being a part of the standard curriculum. since 1977 there have been many investigations into and evaluations of mother tongue tuition in sweden (e.g. hyltenstam & tuomela, 1996; axelsson, m. et al., 2002; tuomela, 2002, skolverket, 2008; taguma et al, 2010; hyltenstam & milani, 2012). these have led to a range of adaptations of and changes to the conditions under which mtt is provided and the ways that it is implemented and organized. in 1985, the definition of who is eligible for mtt changed from those for whom the language was a “living element” to being only available to those for whom it was “the daily language of communication with at least one guardian”. the minimum number of students to form a class was also increased to five. the official national minority languages in sweden, and those spoken by internationally adopted children were exempt from these regulations (hyltenstam & milani, 2012). in-service courses for pre and primary school teachers, designed to raise awareness of the issues involved with working with multilingual children, including the significance of mother tongue tuition have been introduced in some metropolitan municipalities (axelsson et. al., 2002). the number of hours per week allocated to mtt which was “according to need” in the original home language reform, then fluctuating between less than one and two hours per week (tuomela, 2002, p. 16) is still unpredictable. a two-year teacher education programme for mtt teachers was started in 1977 and then stopped in 1988. approximately 1000 teachers were educated to work with mtt and sgmt in 20 languages during the time this programme was available. after 1988, the education of mtt teachers was supposed to continue as a special strand in the primary education programme, but for various reasons, very few teachers have been educated in mtt since then (hyltenstam & tuomelo, 1996, p. 51). the name of the subject itself has been changed. originally called “home language tuition” it is now called “mother tongue tuition”. international electronic journal of elementary education vol.6, issue 1, 95-116, 2013 98 there is some debate surrounding the name of the subject and the phenomenon itself. garcía draws attention to and problematizes the term “mother tongue” by asking the question “how does one identify an individual’s mother tongue when many are spoken by the mother in the home and acquired simultaneouslyj? and how does one then acknowledge the role of the father and other relativesj?” (garcía, 2009, p. 58). these questions are just as relevant in the swedish multilingual context as anywhere else in the world, but while garcía elects to work with the term “home language practices” in her book, this article will use the term ”mother tongue” in line with the current name of the school subject being analyzed. today, mtt remains an elective subject in the swedish compulsory school, available, providing a suitable teacher can be employed, to groups of five students or more who speak a language other than swedish at home. for sweden’s official minority languages (finnish, all sami languages, torne valley finnish (meänkieli), romani and yiddish), no minimum number of students is required. a new teacher education programme at stockholm university for teachers of finnish as a mother tongue has just been approved by the swedish higher education authority and a total of 150 languages were taught in mtt during the school year 2011/2012 throughout the country of sweden. (personal correspondence; j, spetz: language council of sweden, 13 may 2013). fundamental values, tasks, goals and guidelines – the umbrella curriculum of the swedish compulsory school the syllabus for mtt (and all other subjects taught in the swedish compulsory school) is informed by the fundamental values and tasks of the swedish school system and the overall educational goals and guidelines, as determined in ordinances drawn up by the swedish government. these values, tasks, goals and guidelines are discussed in opening sections of the swedish curriculum and form the basis of the overall or umbrella curriculum for the swedish compulsory school. the syllabus for mtt and all other subjects (consisting of the aims of the course its core content) and the knowledge requirements, are authorized by the swedish national agency for education. the discussion on the values opens with a statement of the democratic principles on which the national school system is fundamentally based. understanding and compassion for others, objectivity and open approaches, equivalent education and rights and obligations are the values on which the education system should rest. the tasks of the school are delineated as promoting pupils’ learning and imparting fundamental values “in order to prepare them to live and work in society” (lgr 11, p. 11). the overall goals and guidelines are intended to “specify the orientation of work in school” (p.14). both the overall goals and the knowledge goals are prefaced by statements which indicate the social efficiency ideology (schiro, 2008) underlying the curriculum, namely, “the school should actively and consciously influence and stimulate pupils in embracing the common values of our society, and their expression in practical daily action”, further, “the school should take responsibility for ensuring that pupils acquire and develop that knowledge that is necessary for each individual and member of society” (lgr 11, p. 14). an understanding of the values expressed in the umbrella curriculum should also inform any evaluation of a subject in the swedish syllabus, mtt representing no exception. this will be discussed in more detail in the results section. mother tongue tuition in sweden / reath warren 99 theoretical perspectives constructive alignment. in curriculum theory, a range of different models of curricula have been described, one of these being the social efficiency model. this model is based on ideas that schools, through their curricula, are responsible for shaping knowledge and behaviour in order to prepare their students for life in the society outside the school gates. curricula developed in this model are often structured through clearly definable learning objectives (schiro, 2008). the curriculum for the swedish compulsory school falls clearly into the social efficiency model as can be seen above through the wording of its overarching goals and values and in the structure of the subject syllabuses (see section on intended curriculum below for a more detailed description of the structure of the mtt 7-9 syllabus). the clearly definable learning objectives of the intended syllabus for mtt make constructive alignment an ideal tool for evaluation. the term “constructive alignment” was first used by john biggs in 1994 to describe an approach to outcomes based teaching and learning (biggs and tang, 2011, p. 51) or as a meta-approach to curriculum design (biggs, 2003, p. 10). it combines theories espousing the constructivist approach to learning (whereby learners actively construct knowledge rather than passively absorbing it) with ideas of alignment between components. in a syllabus, course or unit which is constructively aligned, there is clear thread which runs from the “intended learning outcomes” (ilo), through the “teaching and learning activities” (tla) and on to the assessment tasks. this clear thread allows students to more easily understand and thus, through their learning activities in the classroom and assessment tasks, attain the intended learning outcomes. a syllabus, course or unit which is less well-aligned lacks this clear thread, and the connection between the aims, teaching and learning activities, and assessment may not be obvious at all to learners or teachers. other important principles in constructive alignment include the wording of the ilos, tlas and assessment tasks. a crucial factor to consider when writing course aims in this framework is “to stipulate the kind of knowledge to be learned (declarative or functioning), and to use a verb and a context that indicates clearly the level at which is to be learned and how the performance is to be displayed” (biggs and tang, 2011, p. 64). declarative knowledge can be summarised as being content knowledge, the kind of knowledge that indicates that the relevant facts have been learnt and can be reproduced. functioning knowledge on the other hand, is knowledge which learns and understands the facts, then applies them to new contexts to solve problems, generate new ideas and at best, create “new” knowledge (p. 82). the kind of knowledge and the level of the aim should be appropriate for the level of the class they have been designed for, and can be written based on the levels of understanding described in the solo (structure of the observed learning outcome) taxonomy. this taxonomy “jprovides a systematic way of describing how a learner’s performance grows in complexity when mastering many different academic tasks” (p. 87) and thus is a useful resource for writing course aims. verbs specific to those levels can be used when writing the course aims and the content or topic to be learned should also be clearly identifiable in the aims. the levels prescribed in the solo taxonomy build on each other, and reflect, at the lowest level (prestructural), a lack of basic understanding, and at the highest level (extended abstract) an outcome which indicates not only deep understanding but also application of that understanding to a new context. see table 1 for a brief description international electronic journal of elementary education vol.6, issue 1, 95-116, 2013 100 of each level and some examples of verbs which can be used when writing aims at each level (adapted from biggs and tang, 2011, pp. 88-90, 123). table 1. summary of levels of understanding and verbs from solo taxonomy. solo taxonomy level description of learning outcome some examples of verbs to use when describing this level prestructural misses the point entirely or shows little evidence of relevant learning. unistructural addresses the task in part but misses important details indicating a basic understanding of terminology and a start to the task at hand, but little more. memorize, identify, name, tell, write, label, find, count multistructural presents facts but does not connect or see beyond them. can be compared with seeing the trees but not understanding the wood. classify, describe, list, report, discuss illustrate, select, outline, narrate relational ties facts and details together and integrates into a whole indicating deeper sense of understanding. apply, integrate, analyse, explain, review, argue, compare, contrast, examine, predict extended abstract addresses, links and integrates facts into deeper understanding and then conceptualises all in a new context theorise, hypothesize, generalise, reflect, create, invent, compose critical pedagogy and the hidden curriculum the hidden curriculum has been defined by critical theorists as “the unintended outcomes of the schooling process”, including “jthe messages that get transmitted to the student by the total physical and instructional environment” (mc laren, 2007, p.12), “jthe one [curriculum] that no teacher explicitly teaches but that all students learnjthat powerful part of the school culture that communicates to students the school’s attitudes towards a range of issues and problemsj.” (banks, 2001, p. 23). it includes the ways that students get shaped by teaching and learning activities, rules of conduct, the way classrooms are organised and messages transmitted by the physical and instructional environment (mclaren, 2007). classic studies on, for example, gender roles in classrooms (e.g. sadkev and sadkev, 1985), show the way that teacher attitudes to boys and girls in the classroom (giving more academic attention to boys than to girls; showing boys how to complete tasks, but completing tasks for girls) teach boys independence and girls dependence. that kind of attitude and those kinds of instructions are certainly not included in the intended curriculum of any school or subject, but nonetheless impact significantly on schooling outcomes. in her critical analysis of the australian curriculum, ditchburn (2012) contrasts the “two competing narratives” (p. 347) of the document, the overt narrative which promotes a “world class curriculum”, with an alternative, hidden narrative, which indicates that the curriculum rests on conservative and narrow ideologies. this hidden narrative is revealed by extracting statements from the intended australian curriculum, then thematically grouping and analyzing them. the statements (from the intended curriculum) are found to reveal layers of meaning, contradictions and assumptions which, without this analysis, would have gone unnoticed and uncommented but remained influential on teaching and learning procedures and outcomes in australia. ditchburn argues that revealing the hidden narrative can lead to valuable healthy mother tongue tuition in sweden / reath warren 101 debate about the shape of the curriculum itself, and in the best case scenario, influence the way the curriculum is implemented in classrooms in australia. a critical approach to curriculum analysis has a responsibility then, to examine not only the intended curriculum, but also the attitudes, organisational and implementational aspects of curriculum. these aspects are not explicitly stated in the intended curriculum but shape student outcomes and attitudes and arguably influence wider socially held perceptions about schooling and subject value. this examination and awareness can be used to influence implementation of curricula and improve outcomes. methods and materials analysis of the intended curriculum to evaluate the intended syllabus for mtt 7-9 using the framework of constructive alignment, the aims of the syllabus (that which biggs calls intended learning or ilos), followed by the core content (which form the basis for the teaching and learning activities or tlas) and the knowledge requirements (assessment criteria, rather than tasks) as set out in curriculum documents, will first be analysed for internal consistency. internal consistency implies that the aims will be reflected in the core content and knowledge requirements, meaning that students, as well as teachers know what has to be achieved, how they will go about achieving it and then feel that the ways in which they are assessed enable them to demonstrate their knowledge. if the aims can be traced through both content and knowledge requirements, the syllabus is internally consistent and well aligned, the second method for evaluation concerns alignment with principles of course design specified by biggs and tang (2011), the syllabus will be closely examined to see if the words used to describe the aims resemble those recommended in the solo taxonomy, and if a context for the aim is given. uncovering the hidden curriculum for mtt in order to conduct a critical analysis and evaluation of the mtt 7-9 curriculum, awareness of the very particular context of this subject, which is realized in 150 different languages throughout sweden and implemented both administratively and pedagogically under greatly varying conditions, is essential. it is true that in any given school subject, teachers have their own individual way of implementing any given course or teaching any given unit. however, the breadth and variety of the languages taught in mtt as well as the huge variations in its organization, implementation and resources afforded to its teachers (skolverket, 2008) makes the enactment of this curriculum a particularly complex and multifaceted phenomenon to both organize and evaluate. as discussed in the section on critical pedagogy and hidden curriculum, the hidden curriculum is not written, it emerges rather from the range of circumstances by which the subject is surrounded. to uncover the hidden curriculum of mtt, investigations into the subject (skolverket, 2008; taguma et al., 2010; bunar, 2010; hyltenstam and milani, 2012, skolinspectionen, 2010), responses of four mtt teachers in semistructured interviews and observations made during on-going case studies in mtt classrooms are drawn upon. these collective data are analysed inductively, coded and grouped thematically. a category of “challenges” emerges, and these are further categorized into subcategories of challenges (implementational, attitudinal and classroom-based) and then international electronic journal of elementary education vol.6, issue 1, 95-116, 2013 102 further condensed into statements about the subject which have their basis not in curriculum documents but in evaluative literature and experience, documented through interviews and reports on the subject. it is proposed that this series of statements represent the hidden curriculum of mtt. the statements are compared to the underlying tasks, goals and values of the swedish compulsory school expressed in the umbrella curriculum, to see in what respects the hidden curriculum differs from the intended curriculum. in line with critical pedagogical theories, it is suggested that awareness of the hidden curriculum can lead to reflection and discussion on implementation, which may hinder the negative effects of an unexamined hidden curriculum. research and evaluations of mtt research findings and official evaluations of mtt in sweden have revealed long lists of challenges; problems with implementation and organisation, problems with teachers who traipse around between schools to only to meet small groups of students for an hour or less in rooms which they cannot assume will always be available (skolverket, 2008, p. 17). the subject has not been successfully integrated into daily school life and parents are not informed or involved which has had led to the marginalisation and low status of the subject (municio, 1987, in hyltenstam and milani, 2012). a long list of problems associated with the subject was pointed out as early the 1980’s, these including “lack of clear understandings of different teaching models, limited time allocation, inappropriate timetabling, inappropriate textbooks/teaching resources, lack of collaboration with other staff members, inappropriate roomsj..teacher competence, teacher education, teachers’ working conditionsjnegative attitudes to mtt in society in general and in individual schools and negative reports on the subject in the media” (hyltenstam and milani, 2012, p. 65) [author’s translation]. many of these problems remain the same almost 30 years later, as can be read in a report from the swedish schools inspectorate where it is reported that mtt “lives its own life, without connection to or collaboration with all other teaching” (skolinspectionen, 2010, p. 7). in the oecd report on swedish migrant education it is pointed out that while mtt is available to multilingual students, it is not widely taken advantage of or known about. this, in combination with a decentralized educational system sets up a challenge to the implementation of the policies supporting mtt (taguma et al, 2010). case study datasemi-structured interviews. the interview data was collected from four mtt teachers of languages prominent in the migrant community in urban sweden, who were selected to be interviewed through snowball sampling. three of the four teachers interviewed have the majority of their working hours at the same lower secondary school (given the pseudonym of star school). the majority of the fourth teacher’s working hours are at another lower secondary school in the same municipality as the first (given the pseudonym of first school, the same school at which the lesson observations described in the section on analysis of the enacted curriculum were conducted). all four of the teachers work in more than one school, which is a typical situation for mother tongue teachers in sweden (skolverket, 2008). the interview participants, two males and two females, are teachers of northern kurdish, turkish, arabic and spanish respectively. as well as being language teachers, they work with study guidance in the mother tongue for students who have arrived recently in sweden. through a process of inductive analysis, interview responses were coded and themes and categories were identified. these themes and categories were compared, contrasted and cross-checked with the results of investigations into the subject (skolverket, 2008; taguma et al., 2010; bunar, 2010), policy and curricula documents (lgr 11), field notes taken during visits to schools, and photographs of student works mother tongue tuition in sweden / reath warren 103 and classroom environments to strengthen validity (burns, 2010, p. 96). the category in focus in this article is that of “challenges”. analysis of the enacted curriculum to bring an applied perspective to this evaluation, examples from the enacted syllabus for mtt 7-9 will be examined for alignment with the intended syllabus. the data is taken from non-participant observations made in a northern kurdish mtt class for gr 9 students during on-going fieldwork. the classroom is in a school in suburban sweden (given the name of first school to preserve anonymity). 100% of the students attending first school speak a language other than swedish at home and 80% of the students participate in mtt. at first school, concrete measures have been taken by school leadership to influence and change the hidden curriculum and give the intended curriculum for mtt a chance to work. data collected at first school provides many examples of what can be achieved in the subject of mtt when awareness of the hidden curriculum is followed up by direct action to improve the conditions under which the subject operates. at first school, the subject of mtt is written into the schedule of every student in the school who takes the subject, and 80% of the students entitled to mtt do so. mtt lessons take place throughout the school day, not after school is finished, in designated classrooms, at least two of which are specifically resourced for two different language groups. five mother tongue teachers are employed by the school, and they form part of a teaching team called “bridges”, which also includes modern language teachers. these mtt teachers are involved in the daily life of the school to an extent which is unusual for the vast majority of mtt teachers. the teacher of the lesson in focus in this analysis (given the name of cegar to preserve anonymity) was born and educated as a primary school teacher in turkey. he worked as a primary school teacher for five years before moving to sweden. after 11 years of doing other jobs in sweden, cegar saw an advertisement for mtt teachers in northern kurdish, he applied, got the job and has taught mtt in that municipality ever since. during the 13 years he has worked as a kurdish mtt teacher, he has also taken university courses in pedagogy and classroom leadership. cegar is employed at first school and has the majority of his students there although, in common with all mtt teachers, he does work at other schools as well. he is also the leader of a teaching team at first school, which includes french, spanish and other mtt teachers. the 11 students in the classes observed are in year 9, which means they are between15 and 16 years old and all are girls. they were all born in sweden to parents who were born in regions of turkey where northern kurdish is spoken. the unit, “alla mina kurdiska ord”, (all my kurdish words) was introduced to these students in the autumn term of 2012. the description of the unit was printed on a paper hand-out, which was distributed to students in class, and was also accessible to students, teachers and parents online. the first observed lesson involved cegar and the students reading through the document, describing the up and coming unit of work in both swedish (the unit description was written in swedish) and in northern kurdish. in the second observed lesson, students were working with the results of interviews they had conducted with (multilingual) teachers in their school in the course of that unit. working in pairs, students asked these pre-selected (by cegar) teachers questions about their languages, sense of cultural and personal identity and thoughts on being multilingual. three of the students were writing up the responses teachers gave to the international electronic journal of elementary education vol.6, issue 1, 95-116, 2013 104 interview questions on the whiteboard. those sitting down were dictating the responses from their transcriptions. while the dictation of these responses and the writing on the whiteboard was in swedish, discussion between the students and their teacher was mostly in northern kurdish, with swedish being used intermittently. cegar asks the students which teacher’s response corresponds most closely with their own feelings about their own languages and sense of personal and cultural identity, in other words, which teacher students could identify with most closely in those respects. the lessons were audio-recorded, described in detailed field notes, photographs of student work and the classroom environment were taken, and all the swedish used during the lesson was transcribed and translated to english. the teacher also spoke about the lesson in swedish after the students had left, and this was transcribed and translated as well. to evaluate the enacted curriculum of mtt 7-9 in the framework of constructive alignment, classroom activities recorded through photographs, field notes and recordings from lesson observations will be drawn on. the activities undertaken will be evaluated in terms of their alignment with the syllabus for mtt 7-9. reflecting on transcription and translation the lessons observed were in northern kurdish and swedish. the swedish used in the lessons was transcribed, then translated to english. as analysis is on-going, none of the kurdish has been transcribed or translated yet. the interviews were conducted in swedish, transcribed in swedish, then translated to english. this process was quite complex in that it involved interviewing teachers in their second (or third) language and the researcher’s second language then translating it all to english. the production of a transmuted text, as translated transcriptions have been called (halai, 2007, p. 347) involved making decisions about the most appropriate way of representing ideas presented in one cultural context so that it makes sense to readers in completely different geographical, cultural and organizational locations. halai summarises the optimal outcome as “j to translate the interview text in such a way that the basic requirements of (a) making sense, (b) conveying the spirit and manner of the original, and (c) have a natural and easy form of expression were (sic) all met adequately” (p. 351). those reflections underlie the transcription process used in this study. results of analysis of intended and enacted curriculum in the framework of constructive analysis intended curriculum internal consistency and alignment with design principles. this section will first trace the words used in the “aims” of the syllabus for mtt 7-9, to see if they are present in the core content and the knowledge requirements, in other words, to see if those different elements of the syllabus are constructively aligned. secondly, it will examine the wording of the course aims and knowledge requirements to see if they follow the design principles advocated in constructive alignment. the specific aims of mtt state that “teaching in the mother tongue should essentially give pupils the opportunities to develop their ability to: � express themselves and communicate in speech and writing � use their mother tongue as an instrument for their language development and learning. � adapt language to different purposes, recipients and contexts, � identify language structures and follow language norms, mother tongue tuition in sweden / reath warren 105 � read and analyse literature and other texts or different purposes, and � reflect over traditions, cultural phenomena and social questions in areas where the mother tongue is spoken based on comparisons with swedish conditions” (lgr 11, p. 83). analysis reveals that all six aims of mtt 7-9 can be clearly traced through core content and knowledge requirements. the way this alignment is created is similar for all the aims. a detailed description of how this is done with one of the aims is given to exemplify this process. the third aim of mtt 7-9 is that pupils should develop their ability to “adapt language to different purposes, recipients and contexts” (p. 86). this aim is picked up in the core content in the reading and writing, speaking, listening and talking and use of language sub-headings, where the following content is suggested: “strategies for writing different types of texts adapted to their typical structures and language featuresj.adaptation of language, content and structure to purpose and recipientjdifferences in the use of language depending on the context, the person and the purpose of communication” (p. 85). these strategies and adaptations are, in turn, addressed in the knowledge requirements at three different levels. to achieve the highest grade a, “pupils adapt in a well developed [sic] way the contents of their texts and language so that they function well in the situations for which they are intended” (p. 93, my italics). for grade c, the wording is identical, but instead of pupils adapting their language in a “well developed way”, they do it in a “developed way” and instead of the language “functioning well”, it “functions relatively well”. grade e, the passing grade is achieved if students adapt their language in a “simple way” and their language “basically functions [sic]” in the situation. every one of the aims of the mtt syllabus can be clearly traced in the same way as described above, through the core content and knowledge aims, in other words, the intended syllabus is internally consistent. a closer analysis of the aims of mtt 7-9, in terms of the kind of knowledge expected and at what level (as explored in the section on constructive anlingment above) reveals that knowledge at almost every level of the solo taxonomy is expected and described using verbs from the solo taxonomy (or verbs that mean the same). the only knowledge at the unistructural (basic) level refers to identification of language structures and following of language norms. from this level, the course aims move through the relational and into the extended abstract range. the topic or content to be covered is also indicated in the aims and functioning knowledge (higher level) is expected in all but two of the aims. all aims are within the reach of a year 9 student, as will be illustrated by examples in the results section on enacted curriculum. a summary of the aims of the syllabus for mtt 7-9, in relation to the kind of knowledge they presume and at what level of the solo taxonomy they aim, is given in table 2. the solo taxonomy verbs are in bold and the topic or content is italicized. international electronic journal of elementary education vol.6, issue 1, 95-116, 2013 106 table 2. analysis of syllabus aims – mother tongue tuition years 7-9. aim (ilo) declarative or functioning solo taxonomy level express themselves and communicate in speech and writing functioning extended abstract –as to “express oneself” is indicative of creation or generation. use their mother tongue as an instrument for their language development and learning functioning relational as using language as an instrument could be considered as “applying” . adapt language to different purposes, recipients and contexts functioning relational – as “adapting” implies “differentiating” “integrating” “contrasting” and “applying”, verbs all found at the relational level . identify language structures and follow language norms declarative unistructural read and analyse literature and other texts for different purposes declarative relational reflect over traditions, cultural phenomena and social questions in areas where the mother tongue is spoken based on comparisons with swedish conditions. functioning extended abstract the verbs recommended in the solo taxonomy are used in the aims, and describe different levels of knowledge in different context, indicating alignment with the principles of constructively aligned curricula development. this wording used in the knowledge requirement also resembles examples from biggs and tang (2011) of assessment criteria (knowledge requirements) where the ilos are assessed in terms of how well they have been achieved (see example above). it could be argued that there is a need for discussion and development of the wording of these criteria. it is easy to imagine considerable debate arising between teachers who have different perceptions of how “well” “developed” has to be before it is considered “well-developed” and, despite following the principles of constructive alignment, this is an aspect of the mtt 7-9 which could be perceived as less than transparent. results uncovering the hidden curriculum the literature on and evaluations of mtt based on larger scale studies and interviews with four mother tongue teachers working in the same area of suburban sweden, were compared and a number of similarities were found. these similarities were inductively analysed and categorised into themes. one of the themes was that of the challenges mother tongue tuition in sweden / reath warren 107 faced by mtt teachers. these challenges were further categorised into structural, attitudinal and classroom-based, and are summarised in table 3 (below). table 3. summary of challenges faced by mtt teachers structural attitudinal classroom issues too little time for the subject (max. 1 hour per week). mtt is not compulsory. allocation with resources (budget). implementation at municipal and school level. difficult to organize mtt with smaller language groups. having to take children out of other lessons for mtt. negative attitude (of school employees and parents). lack of knowledge about mtt. status of mtt. helping students develop a multicultural identity they are happy with. lack of communication. at the perifery having to chase other teachers to initiate project work. keeping students interested. teaching materials – quality and supply. different levels of ability in the same class. writing texts on the computers which have swedish keyboards in languages which use another script. tired, unmotivated students by examining these challenges and comparing them with the values, tasks, goals and guidelines of the intended umbrella curriculum, a number of inconsistencies arise. while the intended curriculum stresses that “that teaching should promote students’ learning and acquisition of knowledge based on their background, language and knowledge (lgr 11, p.11) empirical data tells us that students who speak a language other than swedish at home, approximately 20 % of students in swedish schools today, have only approximately one hour a week to learn about and in their mother tongue. newly arrived students receiving study guidance in their mother tongue (sgmt) may be allocated more hours initially but these hours are reduced as the student gains better competence in swedish, and finally, taken away altogether unless specifically recommended. swedish research indicates that many schools have little knowledge of what sgmt is or even know that it is available (skolinspektionen, 2010, p. 22). it is difficult to provide students with “a wealth of opportunities for enhancing confidence in their language abilities” (lg11, p. 11) when the only subject taught in their mother tongue is allocated just one hour per week, and where mother tongue teachers come after hours or not at all, and are not part of the pedagogical life of the school to any extent. the lack of resources compounds this problem, as does the low status of the subject, the lack of communication between mtt and other teachers and the periferal positioning of the subject. international electronic journal of elementary education vol.6, issue 1, 95-116, 2013 108 curriculum documents for the swedish compulsory school state clearly that democratic working forms include “jbeing able to choose courses, subjects, themes and activitiesj” , and should be applied in practice in schools. they also state that while education should be “equivalent”, that does not mean “the same”, indicating that each school must take into account their specific context and student body (pp. 10, 11) when implementing and organising curricula. while theoretically this sounds democratic and reasonable, background knowledge of the issues surrounding mtt bring another perspective. in order to be able to choose a course, students and parents must have knowledge about the benefits and knowledge offered in it. when mtt has low status and has been the subject of negative reporting in the media ((hyltenstam and milani, 2012, p. 65), how can students make an objective and positive choice? and how can multilingual students living in both rural and urban contexts in sweden be guaranteed “equivalent” (if not the same) education in mtt? their contexts can be vastly different – they may be one of only five students speaking turkish in a country town, or enrolled in a school where the majority of students speak languages other than swedish, including their own. the concluding comments of the introduction of the curriculum for the swedish compulsory school point out clearly where the responsibility for spreading information about mtt and ensuring equivalent education lies , “the principal organiser has a clear responsibility for ensuring that this [school activities matching up to national goals] takes place” (lgr 11, p. 13). this puts successful organisation and implementation of the subject and syllabus of mtt into the hands of each school leadership group. it follows that a school leadership with knowledge about language development and the importance of the mother tongue may implement the subject in a different way than a school leadership without this knowledge. empirical data supports this claim of irregular implementation of the subject (skolverket, 2008). by viewing the contradictory data presented above through a critical lens, a number of general statements about the syllabus for mtt can be generated: � mtt is non-essential (elective), � mtt is less important than other languages (1 hour per week as opposed to 3 or 4 hours per week in english, swedish and modern languages) � mtt is inconvenient (after-hours, sometimes held in another school, always in another classroom), � mtt is low status (short on resources, books and qualified teachers) and � mtt is disconnected from every other aspect of school life. i suggest that these statements represent the hidden curriculum of mtt, that which is not in alignment with all the fundamental values tasks, goals and guidelines of the swedish compulsory school and which impedes successful enactment of the intended curriculum in many contexts. enacted curriculum – “all my kurdish words” and mtt 7-9 the conclusions reached by the swedish national agency for education’s investigation into the learning conditions for students with a mother tongue other than swedish (skolverket, 2008), have been used to illustrate the implementational and organisational difficulties associated with the subject. however, the results of the report were not all negative. in fact an interesting contradiction arose in terms of the role of the subject of mtt. on the one hand, mtt is described in the report (p. 17) as a highly marginalised elective subject with the majority of teachers working under conditions which can only be described as unsatisfactory. on the other hand, statistical analysis of mother tongue tuition in sweden / reath warren 109 the average merit ratings of student results, reveal that not only do those multilingual students who have participated in mtt have a higher average merit rating than those who have not participated, but they also have a higher average merit rating than students with a swedish background. as the report puts it, it seems almost “remarkable” (p.18) that one hour’s tuition in a marginalised elective subject, which is taken by only half of those entitled to it, should have such a noticeable effect on students’ academic performances. complicating factors (e.g. number of years of tuition, motivation) are also discussed but the strength of the finding stands in stark contrast to other, more negative results from evaluations and investigations of the subject. first school the year following the national agency for education’s 2008 investigation into mtt, a reform known as the “mother tongue tuition reform” was undertaken in the municipality in which first school lies. this reform had many implications, including that a percentage of the mother tongue teachers previously employed centrally by the municipality were moved over to the schools where they had the majority of their students. with years of experience of negative attitudes towards mtt and themselves as teachers of it, the teachers who were moved over to first school were initially concerned that these negative attitudes would become stronger, given that, in accordance to the guiding principles of the swedish curriculum, the principal of the school would now be their employer, and responsible for how their subject would be administered. however, it turned out that a different attitude existed at first school. example 1 den negativa attityden när jag ser (3.0) de skolor som vi har flyttats over till, den fortsätter i (2.0) i i i några skolor. men här i förstaskolan så hade vi, , ja, säga, kanske tur, eller vi hade en skolledning som hade en annan åsikt. that negative attitude when i see (3.0) the schools we were moved over to, that continues in (2.0) in in in some of the schools. but here in first school we were, well, yes, maybe lucky, our school leadership had another attitude. (interview with cegar, northern kurdish mtt teacher. first school). the classroom the following data were collected in 2012 in case studies which are part of a larger-on-going ph.d. project. observations and recordings from the classroom in combination with field notes and photographs of student work reveal how this written document was put into action in the classroom during two lessons. the first stage of the task was the actual presentation of the project. this was done by the teacher bringing up the unit plan, written in swedish, on a projector so the whole class could see it, while also being supplied with their own paper copies. the teacher asked different students to take it in turns to read from the swedish document. after each section was read aloud in swedish, it was discussed in north kurdish. the hand-out describing this unit starts with a series of questions: example 2 vad betyder egentligen kurdiskan för mig? varför studerar jag kurdiska? har jag någon nytta av det? är ett språk bara ett språk, eller påverkar språket min identitet? (vem är jag), min personlighet (hur är jag, vad är jag). är både svenskan och kurdiskan en del av min kulturella identitet? what does kurdish actually mean to me? why am i studying kurdish? do i have any use for it? is a language just a language, or does it influence my identity (who i am), my personality? international electronic journal of elementary education vol.6, issue 1, 95-116, 2013 110 (what am i like? what am i?). are both swedish and kurdish a part of my cultural identity?” (extract from written unit plan– first school) the teacher describes the aim of the unit first in personal terms: example 3 mitt syfte med undervisningen: jag vill att du genom samtal och diskussioner ska tänka/grubbla (tänka djupt) over frågorna ovan my aim with this unit: i want you to think/ponder (think deeply) about the above questions through conversations and discussions. (extract from written unit plan– first school) in response to a question (in kurdish) from a student, cegar expands on this aim in swedish: example 4 man tanker djupt ja. har ni sett kanske mamma eller pappa eller ni själva ibland går och sen pratar och tänker för sig själv va? det betyder grubbla. vi ska tänka högt. vi ska tänka djupt. one thinks deeply yes, have you maybe seen mum or dad or yourself sometimes, walking, talking to themselves right? that is to ponder. we are going to think aloud. we are going to think deeply. (classroom audio recording – first school) tasks that students will undertake during the unit are described (here with english translation only): example 5 � we are going to read 6 short texts in kurdish about how kurdish youth (girls and boys) think about what kurdish means to them. (the texts are from the book “dergûşa nasnameyê”/identity’s cradle). we will also read 2 short texts in swedish. (book “two languages or more”). in the texts i have highlighted several words: language, multilingual, culture, identity, cultural identity. we will look for different meanings for these words and discuss their meaning in large groups. � we will look on the internet for job advertisements in which different companies are looking for multilingual staff. � you and a friend will interview a teacher/member of staff at first school and ask three questions: how many languages do you know? what does swedish/your mother tongue mean to you? is it possible to feel like a kurd/turk/arab/syrian/ and a swede at the same time? or can one feel like none of them? write the answers or record the interview with your phone. then write the answers on your computer. � during the last two lessons you will write a short text on what kurdish means to you and what use you have for it now and in the future. and how you yourself feel: if you feel that you have parts of both swedish and kurdish culture. use the words and concepts you have learnt during this unit. the text will be written on the computer and put in the “kurdish” folder, � we will play the game “hangman”. :)) (extract from written unit plan– first school) cegar showed the class the books that the texts they will be reading come from, and pointed to where the dictionaries were located, so students would know where to look for help. the specific wording of the assessment criteria (table 4), and how they would be used in this unit, was also covered. for example, cegar explained that by achieving the first intermediate aim, students will get the passing grade (e). fulfilment of the fifth and final intermediate aim will give students the highest grade (a). mother tongue tuition in sweden / reath warren 111 table 4. assessment criteria from unit “all my kurdish words” intermediate aim (1) you recognise the concepts “language, multilingual, culture, identity” (you have heard or read the concepts at school, on tv, in the newspaper etc. you can read some kurdish letters with support from the teacher: c-ç, e-ê, i-î, j-y. s-ş, u-û. retell/ summarize med support from the teacher or friends. you partake in conversations. intermediate aim (2) you recognise (you have heard in school, on the tv, in newspapers etc.) the concepts “language, multilingual, culture, identity” and try to use them when you speak. you use the phrases: i think thatj, i feel thatj. becausej. you read aloud with support from the teacher and re-tell/summarize the texts. you partake in discussions. intermediate aim (3) you know what the concepts “language, multilingual, culture, identity” mean and you use them when you speak and write. you use the phrases: i think thatj, i feel thatj. becausej. you read fluently. re-tell in your own words. use lexin and the internet to find out the meanings of words and spelling. partake in discussions with interest. start using the words the words you have learnt. you reflect on other’s thoughts about the subject. intermediate aim (4) you use the concepts “language, multilingual, culture, identity” in speech and writing. you start to use concepts such as cultural identity, personality, multicultural. you use the phrases: i think thatj, i feel thatj. becausej. you read very fluently. use lexin and the internet. you can even “read” between the lines. partake in discussions with interest. start to use the words you have learnt in the correct context. you reflect on (think about what others have said) and compare other’s thoughts with your own. intermediate aim (5) you use the concepts “language, multilingual, culture, identity” in speech and writing. you start to use concepts such as cultural identity, personality, multicultural, diversity. you use the phrases: i think thatj, i feel thatj. becausej. you read very fluently. use lexin and the internet. you can even “read” between the lines. you can argue for what you think using your own experiences (speak about your arguments and say why your arguments are strong, other arguments are weak), reflect on and compare other’s thoughts with your own. the discussion of these aims is intense and in kurdish and swedish. cegar explains both what specific words in the description of the aims mean and how students are to reach those aims. for example, in this discussion (in swedish) about the word “begrepp” (concept) which is used repeatedly: example 6 s1: ja så grunden {well, the basics} t: begrepp {concept} s2:det är som innehåll, det som innehåll i texten {it’s the content, the contents in the text} international electronic journal of elementary education vol.6, issue 1, 95-116, 2013 112 t: mmmm {mmmm} [rising intonation] s3:det viktigaste {the most important} s2: eller det som ingår {or what’s included} t: begrepp är ett ord (1.5) ett ord som till exempel man säger ett betydelsebärande ord i en text alltså en ett viktigt ord i en text som man måste kunna {concept is a word (1.5) a word that for example you can say carries the central meaning in a text that is an important word in a text that one should know} s1: mmmm {mmmmmm} (classroom audio recording – first school) when reading through the intermediate aims, pointed questions and referencing between the different aims helps students understand the difference between them, and the resulting grade that reaching these aims will give them. in example 7, cegar refers to the key concepts to be learnt in the unit, distinguishing between aim 2, which is reached if the concepts are recognised and an attempt is made at using them, and aim 3, where the concepts are used in reading and writing: example 7 t: [indicating and quoting from intermediate aim 2]..försöker anvander när du talar, men härp {ptry to use them when you speak, but herep}[indicating intermediate aim 3] s: [in chorus, quoting intermediate aim 3] man använder { one uses} t: du använder de när du talar och skriver {you use them when you speak and write} over the seven weeks that followed, students worked on the tasks described in the unit both in and outside of class, in pairs, groups and independently. the next group of examples come from the lesson in which students discussed the results of their interviews with multilingual teachers at first school. students were given specific questions to ask the teachers, concerning the languages they spoke, the way they felt about their mother tongue, multilingualism and their sense of personal, social and cultural identity (the key concepts which are in focus throughout the unit, written on the whiteboard and regularly referred to both in swedish and in kurdish)(see figure 1). students were asked to compare their own feelings about their mother tongue and sense of identity with their teachers’ thoughts. some students used kurdish to express these thoughts, others used both swedish and kurdish. while considering the responses of one of the teachers who, although he feels swedish in many respects, cannot discount the influence that food and friends connected to his culture have, one student reflects: example 8 s:jag håller med [namn på lärare]. jag förstod hans svar, jag känner mig svensk på många (2.0) {i agree with [name of teacher] i understand his answer.i feel swedish in many} (2.0) t: [hjälper] avseende [helping] {respects} s: ja på olika sätt, men vissa saker så kan man inte ta bort eftersom.det kan beror på matvanor umgänge./.../. på det sättet känner han sig svensk. {yes, in different ways, but certain things you can’t take away since.it could be food or your circle of friends./.../.in that way, he feels swedish} (kurdish mtt lesson observation, first school) mother tongue tuition in sweden / reath warren 113 figure 1. key words from the unit figure 2. students texts in kurdish (translation of figure 1) multilingual, multicultural, identity, ethnic identity, cultural identity, diversity, personality. or as another students puts it, quoting first the words of a teacher with a serbian background in the interview, then switching to describe her own feelings: example 9. [quoting the serbian teacher’s response to interview questions] ...men det är viktigt att känna sina rötter och kunna förstår vad man är. för att passa in samhälle måste man anpassa sig till den svenska kulturen eftersom vi bor i sverige så känner jag att en del utav mig är svensk och lever i svensksamhället trots att jag är serb men jag tänker trots arr jag är kurd { ...but it’s important to know your roots and be able to understand what you are. to fit into society, you have to adapt to the swedish culture, since we live in sweden i feel as though a part of me is swedish, living in swedish society although i am a serb [changing the narrative so it is about herself in the last phrase] but i think although i am a kurd. (kurdish mtt lesson observation, first school) the final assessment piece (fig. 2) was the writing task, in which all the reading, speaking, and writing students did throughout the unit, came together in a piece of reflective writing done in class. in the essay, in which students write about themselves, their languages and their identity, one student chooses to write in both kurdish and swedish. in the swedish essay, she writes: example 10 jag har växt upp med två kulturer. för att mina föräldrar har en kultur som kommer från deras hemland och där jag är född ifrån har man ett annat kultur. på så sätt kan jag känna mig (pir kulturî). ibland känner jag mig både svenska och kurd. för att jag är född i sverige och mina föräldrar är födda i turkiet. jag går i en skola i sverige, där man pratar svenska och har deras regler. men när jag åker till mitt hemland då känner jag mig mer kurd. i have grown up with two cultures. because my parents have a culture which comes from their land and where i was born from [sic] there is another culture. in that way i feel international electronic journal of elementary education vol.6, issue 1, 95-116, 2013 114 (multicultural) [author’s note, this student used the kurdish word for “multicultural” and bracketed it]. sometimes i feel both swedish and kurdish. because i was born in sweden and my parents were born in turkey. i go to a school in sweden where swedish is spoken and have swedish rules. but when i go to my homeland i feel more kurdish. (student essay, first school) the aim of the unit, that these 11 year 9 students should “think deeply” about questions of language and identity, is directly connected to aims from the mtt 7-9 syllabus which states that students should ” reflect over traditions, cultural phenomenajin areas where the mother tongue is spoken, based on comparisons with swedish conditions” (lgr 11, p.83). the work undertaken in and out of class (see examples in figures 1 and 2 and examples 2-10 above) in the forms of learning new words, reading texts, conducting interviews and then writing and discussing the interview data and finally, writing essays also fulfils at least three other mtt 7-9 aims (see table 2). students “expressed themselves and communicated in speech and in writing” (see fig. 2), they “used their mother tongue as an instrument for language development and learning” and “read and analysed texts” when they read texts in northern kurdish and learnt new words and concepts in that language from the texts (see table 3). in summary, empirical data suggests strongly that students understood the aims, completed the teaching and learning activities and fulfilled at least some of the intermediate aims (knowledge requirements) of the unit “all my kurdish words” and the syllabus for mtt 7-9. the enacted syllabus of mtt 7-9, in this classroom at fırst school thus aligns with the syllabus for mtt 7-9 and is internally consistent. discussions and conclusion the critical analysis and evaluation of a curriculum which can be enacted in at least 150 different languages is not a straightforward task. in order to gain a deeper perspective, the intended syllabus for mtt 7-9 was first examined in terms of its internal alignment, in relation to the macro-context (the curriculum for the swedish school. then, using an approach from the tradition of critical pedagogy, results of evaluations and examples from an on-going research project were examined to locate and evaluate the hidden curriculum. finally, examples of classroom implementation at one school in suburban sweden were evaluated in the framework of constructive alignment. although the intended curriculum for mtt 7-9 has been evaluated as being internally consistent and constructively aligned with the swedish compulsory school’s overall values, tasks and goals, there is a hidden curriculum which in some respects does not align at all. in other words, there is a significant gap between what the subject of mother tongue tuition aspires to do and what it is able to achieve, which is dependent on the specific context in which the subject takes place and limited by the negative associations of the hidden curriculum. in one context, where mtt is implemented effectively in at least one classroom, and the hidden curriculum thus changed, there is improved alignment between the values, task and goals of the schools and all aspects of the subject, despite the limited hours the subject is afforded and its elective nature. it is difficult however to argue that that same constructive alignment exists in settings where the hidden curriculum remains unchallenged and the subject thus not implemented so effectively. this article, using data from a kurdish mtt 7-9 classroom in suburban sweden to complement curricula documents and evaluations, brings the possibility of a new narrative, and an alternative approach to enactment of the syllabus for mtt. in the school as at which these lessons took place, the hidden curriculum of mtt has been mother tongue tuition in sweden / reath warren 115 actively challenged by school leaders, and this has changed the conditions under which mtt teachers enact their curriculum. changed conditions surrounding the implementation and organization of the subject have changed the hidden curriculum at that school, for at first school: � mtt is an elective subject which the majority of students take. � mtt is important and has its own slot in the timetable, although it is still only allocated 1 hour per week as opposed to 3 or 4 hours per week in english, swedish and modern languages. � mtt is convenient (held during the school day in a designated classroom in the “mother tongue corridor”), � mtt has the same status as other subjects (it has a budget for resources and five of the teachers are employed under the same conditions as any other subject teacher at he school) � mtt is an integrated, “normal” part of school life. while this school lies in its own specific sociocultural environment and the results are not generalizable to every other context, the significance of challenging the hidden curriculum is valid for every context. it is proposed that an awareness of the hidden curriculum and implementation of concrete changes to challenge its negative effects, may, in different settings where mtt takes place, be a step in the right direction for future enactment of the mtt syllabus in sweden. • • • anne reath warren is currently enrolled in the ph.d. programme in language education at stockholm university. her research interests include mother tongue tuition, multilingualism, language education and language planning. references apple, m.j. (1990). ideology and curriculum (2nd ed.) new york, ny: routledge. axelsson, m., lennartson-hokkanen, i. & sellgren, m. (2002). den röda tråden. utvärdering an stockholm stads storstadssatsning – målområde språkutveckling och skolresultat. stockholm: språkforskningsinstitutet i rinkeby och stockholms stad. banks, j.a. (2001). multicultural education: characteristics and goals. in j.a. banks & c. mcgee (eds.), multicultural education: issues and perspectives (4th ed., pp.3-30). new york: john wiley and sons. biggs, j. (2003). aligning teaching and assessment to curriculum objectives. imaginative curriculum project. ltsn generic centre. biggs, j. b., & tang, c. (2011). teaching for quality learning at university : what the student does. maidenhead: mcgraw-hill/society for research into higher education/open university press. bunar, n. (2010). nyanlända och lärande. en forskningsöversikt om nyanlända elever i den svenska skolan. stockholm: vetenskapsrådets rapport serie. 6: 2010. retrieved 20 aug international electronic journal of elementary education vol.6, issue 1, 95-116, 2013 116 2013, from http://www.vr.se/download/18.5adac70 4126af4b4be2800017840/rapport+6. 2010.pdf burns, a. (2010). doing action research in english language teaching. ny: routledge. ditchburn, g. (2012). the australian curriculum: finding the hidden narrative? critical studies in education, 53(3), 347-360. freire, p. (1970/2000). pedagogy of the oppressed. new york, ny: continuum. garcía, o. (2009). bilingual education in the 21st century. a global perspective. oxford. wileyblackwell. halai, n. (2007). making use of bilingual interview data: some experiences from the field. the qualitative report, 12(3), 344–355. retrieved 11 september 2013, from http://www.nova.edu/ssss/qr/qr12-3/halai.pdf hyltenstam, k. & t. milani. (2012). modersmålsundervisning och tvåspråkig undervisning. in: hyltenstam, k., m. axelsson & i. lindberg (eds). flerspråkighet – en forskningsöversikt. (vetenskapsrådets rapportserie 5:2012.) stockholm: vetenskapsrådet, p. 55-76. hyltenstam, k. and tuomela, v. (1996). hemspråksundervisning. i k. hyltenstam (ed.), tvåspråkighet med förhinder? invandraroch minoritetsundervisning i sverige. lund: studentlitteratur. lee, jin souk, oxelson, e. (2006) “it’s not my job”: k-12 teacher attitudes toward heritage language development. bilingual research journal, 30(2), 453-477. lgr11. (2011). curriculum for the compulsory school, preschool class and the leisure-time 2011. stockholm: skolverket. mc laren, p. (2007). life in schools. boston, usa: pearson education. sadkev, m. & sadkev, d. (1985). "sexism in the the schoolroom of the 80's". psychology today, march, 55-57. schiro, m.s. (2008). curriculum theory: conflicting visions and enduring concerns. thousand oaks, ca: sage publications. skolinspectionen (2010). språk och kunskapsutveckling for barn och elever med annat modersmål än svenska. kvalitetsgranskning. rapport 2010:16. skolverket. (2008). with another mother tongue students in compulsory school and the organization of teaching and learning. report 321. retrieved 21 oct 2011 from http://modersmal.skolverket.se/engelska/images/stories/engelska/other_mother_tongue_08. pdf taguma, m., kim, m., brink, s. & telteman, j. (2010) oecd reviews of migrant education. sweden. paris: oecd. international electronic journal of elementary education, 2015, 8(2), 223-242. 223 student voice: an emerging discourse in irish education policy domnall fleming  the weir centre, ireland abstract in positioning student voice within the irish education policy discourse it is imperative that this emergent and complex concept is explored and theorized in the context of its definition and motivation. student voice can then be positioned and critiqued as it emerged within irish education policy primarily following ireland’s ratification of the united nations charter on the rights of the child (uncrc) in 1992. initially emerging in policy from a rights-based and democratic citizenship perspective, the student council became the principal construct for student voice in irish postprimary schools. while central to the policy discourse, the student council construct has become tokenistic and redundant in practice. school evaluation policy, both external and internal, became a further catalyst for student voice in ireland. both processes further challenge and contest the motivation for student voice and point to the concept as an instrument for school improvement and performativity that lacks any centrality for a person-centered, rights-based, dialogic and consultative student voice within an inclusive classroom and school culture. keywords: student voice, student council, evaluation, performativity, citizenship. student voice: definition, theorised, motivation, contested definition student voice as an emergent and complex concept refers to students in dialogue, discussion and consultation on issues that concern them in relation to their education, but in particular, in relation to pedagogy and their experiences of schooling whether as a student cohort, individual class groups or within a forum construct like a student council (fleming, 2013). thus, the concept is both defined and described by a wide range of terms and activities that centre on the repositioning of students to facilitate their engagement with their teachers and schools. across a range of research, instructional literature and policy documents on student voice, the language and terminology relating to the concept includes variously: ‘participation of students’, ‘involvement of students’, ‘listening to students’, ‘consulting with students’, ‘dialogue with students’, ‘researching with students’, ‘students’ perceptions’, ‘students’ perspectives’, ‘evaluation by students’ and ‘empowering of students’. these terms are used, often interchangeably, in research and in descriptions of activities that reference the concept of student voice as students being engaged in  domnall fleming, department of education and skills, the weir centre, bandon, co cork, ireland. email: domnall_fleming@education.gov.ie international electronic journal of elementary education vol.8, issue2, 223-242, 2015 224 interaction with peers, teachers and school authorities on matters and issues that affect them in their school experiences (ibid). rudduck (2005) positions the student as the object in a process of conversations directed by teachers seeking advice and inviting opinion and perspective from students, and seeking to re-engage the dis-engaged through student voice. this is a theme that emerges throughout student voice literature (arnot, mcintyre, pedder and reay, 2004; mitra, 2001, 2004, 2007; nieto, 1994; rudduck, 2007; rudduck and flutter, 2004; soohoo, 1993). fielding and mcgregor (2005) emphasize student voice as reflection, dialogue and action combined with discussion as ‘student voice covers a range of activities that encourage reflection, discussion, dialogue and action on matters that primarily concern students’ (ibid, p.2). thomson (2011) focuses on facilitating the child or young person to be agentive in the context of their education. this points to a rights-based definition as ‘student voice refers to the process through which children and young people, individually and collectively are able to speak up about their education (ibid, p.24). being ‘able’ indicates facilitation towards agency and suggests the right of students to have an individual or collective voice, which has volume in pursuit of action. cook-sather (2006) further references students’ rights and introduces ‘power’ within the school hierarchical structure in a definition that seeks ‘meaningful acknowledged presence’ for students implying a change from a position of silence to active engaged participant. for cook-sather, with this acknowledgement of position comes ‘the power to influence analyses of, decisions about, and practices in schools’ (ibid., p.363). a further development of the concept envisions students not only as having a voice, an involvement, and a consultative role in schools, but also acting as participants in critical analysis and research directed at school reform (thiessen, 2007). thiessen positions student voice as co-construction of the school experience, as students become co-participants and researchers within analysis and reform. these ‘initiatives’, it is argued, represent a deep and agentive student voice pointing to a rights-based, emancipatory and democratic orientation for the concept. theorized student voice is theorized within three frames; the voice of the student in the classroom within a socio-cultural theoretical frame that views learning as a social interaction and pedagogy as social constructivism; within a social constructionist theoretical frame that views student voice as dialogue, communication and consultation in classrooms and schools, and through a poststructural theoretical frame that challenges the concept in its assumption that a universal, individual or authentic student voice exists (fleming, 2013). social constructivism positions student voice as the engaged and agentive voices of students in school settings. this is central to participation in the co-construction of knowledge in the classroom. a social constructionist framing reflects a student voice that questions and challenges discourse and practice framed within democracy and active citizenship. this is a student voice of critical pedagogy, emancipation and transformation whether in the classroom or at whole-school level. a post-structural framing positions student voice as complex, contradictory and challenging, and situated within a discourse of power and inequity in their schools and classrooms. this theoretical framing contests both the social constructivist framing of student voice as agentive and interactive in a social context of learning and a constructionist framing of voice as dialogic, consultative and emancipatory, as both are student voice: an emerging discourse in irish education policy / fleming 225 bounded by a power discourse that controls and limits these voices through meanings, constructs and assigned roles and positions that are established and reinforced by practice, authority and imposed policy in schools and classrooms. (ibid) motivation autonomy for children in decision-making relating to life choices was a central argument within a rights-based framing of student voice (dworkin, 1977). student voice can be placed within this self-determining, rights-based agenda as recognition of ‘the moral integrity of children, entitled to equal concern and respect, and entitled to have their autonomy and self-determination recognised’ (freeman, 1987, p. 309). linked to these arguments is the repositioning of children as developing citizens within a democracy with an entitlement and a right to a voice (ibid). the un convention on the rights of the child (uncrc) in 1989, followed by the ratification of the un charter on the rights of the child (uncrc) in 1992, by all countries with the exception of the usa and somalia, were the significant milestones in advancing student voice in the context children’s rights. article 12 of the charter required that: 1) states parties shall assure to the child who is capable of forming his or her own views the right to express those views freely in all matters affecting the child, the views of the child being given due weight in accordance with the age and maturity of the child. 2) for this purpose, the child shall in particular be provided the opportunity to be heard in any judicial and administrative proceedings affecting the child, the views of the child being given due weight in accordance with the age and maturity of the child. (uncrc article 12:1,2, 1992) ratification of the charter was seen as significant in positioning the child as a full human being with the ability to participate freely in society (freeman, 1996). it principally focused the argument for the right of the child to be consulted in…’all matters affecting the child’ (uncrc, 1992). article 12 combines the aforementioned needs of the child in terms of provision and protection with their right to participate in decision-making. thus ‘it brings together the familiar view of children as in need of protection and provision…with a different view, of children as individuals in their own right, as ‘social actors’ who can form and express opinions, participate in decision-making processes and influence solutions’ (bragg, 2007, p. 11). the obligation to incorporate the charter into the legal framework of a country is widely viewed as the catalyst for the development of what became termed as ‘student voice’ in many ratifying countries (rudduck and flutter, 2000). it is noteworthy however that the united states government did not ratify the convention due to concerns about the perceived erosion of the authority of adults (kilbourne, 1998). while contested, the widespread and varied translation of the obligations of article 12 advanced student voice actions and initiatives, and became the framework for policy development and strategies in many jurisdictions (noyes, 2005). however, lundy (2007) argued that the obligations to transpose article 12 into policy and legislation required two elements to be provided to children: the right to express a view; and the right to have the view given due weight. four conditions or structures were identified to fully realize the potential of article 12 as the foundation for deep and meaningful student voice: space within which children can express a view; voice to allow them to express their views; an audience that will listen; and that their expressed views will stimulate a response and action (ibid.). lundy strongly questions the motivation, beyond the rhetoric, of policy international electronic journal of elementary education vol.8, issue2, 223-242, 2015 226 makers and schools to provide the four identified conditions for the development of a meaningful student voice within this rights-based framework. the promotion of personalized learning within education policy in england can also be viewed as a further motivation of student voice from a rights-based perspective. personalization of learning, as an education policy initiative, focused on ensuring that the learning needs of individual students were addressed in schools and classrooms (hargreaves, 2004; ruddock, 2006). using the voices of students was viewed as a key element of personalized learning. personalization required schools to priorities engagement with learning, personal responsibility for learning, independent learning and the development of confidence and maturity in students (dfes, 2004), to be achieved through teachers and students working together to improve learning (hargreaves, 2004). student voice was identified as one of the nine gateways to personalized learning that facilitated students ‘to play a more active role in their education and schooling as a direct result of teachers becoming more attentive, in sustained or routine ways, to what students say about their experience of learning and of school life’ (ibid., p. 7). however, fielding and others challenged student voice in this context, as a neo-liberal administrative strategy aimed at school improvement and performativity rather than at the person-centered learning needs of the individual student (fielding, 2007). engagement in decision-making from a democratic active-citizenship perspective is highlighted throughout student voice research as a further motivation. education for democracy and active citizenship, has emphasized the reinforcing of human-rights based values, the empowerment of stake holders and the involvement of staff, students and parents in all important school decisions (dürr, 2004). durr however argues that while active citizenship requires involvement, debate and participation by students and the school recognized as ‘the preparatory system for citizenship’ (ibid., p. 12), the actual involvement of students in co-responsibility and decision-making has been very limited. in many countries, the need for citizenship education became a concern and the creation of school (student) councils and the provision of taught citizenship programmes was the policy response. the crick report (1998) recommended the introduction of a programme for citizenship into the national curriculum in the uk. the introduction of civic, social and political education (cspe) into the irish junior cycle curriculum in 1995 (des, 1995), and the planned implementation of ‘politics and society’ for senior cycle (ncca, 2009), marked a similar curricular response in ireland. however, it is contended that teaching and learning about democracy will fail unless it takes place within a democratic educational framework and environment (dürr, 2004). it is equally argued that education for democratic citizenship cannot be taught but must be experienced by students (huddleston, 2007; kelly, 1995; mccowen, 2011). the student or school council therefore became the principal policy-driven participatory democratic structure in irish and uk schools with the dual role of providing an opportunity for lived representative prefigurative democracy and a construct to articulate student voice in schools. contested social constructionist, postmodernist and poststructuralist theorists argue against the notion of a universal truth (or a universal voice) within the web of social, emotional, linguistic, political and communicative experiences and contexts (macnaughton, 2003). within a postmodern and poststructural perspective, knowledge is complex, contradictory and subject to change and challenge (dahlberg, moss and pence, 1999). they argue that the concept of student voice should therefore be bounded in both the context and the culture of specific settings and is complex, challenging and contradictory. there is a student voice: an emerging discourse in irish education policy / fleming 227 complex and diverse range of voices many that can make difficult listening (bragg, 2001; fielding, 2004). these voices are contextualized and constructed by power relationships and authority in particular contexts and are circumscribed by issues including age, race, gender and class. engagement of student voice practice therefore has the potential to expose actions and meanings in a classroom by highlighting inequity and issues of exclusion. it can equally free those who are confined or silenced by that context (taylor and robinson, 2009). student voice should therefore be situated in relation to power discourses, gender, class and race that operate in schools and classrooms and the range of other agendas whether rights-based, democratic, or consumerist at the wider policy level (bragg, 2007). deconstructing the concept even from its broadest definition points to a ‘cacophony of competing voices’ (reay, 2006, p. 179) that can be listened to and heard in schools. a deeper analysis of the concept, therefore points to the power positioning of voice in a compulsory school system where the volume of student voice, the extent to which voices are heard, and the awareness of those that are silent or silenced are concerns (arnot, mcintyre, pedder and reay, 2004; fielding, 2001). who is facilitated to speak, who is listening and the question of provision of pedagogical and physical spaces in schools for dialogue (fielding, 2001, lundy, 2007) are further challenges to any view of student voice. reference to an authentic student voice is also challenged as being dependent on the issues raised by those voices that may be privileged to speak, the questions asked of them, and the values and assumptions of those who are asking and of those who are listening (chadderton, 2011; connolly, 1997). students may also express views and opinions that teachers and schools find challenging and unacceptable (mitra, 2003). teachers’ use of adult language and accepted school cultural norms to interpret student-teacher dialogue can also question the validity and integrity of that dialogue (bragg, 2001). the challenge for student voice in practice is, at one level, the difficulty of facilitating empowerment of those whose ‘voices have been silenced or distorted by oppressive cultural and educational formations’ (ellsworth, 1989, p. 309) without directing students or imposing another discourse reflecting the values of the teacher (ibid.). at a deeper level the challenge for the teacher in the classroom is to develop strategies to allow the diversity of students’ voices to be heard through dialogue so as to expose how their experiences have differed based on their perceived social position relative to others (ibid.). student voice, while motivated by right, prefigurative democracy and active citizenship, must thus be viewed through the lens of power. power is expressed in and is exercised through discourse (foucault, 1979). foucault saw social institutions sustaining themselves through a discourse of truths that defined actions (gore, 1993). these truths identified and sustained what was valued and privileged. awareness of distortion, privilege and silence allows us to understand how meaning and discourse are constructed (foucault, 1979). discourse viewed as what is said, who speaks and with what authority, is central to foucault’s analysis of the complexity of power and its possession in an organization. discourses and meanings however, arise from practices and not from written policy or the spoken word (ball, 1990). those with power control discourse through dividing practices, which limit the freedom and identity of members of the organization (marshall, 1990). foucault’s principal of discontinuity further points to discourses and practices that can and do change their meaning as they are deployed at different levels of an organization and are received by different members. it is argued that the range of interpretations, affordances and limitations to student voice must also be viewed through the lenses of dividing practices and the discontinuity of discourse. international electronic journal of elementary education vol.8, issue2, 223-242, 2015 228 foucault’s envisioning of power and how it permeates the school presents a significant critical challenge to student voice as an emancipating, inclusive and equalizing concept (fleming, 2013). institutionalized practices translated into the irish post-primary school context are grounded in a discourse that focuses students’ learning on a curriculum and examination that reflects the needs of the economy and the adult world and that does not permit consultation with student on decisions in schools or classrooms (ibid.). in this context, students are classified by age and labelled and numbered by group, ability level and programme. the daily experience of students in irish schools suggests that the voices of school management and teachers are privileged above those of students and parents (devine, 2001 2002, 2003, 2009; lynch and lodge, 2002). using the lens of foucault, it is argued that student voice as consultation, dialogue, partnership and participation in a school structure will be challenged and limited by the dominant power / knowledge discourse, by the policy discourse of schools and education authorities, and by discontinuity in the way discourses are internalized and normalized within school culture. in the context of ireland and england, examination and surveillance in the context of the centrality of curriculum delivery and internal or externally based assessments of students, or overall school performance, have created a performanceorientated and outcomes-driven script for teachers reflective of a neo-liberal and consumerist agenda (arnot and reay, 2007; fielding, 2007, 2011; lodge, 2005, 2008). student voice initiatives risk tokenism within the constraints of these discordant discourses and the power hierarchy of student, teacher, principal and government, which seeks to preserve, rather than challenge or transform (taylor and robinson, 2009). student voice is therefore visualized as an on-going process that is contextualized, situated and negotiated in a contested space (taylor and robinson, 2009). a poststructural and postmodern theoretical articulation of student voice points to an appreciation of the complexity and challenge of the interaction of power with any authentic and emerging student voice in schools. the utopian notion of one unifying authentic student voice, reflecting equality, justice and democracy (ellsworth, 1998), is dismissed, and new meanings, different interpretations, and a variety of discourses are visualised (taylor and robinson, 2009). the student voice policy discourse in ireland as in other jurisdictions, the concept of student voice emerged in the irish educational policy discourse following the ratification of the un charter on the rights of the child (uncrc) in 1992. article 12 forms the basis for policy development relating to participation and consultation with children in relation to matters that might affect them. in educational policy in ireland, the ‘actioning’ of the requirements of the uncrc article 12 has resulted primarily and almost exclusively in the formation of student councils in postprimary schools. a period of partnership in policy development that emerged in irish education in the latter years of the twentieth century (devine, 2004; granville, 2004; trant and ó donnabháin, 1998), and reached an important focus in the national education convention in 1994. the convention, which involved all education stakeholders, with the exception of student representatives, resulted in the drafting of the white paper on education (1995) that informed the education act (1998). the report of the convention made very limited reference to any role for students in decision-making or school governance. it did however make reference to the desirability of ‘a shared dialogue on the core values of the school, embracing the patron, trustees, board, principal, staff, parents and students’ (national education convention secretariat, 1994, p. 28). student voice: an emerging discourse in irish education policy / fleming 229 the white paper on education in 1995 was first to mention a student council in policy as a mechanism for student participation in schools: likewise, school policies should be developed in close consultation with parents, and with students where appropriate. in order to facilitate this consultation, the board of management of each second-level school will be encouraged to promote the formation of a students' council, which will work in collaboration with the staff and the parents' association (government of ireland, 1995, p.181). student voice, envisioned as a consultative role, was positioned in the policy discourse for the first time by this single reference to a student council. the role was envisioned as one of consultation on school planning within a structure that was promoted by the board of management. while the conditionality of ‘where appropriate’ was used, and both control and power were vested with the board of management, this represented an aspiration for meaningful involvement of students (and parents) in a significant aspect of the work of the school. these involvements were structured solely within the establishment of student councils in schools. the education act (1998) made first mention of a role for students in its outline of the functions of the school principal. this role was envisaged as consultation in relation to school objectives as ‘under the direction of the board and, in consultation with the teachers, the parents and, to the extent appropriate to their age and experience, the students, set objectives for the school and monitor the achievement of those objectives’ (education act, 1998, 23:2). while students were mentioned in the context of consultation appropriate to their age and experience, the imbalance in favour of the roles for parents, teachers and staff in following subsections was obvious, as the principal ‘shall encourage the involvement of parents of students in the school in the education of those students and in the achievement of the objectives of the school’ (ibid., 23:2), and ‘wherever practicable, the principal shall, in exercising his or her functions under this section, consult with teachers and other staff of the school’ (ibid., 23:6). mention of students was excluded from this consultation process. a role for students in the school received further attention however, in reference to communication with students, their involvement in the school, and through the establishment of a student council: a board shall establish and maintain procedures for the purposes of informing students in a school of the activities of the school (ibid., 27:1) and the board: shall facilitate the involvement of the students in the operation of the school, having regard to the age and experience of the students, in association with their parents and teachers (ibid., 27:1) the establishment and maintenance of these procedures was not outlined or developed in the act although the establishment of a student council was given some more attention: students of a post-primary school may establish a student council (ibid., 27:3). nevertheless, though the act provided for the establishment of a council, it was made clear that such a council was not obligatory, and therefore it ‘may’ rather that ‘should’ or ‘shall’ be established. the function of the council was outlined as: ‘a student council shall promote the interests of the school and the involvement of students in the affairs of the school’ (ibid., 27:3). international electronic journal of elementary education vol.8, issue2, 223-242, 2015 230 section 27 of the act represents the key defining reference to student voice in the policy discourse in ireland at this time and has defined developments, specifically in relation to the role of the council in the school, to date. it placed promotion of the interests of the school as a primary function of the council, followed by the secondary role of ‘involvement of students in the affairs of the school’. this represented a significant reduction in the role of the council from that of consultation on policy development as envisaged in the white paper (1995) and reflected a very limited role in decision-making. the act was not specific on how involvement in the affairs of the schools might be structured or operationalised. clearly, power and control of the council was vested in the board of management as: ‘the rules for the establishment of a student council shall be drawn up by the board’ (ibid., 27:4). the national children’s strategy (2000) was the key national strategic document that was developed in response to the requirements of ratification of the uncrc in 1992. it represented ‘a major initiative to progress the implementation of the convention’ (national children’s strategy, 2000, p. 6) and set out a vision based on democratic citizenship and participation for ‘an ireland where children are respected as young citizens with a valued contribution to make and a voice of their own’ (ibid., p. 4). in the context of education, the strategy reflected the education act (1998) by focusing on the establishment of ‘school’ (student) councils and by channelling student voice through this construct with an emphasis on the development of democratic citizenship: the education system has a special role in developing children’s sense of civic responsibility. school councils are being established to give children at post-primary level a direct involvement in the running of their schools (ibid., p. 31). it is noteworthy that this strategy document envisaged the most significant role for students and student councils in the context of democracy and citizenship. it pointed to students having…’a direct involvement in the running of their schools’, in contrast to…‘the involvement of students in the affairs of the school’ (education act, 1998, 27:4). in a further response to the uncrc (1992), a national children’s office within the office of the minister for children and youth affairs (omcya) was established in 2001, as was the office of the ombudsman for children, following the enactment of the ombudsman for children act in 2002. both developments highlight a changing position for the children’s rights discourse in irish society. ten years later, in 2012, the thirty-first amendment to the irish constitution (1937) replaced article 42 with a more developed and elaborated text primarily focusing of the rights of children in judicial proceedings and in the care of the state. the insertion of section 4.2 into the amended article 42a (referendum commission, 2012) directly reflected the wording and the intention of the uncrc (1992) particularly in the provision, by law, for the need to engage with the views of the child. while the amendment clearly establishes the rights of the child under the constitution, its provision for the voice of the child to be heard concerns children in care, adoption, guardianship, custody and access issues following judicial proceedings. the discussion and subsequent passing into law of the amendment however heightened public awareness of consultation with children in wider public discourse and particularly in the interfaces between the child and state services, including schools. the national policy framework for children and young people 2014 – 2020 (2014) has become the most recent element of the national policy discourse that has informed the emergence of student voice. the strategy identifies ‘listening to and involving children and young people’ as one of six transformational goals. commitments arising from this goal student voice: an emerging discourse in irish education policy / fleming 231 include ‘consultation with children and young people on policies and issues that affect their lives’ and the creation of ‘mechanisms to provide children and young people with the opportunity to be heard in primary and post-primary schools through student councils or other age-appropriate mechanisms.’ (ibid.,p 32) student voice as positioned in irish post-primary schools the exclusion of children from policy and decision-making at school level has been recognised throughout the period since the national education convention (1994). the absence of children or students from this early partnership process has been noted, as was the absence of any overarching policy for the inclusion or participation of children in decision-making. devine, (2004) outlined the position of student voice in the context of the power and authority of schools highlighting its ‘adult-centred terms, with children confined to independent initiatives in schools rather than through any prescribed obligation to include their voice on policy decisions made (ibid., p. 115). however, devine also notes that though students were largely excluded, the increased involvement of parents could only be viewed as a positive development pointing towards a gradual shift in power positioning within education and schools to a situation where ‘children, as is increasingly the case with parents, are perceived along with teachers, to be partners in education with a voice to be heard and expressed’ (ibid., p. 124). despite this, a more pessimistic and exclusionary view has also been articulated in terms of equality and the inclusion of the voices of the marginalized, including students from minority groups and those with special educational needs, as lodge, devine and deegan (2004) recognized that ‘some voices came to be excluded and marginalized, while others continue to be prioritized’ (ibid., p. 3); a finding that was both recognized by shevlin and rose (2003) and challenged through advocacy for a personalized and situated student voice (rose and shevlin, 2010). arising from the education act (1998), the student council became the central and only construct that could provide a platform for students’ inclusion and participation in decision-making in schools and equally provide an experience of prefigurative democracy and active citizenship. it was the publication of the guidelines document for schools: ‘student councils: a voice for students’, by the department of education and science (des) in 2002 that provided some guidance on how the council should be established and developed. the title of the document, the only such guidelines to be published, directly linked the student council to student voice, and placed the concept of voice as central to the student council. the document used the terms ‘involvement’ and ‘partnership’ and made reference to ‘the affairs of the school’ and ‘for the benefit of the school’, but, reflecting the text of the education act (1998), at no point referred to consultation and dialogue, or any central role for students in decision-making. nevertheless, the guidelines did extend and deepen the policy discourse on the role of the student council, perceiving it as representing the views of the student body to school management, promoting good communication, and supporting educational development and students’ contribution to policy development (des, 2002). an additional action, that of ‘assisting’, a term that is not used in earlier documents or in the education act, was also introduced in the guidelines document. this additional role of the council was envisaged as ‘assisting with induction of new students, assisting with sporting and cultural activities, and assisting with fundraising events for charity’ (ibid., p. 11). the verbs used throughout this document to describe the work of the council include: representing; promoting; supporting; contributing, and assisting. these terms outline the limitations placed on the role of a student council as envisaged by the department of international electronic journal of elementary education vol.8, issue2, 223-242, 2015 232 education and skills, in 2002, a role that can be viewed as supportive and advisory but not as consultative or dialogic. these guidelines also reflect a reductionist view of the role and potential of the council as an instrument for student voice in a post-primary school setting. the student council as a construct, and the language used to describe its role, limited any sense of empowerment as envisaged by any general interpretation of the uncrc that referred to ‘the right to express those views freely in all matters affecting the child’ (uncrc, article 12:1, 1992). any potential for empowerment and transformation was merely envisioned as an ‘involvement in the affairs of the school’ (education act, 1998, 27:3). the potential for a transformative voice or role for the students had been significantly diminished by the text of these guidelines. when interrogated, the roles suggest very limited and vague involvements for students in consultation, dialogue and participation in the running of schools and in school decision-making. it is equally questionable whether the student council construct and role, as outlined in the guidelines document, is provided with space, voice, audience and a facility for response in schools (lundy, 2007) to allow for the development of a deep and meaningful voice for students. no further written policy direction was provided to post-primary schools relating to the student council although a support service was established which provided training and support literature for students, schools and student council liaison teachers between 2002 and 2011. the student council: definition and role the aforementioned department of education and science guidelines document ‘student councils: a voice for students’ (2002) is the only policy document to provide a definition of the structure and role of a council in an irish post-primary school setting. a student council was set out as ‘a representative structure through which students in a postprimary school can become involved in the affairs of the school, working in partnership with the school management and staff and parents for the benefit of the school and its students’ (des, 2002, p. 8). this definition utilized the wording of the education act (1998), focusing on ‘involvement in the affairs of the school’, but introduced the idea of a partnership and the representative nature of the council. the inclusion of these terms pointed towards an agenda of participative democracy and active citizenship ‘for the benefit‘ of all stakeholders. this envisioning of the role of the council can be viewed in particular contrast to that of the education act, uk (2002) that outlined the requirement for english schools to consult students about decisions that affect them. the education and skills act, uk, (2008) further extended student voice, as the governing body of a school was required to appropriately consider the views of students when making decisions. an early description of a student council identified ‘a body, in part nominated or elected by pupils which meets from time to time from weekly to annually and whose chief function is to advise the school authorities or to take decisions which they may or may not implement’ (chapman, 1970a, p. 268). a later description of student councils by the department of children, schools and families (dcsf, 2008) reflected a greater emphasis on representative democracy, but with a particular reference to partnership with students in their education. student councils in this definition were ‘democratically elected groups of students who represent their peers and enable students to become partners in their own education, making a positive contribution to the school environment and ethos’ (school councils uk, cited by whitty and wisby, 2007, p. 30). a more simplified description of the construct focused on the representative nature of a council as a means of presenting students’ views was outlined in research carried out in student voice: an emerging discourse in irish education policy / fleming 233 advance of promoting the establishment of student councils in northern ireland viewing a school council as ‘a group of pupils within a school, elected by their peers to represent them and their views’ (northern ireland assembly, 2011, p. 4). based on the experience of the growth and development of student councils in scottish schools over time ‘the dominant model of a pupil council in scotland today is one in which a reasonably representative group of students are elected as pupil councilors and perform a consultative and collaborative influencing role within their schools, particularly around school life issues of direct and immediate importance to students themselves’ (children in scotland, 2010, p. 8). scottish councils emphasize a deeper role of influence for the student council through reference to consultation and collaboration with a focus on issues that impact directly on students. wales is the only jurisdiction in the uk that requires schools to establish a student council (welsh assembly government, 2005). the welsh assembly government provides a detailed outline of a council that includes direction on roles and involvements in planning, governance and recruitment of staff. in the welsh context a school council is seen as: a group of pupils elected by their fellow pupils to represent their opinions and raise issues with the headteacher and governors in the school. the school council can also take forward projects on behalf of the pupils, and be involved in planning and things like the school development plan, governing body meetings and interviewing staff (ibid.). these definitions from policy makers in five jurisdictions on the islands of ireland and britain emphasize a representative democratic and participative role for student councils. they focus particularly on the elected, representative nature of the council that has a varying and sometimes unspecified role in school activities and decision-making. the range of emphases however, extends from students’ involvement and partnership, to advice, consultation and collaboration, and to deeper involvements in school decisionmaking. the extent to which student councils had been established in schools equally varied across the jurisdictions cited. a total of 68% of irish post-primary schools were found to have a functioning student council in operation in a survey completed by the democracy commission (2005). in scotland, 90% of schools had ‘whole school’ pupil councils (children in scotland, 2010), while 95% of schools in england were found to have functioning school councils (whitty and wisby, 2007). since 2005, all publicly funded schools in wales were required to have a student council (welsh assembly government, 2005, 2009) while the department of education in northern ireland was actively supporting the establishment of student councils in 2011 (northern ireland assembly, 2011). a critique of the student council construct in irish education the growth in student councils in irish post-primary schools was initially viewed in the context of enabling participation, responsibility and accountability on the part of students and thereby providing an important exercise in democracy (o’gorman, 1998). the limited but more recent research on student councils in ireland however reveals a different perspective. tokenism and limited democratic engagement in decision-making has been an ongoing and regularly cited feature of students’ perception of school or student councils across all five jurisdictions. this criticism challenges the concept of a student council as an instrument of student voice and as a construct for representative democratic participation international electronic journal of elementary education vol.8, issue2, 223-242, 2015 234 (alderson, 2000; fielding, 2007, 2011; hargreaves, 2004; keogh and whyte, 2005; rudduck and flutter, 2000; whitty and wisby, 2007; wyse, 2001). an experimental student council in an irish primary school identified a subordinated role for students (mcloughlin, 2004). frustration was evident on the part of students due to the slow pace of change, poor communication with the student body and the student’s perception of adult control in reference to teachers’ veto on discussion and decisions, and their imposition of sanctions (ibid.). while student voice is presented through the student council construct as a rights-based and citizenship project, the spectre of tokenism emerges: the reality of children’s present subordinate and ‘incomplete’ citizenship presents irish educators with an immediate challenge. token student councils should not be tolerated whereby they perpetuate this subordination (ibid., p. 141). research involving fourteen student councils in post-primary schools in ireland found differing perceptions of the council by school management and students (keogh and whyte, 2005). boards of management viewed the council as a consultative group, as a provider of information, as a communication channel and as a resource. teachers in the same study, perceived the student council as a forum for students’ concerns, peer support particularly in the area of social isolation and potential bullying, and as having a role in improving the school atmosphere. all the adults in the study identified the educational opportunity provided by the council to facilitate students to learn about teamwork, democratic processes, negotiation skills, and to identify issues and strategies for sustainable change in the school (ibid.). students’ views, in contrast, saw the expected role and purpose of the council as one of action: listening to students; representing students’ views; contributing to policy; providing feedback to the student body, and changing things like ‘school uniform, changing food in the canteen, fixing things, dealing with issues, solving problems, helping students and organizing events’ (ibid., p. 55). however students’ views on the effectiveness of the council were largely negative, citing apathy among the student cohort relating to difficulties in communication, representation and feedback. elitism were also identified as, citing one student, ‘only people who stand out and who work would get a position on the student council…people seen as messers wouldn’t have a chance’ (ibid., p. 83). its reflection of prefigurative democracy is also questioned as, in the view of another student ‘the student council is all for show, so that they can say it’s like a democracy, but it’s not…because at the end of the day, only the staff have a say’ (ibid., p. 83), a finding that echoes those of (arnot, mcintyre, pedder and reay, 2004; fielding, 2001). responses to a national audit of student councils conducted by the office of the minister for children and youth affairs (omcya) in 2011 indicated quite negative views from students. just 208 councils from a total of over 720 post-primary schools responded. the 29% response rate, in itself, was indicative of a weak visibility for student councils in irish post-primary schools. of those that did respond to the audit, the majority of responses pointed towards tokenistic and very limited involvements for the student council in decision-making with just 50% of councils feeling that their views were taken seriously by school management (omcya, 2011). while the majority of councils felt that they were consulted on issues relating to school rules and policy formation, a majority of council members also identified their limited impact on decisions making. case-study research on three student councils in post-primary schools, tracing council activities and involvements over one school year identified to a conflict between rhetoric and practice in students’ view of role (fleming, 2013). the students demonstrated that they had embraced the language of representation, of a student forum, of advocacy, action student voice: an emerging discourse in irish education policy / fleming 235 and support for students, however their of rhetoric and reality varied considerably from that of the student council liaison teachers and the school principals. it was evident that each student council could provide a construct for meaningful participation, prefigurative democracy and an experience of active citizenship for the students. however, the reality of the students’ experiences reflected tokenistic activity, contrived involvements with decision-making, and a significant focus on school event organisation or charity fundraising. the study concluded that the opportunity for a deep, person-centred student voice reflecting rights, participative democracy and active citizenship was not being realised through the current student council construct in these schools (ibid). thus, though the rhetoric of the role for the council in schools points towards visibility and potential for engagement and participation in democratic practice and for dialogic consultation in school decision making, the lived experience appears to be largely negative and somewhat tokenistic (fleming, 2013). a statement by the democracy commission (2005), [whose attention was focused on the capacity for democracy across the island of ireland ‘to be inclusive, participatory and egalitarian’ (ibid., p. xi) in the context of the good friday peace agreement between the parties in northern ireland] seems prescient as it concluded that: student councils give students a voice but not a say (democracy commission, 2005, p. 33) student voice: the motivation of evaluation external evaluation. it was the introduction of external evaluation and school selfevaluation during the period between 2002 and 2012 that extended the policy discourse on student voice in ireland following the establishment of student councils and the publication of the student council guidelines. the establishment of whole-school evaluation (wse) in 2004 had created a focus on the student council as a representative structure for students in a post-primary school as inspectors outlined their intention to interview the student council during evaluations to represent the views of students (des, 2004). a refinement of wse as whole-school evaluation: management, leadership and learning (wse-mll) included standardised questionnaires for students and their parents as part of school inspection (des, 2011a). this development provided a further channel for student voice in external evaluation. the questionnaires however contained closed questions and did not give the students provision for comment on issues or on their experiences in their school. inspectors also interviewed a representative focus group of students that included members of the student council in wse-mll evaluation process (ibid.). the combination of focus-group interviews and questionnaires visibly widened the voice of students in external evaluation. it is questionable however, whether this voice gave students the agency or power to effect changes in their experience of school. due to the nature of these exchanges with inspectors and their context within external evaluation it could be argued that the voice of the students was largely subjugated to that of a datasource in the evaluation process (fleming, 2013). similarly, while the wse process provided for direct post-evaluation oral feedback and dialogue on the findings of the evaluation to the board of management, teachers, and to the principal and deputy principal, this facility for feedback was not afforded to students (des, 2004, 2011a) beyond the provision of a draft copy of the evaluation report to the student council, through its chairperson. while it is open to school management to use comment from students to inform a school response to an inspection report in advance of its publication, guidance and procedures as to how comment from or feedback to students international electronic journal of elementary education vol.8, issue2, 223-242, 2015 236 might transact in schools were not outlined or discussed in evaluation guidelines (des, 2004, 2011a). school self-evaluation. the concept of school self-evaluation as a further evaluative instrument that emerged on to the education landscape during this period has the potential to increase the volume of student voice within post-primary schools. the initial development of school self-evaluation began with the publication of criteria for school self-evaluation (des, 2003). the document, ‘looking at our school’ also made reference to the student council within a very agentive subsection referring to ‘involvement of students in the operation of the school’ (ibid., p. 40). however, the language of the descriptor, from which the school would self-evaluate, reflects the language of the education act (1998) and the student council guidelines (des, 2002) by focusing the school on the evaluation of ‘the extent to which the school’s student council, in cooperation with management, parents and teachers, promotes the interests of the school and the involvement of students in the affairs of the school’ (ibid.). this descriptor, as with the text of the act, reduced ‘involvement’ in the ‘operation’ of the school to one of ‘promotion’ of the school and involvement ‘in the affairs of the school’. school self-evaluation was also included in the education pillar of the aforementioned ‘towards 2016’: the national economic and social partnership agreement (2006). the agreement stated that ‘the parties have agreed that each school will utilize the department of education and science publication(s) “looking at our school – an aid to self-evaluation in second level schools (2003)” to conduct a self-evaluation of school performance’ (towards 2016, 2006, 31.3, p. 126) at government policy level, reference in the programme for government presented in 2011, raised the profile of school self-evaluation. a specific reference to school performance in the context of a national literacy and numeracy initiative in 2011, also further raised the stakes for performativity as ‘a new system of self-evaluation will be introduced, requiring all schools to evaluate their performance year by year and publish information across a wide range of criteria’ (des, 2011b). it was the publication of school self-evaluation guidelines (inspectorate, 2012) however, that significantly advanced the student voice agenda in both primary and postprimary schools. the term student voice was particularly mentioned in the document in the context of a role for students in the process viewed as ‘the inclusion of the voice of students and parents in school self-evaluation processes’ (ibid., p. 9). significantly, however, these guidelines located student voice for the first time in the context of classroom practice where it was placed as an evaluation criterion for teachers underscoring the pedagogical importance of engagement and consultation. teachers were encouraged to evaluate student voice in their classrooms based on descriptors reflecting these dual elements: students’ contributions and questions are encouraged and welcomed in the classroom due account is taken of students’ views and opinions in accordance with their age and maturity (ibid., p. 42). the guidelines do not make any reference to a role for the student council in school self-evaluation but focus on a wider concept of student voice as ‘eliciting the views of students’ (ibid., p.50). it is clear that these references to student voice viewed it as largely instrumental in the context of low-level participation in the provision of data. the guidelines fail to develop any sense of deep dialogic consultation with students or the methods to structure these engagements though they do make reference to the use of questionnaires, interviews and reflections (fleming, 2013). nevertheless, an emphasis is student voice: an emerging discourse in irish education policy / fleming 237 placed on accountability reflected in the requirement on schools to produce a school selfevaluation report and a school improvement plan (inspectorate, 2012). school self-evaluation, as introduced in 2012, arguably represents the most significant and visible advance for the voice of students in pedagogy and in consultation in school decision-making in ireland to date (fleming, 2013). encouraging ‘students’ contributions and questions’ and ‘students’ views and opinions’ within an official educational policy document is a significant advance from ‘involvement of students in the affairs of the school’ (education act, 1998, 27:3). despite this, two significant risks emerge from these developments. in moving away from the student council as a representative student voice, arguably, an instrumentalist voice is being encouraged and directed primarily towards the gathering of data to inform and measure school performance and school improvement. consequently, there is a risk that the opportunity to develop and embed deep student voice, as meaningful consultation, co-construction and the creation of rights-based, dialogic, person-centred democratic and inclusive schools, will be diminished or even lost. equally, the growing association between student voice, school improvement and performativity, through various forms of evaluation presents a further risk as these imperatives could lose any interactive potential for change at school and classrooms level based on right, trust, relationship and learning. the opportunity to advance student voice as a human dialogic interaction within an inclusive classroom and school culture could be intercepted by a drive towards measurable improvements in standards (fleming, 2013). conclusions as a concept, student voice is complex and contested in its positioning within policy and practice as reflected in irish schools. as an emerging discourse in irish education, its definition, motivation and translation into practice still requires discussion, debate and clarification at both policy and school level. student voice has emerged on to the irish education landscape from a rights-based perspective arising from the uncrc. it was developed through the policy discourse within a student council construct that was motivated by a combination of the vindication of the rights of the child to have voice in consultative decision making in schools, the emergence of a recognition of the child and young person as active and agentive in their learning, engagement and participation in schools, and the need to develop and sustain education for democratic citizenship. notwithstanding citizenship curricular initiatives, it was the student council construct that became the main vehicle for student voice within each of these motivations. while the policy discourse defines and bounds the operation of the student council in schools, it is clear that the construct has been circumscribed by the power and authority discourses of school management. it is similarly circumscribed by the perceived pressure of curriculum delivery, by internal and external assessments of students, and by school evaluation, arguably within a performance-orientated and outcomes-driven script that reflects a neoliberal and consumerist agenda for schools and education. in this context, the student council, as a construct for student voice within a right-based, consultative and democratic citizenship perspective, is largely tokenistic and functionally redundant. student voice has also emerged from the policy discourse initially within external evaluation of schools and latterly through the school self-evaluation policy discourse. while this discourse reflects the language of a rights-based perspective for student voice it also includes reference to school performance and has the potential to further subvert any pedagogical and wider rights-based and democratic-citizenship motivations and potential for student voice. although school self-evaluation arguably represents the most significant and visible advance for student voice in schools in ireland to date, its motivation and focus, as an international electronic journal of elementary education vol.8, issue2, 223-242, 2015 238 externally-mandated policy initiative is clearly on school improvement and performativity. this policy initiative lacks any significant motivation towards student voice as a personcentred dialogic interaction within an inclusive classroom and school culture. the opportunities, challenges and contestation of student voice within policy and practice in irish education therefore continue. • • • acknowledgements the author wishes to acknowledge and sincerely thank dr. mary horgan (ucc) for her assistance and support in the completion of the research that has formed the foundation for this paper. domnall fleming, received his ph.d. degree in education from university college, cork, in 2013 and is a senior inspector with the inspectorate of the department of education and skills. his research interests lie particularly in the area of student voice and agency in education. references alderson, p. (2000). school students’ views on school councils and daily life at school. children and society, 14 (2), 121-134. arnot, m. and reay, d. (2007). a sociology of pedagogic voice: power, inequality and pupil consultation. discourse: studies in the cultural politics of education, 28, 311-325. arnot, m., mcintyre, d., pedder, d. and reay, d. (2004). consultation in the classroom: developing dialogue about teaching and learning. cambridge: pearson publishing. ball, s. (ed.) (1990). foucault and education: disciplines and knowledge. london: routledge. bragg, s. (2001). taking a joke: learning from the voices we don’t want to hear. forum, 43, 70-73. bragg, s. (2007). consulting young people: a review of the literature. london: arts council england. chadderton, c. (2011). not capturing voices: a poststructural critique of the privileging of voice in research. in czerniawski, g. and kidd, w. (eds.), the student voice handbook: bridging the academic / practitioner divide. bingley: emerald group publishing ltd. chapman, j. (1970a). the origins and development of school councils. new era, 51(8), 268-279. children in scotland (2010). having a say at school and research on pupil councils in scotland. research briefing paper 1: local authorities and pupil councils. connolly, p. (1997). in search of authenticity: researching young children’s perspectives. in pollard, a., thiessen, d. and filer, a. (eds.), children and their curriculum: the perspectives of primary and elementary school children. london: the falmer press. cook-sather, a. (2006). sound, presence, and power: “student voice” in educational research and reform. curriculum inquiry, 36, 359-390. crick, b. (1998). education for citizenship and the teaching of democracy in schools: final report of the advisory group on citizenship. qualifications and curriculum authority (uk). dahlberg, g., moss, p. and pence, a. (1999). beyond quality in early childhood education and care: postmodern perspectives. london: falmer press. student voice: an emerging discourse in irish education policy / fleming 239 democracy commission (2005). the report of the democracy commission: engaging citizens: the case for democratic renewal in ireland. department of children and youth affairs, (2014). the national policy framework for children and young people 2014 – 2020. dublin. department for education and skills (dfes, uk) (2004). a national conversation about personalised learning. department for children, schools and families (dfcsf, uk) (2008). working together: listening to the voices of children and young people. department of education and science (des) (1995). civic social and political education syllabus. dublin: the stationery office. department of education and science (des) (2002). student councils: a voice for students. dublin: the stationery office. department of education and science (des) (2003). looking at our school: a guide to school selfevaluation in post-primary schools. dublin: the stationery office. department of education and science (des) (2004). whole-school evaluation in post-primary schools: guidelines for schools. dublin: the stationery office. department of education and skills (des) (2011a). a guide to whole school evaluation – management, leadership and learning. dublin: the stationery office. department of education and skills (des) (2011b). literacy and numeracy for learning and life. dublin: the stationery office. devine, d. (2001). locating the child's voice in the primary school. in cleary, a., nic ghiolla phádraig, m. and quin, s. (eds.), understanding children: state, education and economy. cork: oaktree press. devine, d. (2002). children’s citizenship and the structuring of adult/child relations in the primary school. childhood, 9, 303-321. devine, d. (2003). children, power and schooling: how childhood is structured in the primary school. stoke-on-trent: trentham books. devine, d. (2004). school matters listening to what children have to say. in deegan, j., devine, d. and lodge, a. (eds.), primary voices: equality, diversity, and childhood in irish primary schools. dublin: institute of public administration. devine, d. (2009). mobilising capitals? migrant children's negotiation of their everyday lives in school. british journal of sociology of education, 30, 521-535. durr, k. (2004). education for democratic citizenship 2001-2004. the school: a democratic learning community. the all-european study on pupils’ participation in school. council of europe publishing. dworkin, g. (1977). taking rights seriously. london: duckworth. education act (1998). dublin: government publications. education act (2002). london: uk government. ellsworth, e. (1989). why doesn’t this feel empowering? working through repressive myths of critical pedagogy. harvard educational review, 59 (3), 297-324. fielding, m. (2001). beyond the rhetoric of student voice: new departures or new constraints in the transformation of 21st century schooling. forum, 43, 100-110. fielding, m. (2004). new wave’ student voice and the renewal of civic society. london review of education, 2, 197-217. international electronic journal of elementary education vol.8, issue2, 223-242, 2015 240 fielding, m. (2007). the human cost and intellectual poverty of high performance schooling: radical philosophy, john macmurray and the remaking of person-centred education. journal of education policy, 22(4), 383-409. fielding, m. (2011). student voice and the possibility of radical democratic education: re-narrating forgotten histories, developing alternative futures. in czerniawski, g. and kidd, w. (eds.), the student voice handbook: bridging the academic / practitioner divide. bingley: emerald group publishing ltd. fielding, m. and mcgregor, j. (2005). deconstructing student voice: new spaces for dialogue or new opportunities for surveillance? paper presented at the american educational research association (aera), montreal, canada. fleming, d. (2013). student voice in irish post-primary schools – a drama of voices. university college cork, unpublished phd thesis. foucault, m. (1979). discipline and punish: the birth of the prison. new york: random house. freeman, m. (1987). taking children’s rights seriously. children and society, 4, 299-319. freeman, m. (1996). children’s education; a test case for best interests and autonomy. london: fulton. gore, j. (1993). the struggle for pedagogies: critical and feminist discourses as regimes of truth. london: routledge. government of ireland (1995). charting our education future: white paper on education. dublin: government publications. granville, g. (2004). politics and partnership in curriculum planning in ireland. in sugrue, c. (ed.), curriculum and ideology: irish experiences international perspectives. dublin: the liffey press. hargreaves, d. (2004). personalising learning 2: student voice and assessment for learning. london: specialist schools trust. huddleston, t. (2007). effective practice in democratic school governance in european schools. london: citizenship foundation. inspectorate (2012). school self-evaluation: guidelines for post-primary schools. dublin: department of education and skills. kelly, a. v. (1995). education and democracy: principles and practices. london: paul chapman publishing ltd. keogh, a, and whyte, j. (2005). second level student councils in ireland: a study of enablers, barriers and supports. dublin: the children’s research centre, trinity college dublin. kilbourne, s. (1998). the wayward americans—why the usa has not ratified the united nations convention on the rights of the child. child and family law quarterly, 10, 243-256. lodge, a., devine, d. and deegan, j. (2004). equality, diversity and childhood in irish primary school. in deegan, j., devine, d. and lodge, a. (eds.), primary voices: equality, diversity, and childhood in irish primary schools. dublin: institute of public administration. lodge, c. (2005). from hearing voices to engaging in dialogue: problematizing student participation in school improvement. journal of educational change, 6, 125-146. lodge, c. (2008). student voice and learning-focused school improvement. research matters, 32, 2-15. lundy, l. (2007). 'voice’ is not enough: conceptualising article 12 of the united nations convention on the rights of the child. british educational research journal, 33, 927-942. lynch, k. and lodge, a. (2002). equality and power in schools: redistribution, recognition and representation. london: routledge falmer associates.. student voice: an emerging discourse in irish education policy / fleming 241 macnaughton, g. (2003). shaping early childhood: learners, curriculum and contexts. berkshire: open university press. marshall, j. d. (1990). foucault and educational research. in ball, s. (ed.), foucault and education: disciplines and knowledge. london: routledge. mccowan, t. (2011). political learning through the prefigurative: the case of the brazilian landless movement. in czerniawski, g. and kidd, w. (eds.), the student voice handbook: bridging the academic / practitioner divide. bingley: emerald group publishing ltd. mcloughlin, o. (2004). citizen child – the experience of a student council in a primary school. in deegan, j., devine, d. and lodge, a. (eds.), primary voices: equality, diversity, and childhood in irish primary schools. dublin: institute of public administration. mitra, d. (2001). opening the floodgates: giving students a voice in school reform. forum, 43, 91-94. mitra, d. (2003). student voice in school reform: reframing student-teacher relationships. magill journal of education, 38, 289-304. mitra, d. (2004). the significance of students: can increasing" student voice" in schools lead to gains in youth development? teachers college record, 106, 651-688. mitra, d. (2007). student voice in school reform: from listening to leadership. in thiessen, d. and cook-sather, a. (ed.), international handbook of student experience in elementary and secondary school. new york: springer. national children's strategy (2000). dublin: government publications. national council for curriculum and assessment (ncca), (2009). politics and society: draft syllabus for consultation. dublin. national education convention secretariat (1994). report on the national education convention. nieto, s. (1994). lessons from students on creating a chance to dream. harvard educational review, 64, 392-426. northern ireland assembly (2011). school councils. research and information service research paper. noyes, a. (2005). pupil voice: purpose, power and the possibilities for democratic schooling. british educational research journal, 31, 532-540. office of the minister for children and youth affairs (omcya), (2011). an audit of children and young people’s participation in decision-making structures. dublin. o’gorman, a. (1998). student councils why bother? issues in education, 3, 183-187. reay, d. (2006). i’m not seen as one of the clever children: consulting primary school pupils about the social conditions of learning. educational review 58, 171-181. referendum commission (2012). the children referendum: www.referendum 2012.ie. rose, r. and shevlin, m. (2010). count me in! ideas for actively engaging students in inclusive classrooms. london: jessica kingsley publishers. rudduck, j. (2005). pupil voice is here to stay! qualifications and curriculum authority qca. (uk). rudduck, j. (2006). the past, the papers and the project. education review, 58, 131-143. rudduck, j. (2007). student voice, student engagement and school reform. in thiessen, d. and cooksather, a. (eds.), international handbook of student experience in elementary and secondary school. new york: springer. rudduck, j. and flutter, j. (2000). pupil participation and pupil perspective: `carving a new order of experience’. cambridge journal of education, 30(1), 75-89. rudduck, j. and flutter, j. (200). how to improve your school: giving pupils a voice. london: continuum. international electronic journal of elementary education vol.8, issue2, 223-242, 2015 242 shevlin, m. and rose, r. (2003). recognising voices. in shelvin, m. and rose, r. (eds.), encouraging voices: respecting the insights of young people who have been marginalised. dublin: national disability authority. soohoo, s. (1993). students as partners in research and restructuring schools. the educational forum, 57, 386-393. taylor, c. and robinson, c. (2009). student voice: theorising power and participation. pedagogy, culture and society, 17(2), 161-175. thiessen, d. (2007). researching student experiences in elementary and secondary school: an evolving field of study. in thiessen, d. and cook-sather, a. (eds.), international handbook of student experience in elementary and secondary school. new york: springer. thomson, p. (2011). coming to terms with voice. in czerniawski, g. and kidd, w. (eds.), the student voice handbook: bridging the academic / practitioner divide. bingley: emerald group publishing ltd. towards 2016 (2006). ten-year framework social partnership agreement 2006-2016. dublin: government publications. trant, a. and ó donnabháin, d. (1998). the way forward. city of dublin vocational education committee curriculum development unit. united nations (1992). united nations charter on the rights of the child. welsh assembly government (2005). the school councils (wales) regulations. welsh assembly government (2009). school councils in wales. whitty, g., and wisby, e. (2007). real decision making? school councils in action. institute of education, university of london. wyse, d. (2001). felt tip pens and school councils: children's participation rights in four english schools. children and society 15, 209-218. microsoft word iejee_5_1_editorial international electronic journal of elementary education, 2012, 5(1) issn:1307-9298 copyright © iejee www.iejee.com dear iejee readers, this special issue of the international electronic journal of elementary education (iejee) is devoted to one of the most current and important educational topics of our time: learning and instruction in the natural sciences (k-10 focus). drs. florencia k. anggoro and benjamin d. jee of the college of the holy cross, usa, have done a great job as special issue editors. i am very grateful for the scientific editorialship they have accomplished as two active scholars in their field. our generation has experienced many technological developments, innovative solutions and challenging aspects of information society and knowledge society. as contemporary educators we have been witnesses to the debates on learning and instruction in the natural sciences in our countries. since the 1990s comparative international studies, i.e. the trends in international mathematics and science study (timss) and the new demands in the national and globalized labor markets revealed that the educational systems in all countries have to reconsider their existing approaches to the teaching of natural sciences. as the world is transforming to a knowledge society, our educational systems necessitate appropriate perspectives on and approaches to curriculum development, teaching methods, and the creation of learning environments for natural sciences. all are for the sake of our children, our environmental consciousness, and our future. dr. anggoro and dr. jee have succeeded in bringing together papers on this challenging topic from prominent researchers. i am sure, as a reader, you’ll be delighted to have the opportunity to read the papers that were written with the aim of contributing to enhancing learning and instruction in the natural sciences. you’ll find creative papers and papers that present new ideas and trends and papers that inform us about the new approaches in this area. regardless of whether we are teachers, teacher educators, and/or educational researchers, this special issue of the international electronic journal of elementary education (iejee) will give us several useful ideas originating from applied cognitive theory. this well presented special issue hopefully will be one of our sources of inspiration to continue to be a part of the amazing educational field of learning and instruction in the natural sciences. this affiliation include some commitments: working for a better learning and teaching environments in natural sciences in all stages of our educational systems. this is due to the fact that the natural sciences as a school subject suffers alack of qualified teachers and interest from young students. i hope this special issue will make its contribution and will inspire more and more youngsters in our countries and environments to be a part of our circle of natural scientists. this is the fourth special issue of iejee since 2006. without tireless efforts of dr. anggoro and dr. jee, it would not be possible to present this issue. i want to express my gratitude to both of them. i also want to thank all the contributors to this special issue. it’s a great honor for me to make their knowledge accessible to the readers in the entire world. international electronic journal of elementary education vol.5, issue 1, 2012 vi i also want to thank dr. karen michele zabrucky, dr. turan temur and dr. gökhan özsoy for all the academic advising, professional coordination, and the tireless technical contribution they have done for the realization of this special issue. dr. kamil özerk editor in chief microsoft word iejee_4_3_luongo international electronic journal of elementary education, 2012, 4(3), 519-544. issn:1307-9298 copyright © iejee www.iejee.com increasing elementary school teachers' awareness of gender inequity in student computer usage nicole luongo ∗∗∗∗ saint peter’s university, united states received: july, 2012 / revised: january, 2012 / accepted: april, 2012 abstract this study was designed to increase gender equity awareness in elementary school teachers with respect to student computer and technology usage. using professional development methods with a group of teachers, the writer attempted to help them become more aware of gender bias in technology instruction. an analysis of the data revealed that teachers who were exposed to gender equity professional development training sessions were more likely to exhibit gender equitable teaching behaviors than they did prior to the sessions. the data also indicated that teachers provided more equitable assistance to their classroom students after being presented with gender equity interventions. keywords: technology, gender, bias, elementary introduction after the ratification of title ix of the education amendments of 1972, america's elementary schools have been mandated to provide equal educational opportunities for both girls and boys (shapiro, kramer & hunerberg, 1981). title ix stated, "no person in the united states shall on the basis of sex be excluded from participation in, be denied the benefits of, or be subjected to discrimination under any education program or activity receiving federal financial assistance" (sec. 1681). according to american association of university women (aauw) educational foundation (1992), even with title ix of the education amendments of 1972 guidelines, teachers and school administrators often struggle with the law and its regulations. this study examined the attitudes of teachers regarding gender and technology in the elementary school. title ix of the education amendments of 1972 produced parameters that ∗ nicole luongo, saint peter’s university, 2641 john f. kennedy boulevard , jersey city, nj 07306, united states phone: +12017616187. e-mail: nluongo@spc.edu. international electronic journal of elementary education, 2012, vol.4, issue 3, 519-544 520 educators are required to follow help prevent gender discrimination (aauw educational foundation, 1992). the study connected those boundaries to the fields of technology and technology education in a school in the state of new jersey. the school involved in this study is a public institution located in a suburban middle-class section of northern new jersey. statement of the problem the problem that this study addressed was that primary-level teachers in the school district were not providing equal or equivalent technology experiences to both genders. previous research (sanders, koch & urso, 1997) suggested that male and female students are on the same level in terms of technology ability and usage until middle school but, thereafter, the abilities of male students surpass that of the female students. this study, therefore, examined whether teacher attitudes from the primary years had any effect on the computer usage of boys and girls in elementary school. through informal interviews, the writer found that the teachers in this elementary school agreed that primary-level teachers in the school district did not provide equal or equivalent technology experiences for both genders. several of the teachers admitted that they often favored the male gender in using educational technology. furthermore, several teachers found that even as early as elementary school, the girls did not choose to use computer technology in school projects as often as did the boys. some of the participating teachers claimed that the female students selected traditional methods of learning and research over technological approaches because they did not have as much access to or experience with computer technology as did the male students. the study was conducted in a kindergarten to grade 8 school district located in a suburban, middle class section of new jersey. the town has a population of 6,473 individuals (u.s. census, 2000). of these individuals, 49% are male and 51% are female. at the time of this study, the socioeconomic status of the average resident was middle class with 74.9% of the residents in the u.s. work force. in the school district, there is one elementary school that accommodates prekindergarten through third-grade students and one middle school that provides educational services to fourth-grade through eighth-grade students. at the time of the study, there were approximately 615 students enrolled in the school district with 290 in the selected school. the two schools in the school district have been wired for the internet. in fact, all classrooms have internet access and are equipped with at least two computers for student use. an underground fiber-optic line connects the two schools and the administration building. the school district encourages teachers to incorporate the use of technology into their instruction; the teachers are often provided with professional development opportunities to successfully infuse technology into the curriculum. when this study was conducted, the school district staff consisted of 60 teachers, 1 superintendent, 1 middle school principal, 1 elementary school principal, 1 library media specialist, and 1 technology coordinator. the library media specialist did not teach computer technology. the technology coordinator worked 4 days a week and provided computer instruction for the secondand third-grade classes. there was 1 full-time technology teacher who provided computer instruction to all fourthto eighth-grade students. the purpose of this project was to increase teacher gender equity awareness levels with respect to student computer technology usage. additionally, the writer aimed to help teachers become aware of gender bias in technology instruction and to provide a more gender equitable approach to technology usage. increasing elementary school teachers' awareness of gender inequity / luongo 521 the administration of the school district and the school district's technology committee argued that the abovementioned technology improvements allowed teachers to provide all students with an equitable high-quality technology education. moreover, the administration provided much encouragement to teachers who participated in technology training and implemented its use into their teaching. the administration urged teachers to use technology in their teaching, but it was not obligatory. with these accessible technological opportunities, teachers should have been able to provide equal or equivalent technology educational experiences to all students--but were they? the writer's purpose in this project was to increase the teacher gender equity awareness to meet these goals. research questions this study examined several gender and technology-related questions. these questions guided the study, and were reviewed and reevaluated during the course of the study: 1. did the teachers and students perceive that all students had equal and adequate access to the classroom computers? 2. what were the teachers' attitudes regarding gender and educational ability? 3. did the teachers' attitudes regarding gender correlate with the provisions they made for technology use in their classrooms? methods the study involved a general group of 20 elementary school teachers, some of whom were later selected as a sample group of teachers who were directly involved in the study. it also involved the 80 elementary school students from the classes of the participating teachers. the selected group of teachers was identified by means of a preliminary questionnaire. after the 20 teachers completed the questionnaire, the writer randomly chose one teacher from each grade level in the school from kindergarten through third grade by placing the names into a pool and choosing them without preference. in table 1, the number of students in each of the selected teachers' classes is displayed. table 1. number of students from the selected teachers' classes teacher no. of students kindergarten 19 grade 1 18 grade 2 23 grade 3 20 the target population for the study included all teachers in the school (gall, borg & gall, 1996). the study then utilized an accessible population, which was a practical option of using individuals who could realistically be included in the sample. the general group of teachers was classified as those teachers who replied to the initial questionnaire. gall et al. explained that a convenience sample is a group of cases that are selected simply because they are available and accessible. a convenience sample group of four teachers was selected from within this general group, which became the selected sample group of teachers for the study. international electronic journal of elementary education, 2012, vol.4, issue 3, 519-544 522 the student population included all of the students in the classes of the selected sample group of participating teachers in the study. in order to choose students for the focus groups, the writer randomly selected students from the population by using a systematic sampling procedure. a list of all students in the population was created. then, the population of students was divided by four, which was the number needed for the sample (gall et al., 1996). next, the writer selected the students and distributed permission slips to them. the permission slips were photocopied and handed to the children's homeroom teachers to be distributed. all of the children in the selected classes returned the permission slips. the children who returned the permission slips were placed into a group of participants. instrument the quantitative research measure that was used in this study was the questionnaire, and was accompanied by a cover letter. the purpose of the questionnaire was to obtain information about all the teachers in the school, their technology backgrounds, and their views of gender and technology. the study included one computer laboratory observation of the selected sample group of teachers and their students before the intervention and one after the intervention. the school computer laboratory was the location of each observation during the teacher's scheduled technology period. the writer measured the amount of time the teachers spent with girls and with boys in connection with computer technology education. during these observation periods, the writer acted as a complete observer and maintained what gall et al. (1996) called "a posture of detachment from the setting being studied"(p. 345). structured observation methods were used to observe specific behaviors of the teachers regarding their treatment of the boy and girl participants. the writer used a quantitative, tailored observation system (glickman, gordon & ross-gordon, 1995) designed to collect these four specific types of behaviors: 1. how often the teacher assisted male students. 2. how often the teacher assisted female students. 3. how often the teacher provided positive feedback to male students and female students. 4. how often the teacher provided negative feedback to male students and female students. an example of the observation instrument can be viewed in figure 1. the writer used this set of data to examine the patterns of the selected teachers in relation to their interactions with male and female students and their technology teaching behaviors. the data by analyzed by tallying the total number of instances of assistance to the female students and the total number of instances of assistance to the male students. the number of positive responses given to male students and compared that number to the number of positive responses given to female students were tallied. likewise, the writer distinguished between the number of negative responses given to male students and the number of negative responses given to the female students. the analysis included a gender comparison of these variables. increasing elementary school teachers' awareness of gender inequity / luongo 523 b +++ g l b l b -- g g + b -- b b b ++ll g +++ g g --- g +++++ g ll b l b --ll b ll g llll b ll++- g figure 1. question-response observation instrument for measuring gender bias. b = boy, g = girl, 4 = teacher assistance to student, = positive response to student, + = negative response to student. additionally, the writer conducted individual semistructured confirmation survey teacher interviews with the four selected teachers before and after the intervention. an interview guide was used and, although it was structured, the interviewer was prepared to make any necessary modifications if the interview did not go as planned. furthermore, the writer used focus group interviews to examine student perceptions of gender equity in their teacher's instructional methods. the writer conducted four sets of focus group interviews. each group was interviewed before and after the intervention. throughout the study, the writer monitored the four selected teachers in the computer laboratory completing the preintervention observations. the writer met with each participant before the observations to discuss appropriate observation times. the tailored question-response observation instrument for measuring gender bias was used to collect data on the gender-biased behaviors (see figure 1). additionally, the writer interviewed each teacher and conducted the student focus groups at convenient times and places. after collecting the initial data, an action project was implemented in which the writer worked with the four selected teachers to promote gender equity in technology education. rubin (2000) discussed certain strategies that could be used to transform attitudes toward gender stereotypes and behavior through school programs. there were three goals of the teacher intervention strategies: 1. to learn about gender neutral teaching strategies. 2. to learn about ways computers could support and enhance student learning. international electronic journal of elementary education, 2012, vol.4, issue 3, 519-544 524 3. to consider ways to use the available computer resources equitably in their classrooms. research (lundeberg, 1997; sanders, 2003b) documented intervention strategies that include the presentation of gender equity sessions. four after-school workshops were conducted. sanders explained that a one-shot workshop by itself is unlikely to do much. for the workshop to be effective, it must have a follow-up. follow-ups can include multiple workshops and specific activities that occur as a result of the workshop. when creating a gender equity workshop, sanders (2003b) suggested using six rules: (a) prepare extensively, (b) be factual, (c) use no blame, (d) do not bash males, (e) demonstrate support, and (f) remember the what's in it for me? rule. sanders explained that the workshop must be relevant to the participant's concerns and it must demonstrate that there is a gender problem that needs to be addressed. because a problem was established in the pretesting phase of the study, the writer was able to present this finding at the workshop. during this time, the writer conducted four after-school gender equity sessions with the four selected sample group of teachers. these sessions presented gender equity and technology information to the teachers in order to improve their knowledge of gender equity and technology education. after the intervention strategies were completed, the outcomes of the project were evaluated by reevaluating the attitudes of teachers and students. the questionnaire was administered to the general group of teachers, and compared the original questionnaire results with that of the subsequent one. the writer looked for a change in the responses of the participants. the follow-up computer laboratory observations were conducted in the computer laboratory. the writer observed the selected sample group of teachers. the writer and teachers decided on appropriate observation times and discussed what lessons would be occurring during those time periods. additionally, the writer interviewed the selected sample group of teachers and students to see if they perceived a change in the teacher attitudes. during the interviews, the same questions in the initial interview were asked. the writer compared these findings to the initial results to see if a change had occurred. results & discussion the purpose of this study was to increase the study participants' awareness of genderrelated issues in association with technology. the pretesting results had indicated that the teachers in the school district were not providing equal or equivalent technology experiences for both genders. in order to remedy this situation, the elementary school teachers were exposed to several gender equity intervention strategies. based on follow-up interview and observational data, the intervention strategies had proved to be successful. it was found that the participating teachers in this study from all of the grade levels had made the effort of distributing equitable attention to the boys and girls. they understood the impact that their attitudes and behaviors regarding gender could have had in their classrooms and computer laboratories. in this study, several gender and technology-related questions were examined. the questions were divided into three main areas of interest: (a) technology access, (b) teacher attitudes regarding gender and technology, and (c) technology provisions and gender. increasing elementary school teachers' awareness of gender inequity / luongo 525 technology access three questions were asked on technology access: 1. did the teachers and students perceive that all students had equal and adequate access to the classroom computers? 2. what were the teachers' attitudes regarding gender and educational ability? 3. did the teachers' attitudes regarding gender correlate with the provisions they made for technology use in their classrooms? the teacher questionnaire, the teacher interviews, the student interviews, and the intervention session discussions provided the data with respect to these questions. these data-gathering techniques allowed the writer to obtain information in a variety of ways to see if the verbal and written reactions corresponded with the actions of the teachers in the study. they provided an insight to the question of the student perceptions of technology access matching the teacher perceptions. the general group of teachers who responded to the questionnaire and the selected sample group of teachers seemed to hold similar views at the initial data collection point. on the whole, the majority of the teachers in these two groups claimed that they were not satisfied with the amount of time given to technology use and the computer laboratory. the teachers in the general group produced a variety of reasons why they were dissatisfied with the amount of time given to the use of technology. a first-grade teacher explained that the lack of use technology as much as was due to pressures and responsibilities of teaching. another teacher explained there were not enough computers in the school for those students to have access when they were needed. a second-grade teacher claimed, "when i want to get into the lab, the schedule is usually full. also, the computers and printers in my classroom are often broken and the help does not get here quick enough." one third-grade teacher contended that "there is not enough time in the day" to get all the students on the computers. another third-grade teacher maintained, "i do not get to use the computers as much as i would like to. i just do not have enough time to use them." likewise, the selected sample group were displeased with the amount of time spent on technology. the kindergarten teacher claimed, "i would like to use the computers more with the children. it is very difficult to get my students logged on to the computers in the lab by myself; the children are just learning to spell their names." the first-grade teacher asserted, "i would love to have a formal computer class taught by a computer teacher." the third-grade teacher stated that that class did not get into the computer laboratory as much as is desirable. on the other hand, there were several teachers in the general group who were content with the amount of time they spent using technology in the classroom and the computer laboratory. one third-grade teacher explained that "the students get 40 minutes of instruction and 30 minutes of practice," which that teacher claimed was enough time for those students. a first-grade teacher claimed having a belief that the additional time the technology director assigned for using the computer lab furnished adequate time for using technology. only one teacher in the selected sample group perceived satisfaction with the amount of time used in technology. the second-grade teacher maintained, "it does not dictate or control my lessons, but it adds to the learning." regarding the computer laboratory, the same teacher explained, "there is so much to do in my classroom that for now, it allows me international electronic journal of elementary education, 2012, vol.4, issue 3, 519-544 526 to do what i want to do." that teacher explained that the technology adds to the class lessons, but is not necessary to all of that teacher's goals. many of the views and attitudes of the general group and selected sample group changed regarding technology after the intervention sessions were conducted. although the general group of teachers did not take part in the intervention sessions, they did experience certain changes. these changes were focused on common technology issues, not necessarily gender issues. the selected sample of teachers took part in the intervention sessions, and experienced evident changes in their opinions specific to gender and gender equity. the changes in the general group's opinions after the intervention sessions focused on common technology usage issues. a first-grade teacher from the general group explained that that teacher started using the interactive whiteboard in class lessons due to the professional development training that the school provided. another first-grade teacher agreed that the training received from workshops has made that teacher more comfortable using the technology. a third-grade teacher explained that having an assigned time in the computer lab motivates the teacher and the students. most of these teachers did not mention gender equity issues in their postintervention responses. after the intervention sessions, the selected sample group's view changes were directed towards gender equity and technology access. the second-grade teacher in the sample group asserted, "i started using the lab more often this year because it was necessary to get all the kids on the computers." that teacher explained that all girls and all boys needed to get more access to technology through the process of this study. the first-grade teacher expressed an awareness of the gender inequity in the classroom. the third-grade teacher was interested in following up by making sure the future technology classes of that teacher were provided with equitable time and access. during both rounds of student interviews, most of the students perceived that they did not have equal access to the computer. however, a single reason was not apparent. each group of students gave a different reason why they did or did not have equal access to technology. the kindergarteners believed that the students who finish their morning snacks were the most likely ones to use the computers. the first-grade boys believed that their teachers got to use the computers the most of the time; the first-grade girls said it was equal within the class. three of the second graders claimed that there was a specific male student who made use of the computers the most. one of the students explained that that male student needed remedial help. although the third graders perceived that they were able to use technology equally, they were unable to give any reason for this explanation. there was a correlation between the selected sample group of teachers' and students' views of technology access. on a whole, the teachers believed that they did not have sufficient access to technology. likewise, the students did not feel they had enough access to technology. however, the percentage of students who were displeased decreased after the intervention sessions were completed. in the preintervention interviews, 69% of the students in all grades did not believe they had enough access to technology. in the postintervention interviews, the number of students decreased to 56% of students who believed they did not have enough access to technology. teacher attitudes regarding gender and technology there were three questions asked concerning teacher attitudes on gender and technology: 1. did the teachers and students perceive that all students had equal and adequate access to the classroom computers? increasing elementary school teachers' awareness of gender inequity / luongo 527 2. what were the teachers' attitudes regarding gender and educational ability? 3. did the teachers' attitudes regarding gender correlate with the provisions they made for technology use in their classrooms? the questionnaire, the teacher interviews, and the intervention session discussions collected information about the teachers and their attitudes about gender and gender bias. the initial and follow-up questionnaire results provided data regarding the teacher levels of perceived gender bias at both the preintervention and postintervention sessions (best & kahn, 1993). the teacher responses in the questionnaire provided the writer with data to rank them as possessing high, moderate, or low levels of perceived gender bias. overall, the sample group of selected teachers involved in the intervention showed more of a change of attitude in terms of perceived gender bias than the general group. although some of the teachers involved in the intervention sessions initially claimed that their classes were free of gender bias, they discovered and admitted they were surprisingly mistaken. through the discussions and activities that occurred during the intervention phase of the study, the selected sample group of teachers realized that they did possess certain gender biases that they exhibited in their classrooms and the computer laboratory. they claimed that they often provided more assistance to the boys for a variety of reasons. the most popular reason given by the teachers during the interviews and informal discussions was the aggressive nature of boys. the teachers explained that boys often call out more often and seek assistance from the teacher; the girls are more passive and wait for the teacher to approach them. the results of the preintervention computer laboratory observation of the selected sample of teachers can be viewed in table 2. the writer collected data on the number of male and female students as well as the number of times the assisting teacher gave positive or negative responses to each gender. the writer calculated the results to show the total number of assists for each gender, which can be seen in table 3. table 2. computer laboratory preintervention observation data category kindergarten grade 1 grade 2 grade 3 total no. of students girls 9 6 10 9 34 boys 10 11 10 13 44 teacher assistance girls 8 10 17 1 36 boys 19 15 20 11 65 positive responses girls 1 2 3 1 7 boys 11 3 8 3 25 negative responses girls 0 1 0 7 1 boys 1 7 7 5 20 at the conclusion of the intervention sessions, the writer observed the selected sample group of teachers in the computer laboratory. the writer collected data with respect to the international electronic journal of elementary education, 2012, vol.4, issue 3, 519-544 528 number of male and female students and the number of times the assisting teacher gave positive or negative responses to each gender. the results of the postintervention observation can be seen in table 4. after tabulating the initial data, the writer calculated the results to show the total number of assists for each gender (see table 3). table 3. total number of preintervention and postintervention responses by gender responses of the girls responses of the boys ______________________________ _____________________________ category preintervention postintervention preintervention postintervention kindergarten 9 31 27 26 grade 1 13 25 32 34 grade 2 20 35 15 17 grade 3 2 19 19 21 total 44 110 93 98 the intervention sessions that were conducted with the selected sample group of teachers presented information that added to the development of conclusions to the study. all of the computer professional drawings that the teachers created during the intervention sessions were of women. the students' drawings contained both female and male computer professionals. in table 5, the division between male and female computer professional student drawings by grade level is provided. the drawings were divided according to gender; the teachers discussed specifics of what their children drew and the reasons behind the drawings. during the second intervention session, the writer shared the data that was obtained from the computer laboratory observations (see table 2) with the selected sample of teachers. the teachers and the writer discussed the statistics as well as the reasons behind the results. this meeting provided all teachers present with an opportunity to collaborate and examine the data. table 4. computer laboratory postintervention observation data category kindergarten grade 1 grade 2 grade 3 total no. of students girls 9 7 10 9 42 boys 11 10 10 11 35 teacher assistance girls 12 18 7 8 45 boys 11 22 8 8 49 increasing elementary school teachers' awareness of gender inequity / luongo 529 table 4 (continue). computer laboratory postintervention observation data category kindergarten grade 1 grade 2 grade 3 total positive responses girls 9 8 5 5 27 boys 5 7 5 4 21 negative responses girls 6 6 3 6 21 boys 10 5 4 9 28 during the third intervention session, the selected sample of teachers claimed to have had difficulty finding pictures of women using technology. the third-grade teacher found one picture of a woman using a cell phone; the second-grade teacher presented two pictures from a women's magazine with a woman using a laptop. the other teachers found no pictures. collectively, the teachers agreed that the majority of technology advertisements and pictures in the media were focused on male consumers. furthermore, this session's activity presented some information on how the teachers viewed girls and boys. the results of the activity can be seen in table 6. during the teacher interviews, the participating teachers were asked to describe the boys and girls in their classes using two adjectives. the descriptions the teachers in the study used to generalize the behaviors of students in their classes can be viewed in table 7. table 5. division of male and female drawings teacher male female both kindergarten 14 3 1 grade 1 5 11 0 grade 2 6 9 0 grade 3 10 30 1 note. total drawings of females were 35 and of males were 30; 1 drawing had both. technology provisions and gender in addition to the questions regarding technology access, there were three questions dealing with technology provisions and gender: 1. did the teachers and students perceive that all students had equal and adequate access to the classroom computers? 2. what were the teachers' attitudes regarding gender and educational ability? 3. did the teachers' attitudes regarding gender correlate with the provisions they made for technology use in their classrooms? from the completion of the questionnaire and the teacher interviews, information was collected about the teachers and the provisions they made for technology use. the student interviews and computer laboratory observations provided supplementary information for analysis. international electronic journal of elementary education, 2012, vol.4, issue 3, 519-544 530 table 6. description of the gender description activity initially, the majority of both the general group of teachers and the selected sample group of teachers claimed that they provided equal or equivalent technology provisions for all students in their classrooms. the general group of teachers believed that gender did not play a part in their behaviors in the classroom. a second-grade teacher explained, "they use the computer when we go to the lab. there are no special provisions made for boys or girls." a first-grade teacher asserted that the children all had equal access to the technology in that room as well as the school. many of the teachers claimed to have a procedure for assigning children to classroom and computer laboratory computers. table 7. teachers' preintervention and postintervention description of boys and girls similarly, the selected sample group of teachers believed that they provided equal or equivalent provisions for boys and girls in their classes. the kindergarten teacher stated, "the children use the computers when we all go into the lab. they also rotate on the computers in age descriptions of males (john) descriptions of females (jane) infancy influenced by mom. plays with balls and trucks. influenced by mom. plays with dolls and bottle. nursery school influenced by dad. plays with blocks. wears blue, green, or brown. influenced by teacher. plays with barbie. wears a skirt elementary school influenced by dad, teacher, and mom. draws, reads, and plays games. wears whatever parents buy. influenced by friends and teacher. reads and draws. wears a skirt. high school influenced by friends. plays sports and with video games. wears sweatpants. influenced by friends. interested in jewelry. wears a skirt. college influenced by mom, dad, friends, and girlfriends. plays with video games. wears sweatpants. influenced by friends (boys and girls), and boyfriends. interested in socializing. wears a skirt. category description of boys description of girls kindergarten preintervention active an loud calm and sensitive postintervention talkative and active caring and kind grade1 preintervention loving and physical loving and centle postintervention loud and lively organized and calm grade 2 preintervention rowdy and loud quiet and worriers postintervention bright and enthusiastic obedient and interested grade 3 preintervention loud and rambunctious quiet and calm postintervention talkative and louder sensitive and quiet increasing elementary school teachers' awareness of gender inequity / luongo 531 the classroom." the selected first-grade teacher agreed, adding that the children in that room followed a certain procedure that permitted them to take turns on the computers in the room. during the preliminary round of interviews and questionnaires, the general group of teachers documented a variety of ways of how they selected children to use the computers in the classroom. one second-grade teacher explained the use of an alphabetical list of the students' names; the teacher checked off each name as each student used the computer. in first grade, two of the teachers asserted that the students used the computer to type their morning writing journals. another first-grade teacher explained that teacher's procedure, "i use a new class list everyday. the student highlights his name when he begins a session of the software. everyone has a turn." several teachers maintained that students who required extra help used the computer to engage in educational games or certain software. likewise, the selected sample group of teachers acknowledged how they selected children to use technology in the classroom. the selected second-grade teacher explained that a rotating schedule had been set up in that classroom. the third-grade teacher maintained that students who finished their work were permitted to use the computer. the kindergarten teacher explained that the children used the computer during center time and free time. some of the teachers in the general group explained that it was difficult for all students to get equal time due to individual circumstances. one teacher claimed that students circulated through that classroom at different times. that teacher explained that this movement hindered the ability to ensure equal time to all students. another teacher did not keep track of students on the classroom computers; they were allowed to use the computers when they were finished with other work. that teacher expressed concern over this inequality, but claimed to not know how to make it more equitable. the teachers in the selected sample group did not express difficulty in providing equal time to all students. during the first round of interviews, none of the teachers in the selected sample believed that their attitudes about gender affected their students' beliefs. the participating secondgrade teacher explained that attempts were made to be as fair in classroom as possible, but no matter what the attempts, the students used the computers at their own paces. this same teacher did not see any reason how the style of teaching the class could affect how the students perceived themselves. on the other hand, the kindergarten teacher was concerned not only with the way that the children were taught, but by the number of boys and girls in each class. that kindergarten teacher contended that when there were more boys than girls in a class, the focus turned to the boys. that teacher also believed that the girls were more likely to find alternate solutions when using the computers rather than asking the teacher. a first-grade teacher claimed that the class technology allotment was fair, but acknowledged that not every student used the computer daily. however, that first-grade teacher believed that it did not any affect how students perceived using technology or themselves. during the computer professional drawing activity in the intervention session, all of the computer professional drawings that the teachers created during the intervention sessions were of women. the writer and the teachers discussed the reasons why they drew women. they explained that they thought of the school technology director who is female as well as other computer professionals that they knew. one of the teachers described a former roommate who was a computer technician. the group discussed the details of the drawings and concluded that in the past they would have probably drawn men. more recently, women have emerged as computer professionals. the writer concluded that the teacher views of gender were changing due to the strong emergence of women in the technology field. international electronic journal of elementary education, 2012, vol.4, issue 3, 519-544 532 the data that the writer collected regarding the students' computer professional drawings (see table 5) was divided according to gender. during the second intervention session, the teachers discussed specifics of what their children drew and the reasons behind the drawings. the findings suggested that the students in these grades did not hold a strong view of a certain gender holding the profession of computer professional. furthermore, there was no established difference between the boys' and girls' drawings. the computer laboratory observations provided information about the teachers' behaviors towards using technology. this setting provided each student with an equal opportunity to use a computer. there were enough computers in the laboratory for each student to have access. the writer used the observation tool in figure to track the number of times each teacher assisted the boys and the girls. in the original observation period, the teachers provided a total of 44 responses to the girls and 110 responses to the boys. in the follow-up period, the teachers provided a total of 93 responses to the girls and 98 responses to the boys. these numbers indicated that there was reduction in gender bias in the classroom. during the intervention sessions and teacher interviews, the selected sample group of teachers discussed the behaviors of both boys and girls. the first-grade teacher explained that the boys are often more verbal about their needs, both in using technology and in other aspects of the classroom. the second-grade teacher claimed that girls will often sit and wait patiently while the teacher assisted the boys first. throughout the intervention sessions, the teachers discussed that they were more aware of these aggressive male behaviors. the follow-up interviews presented similar explanations for how the selected sample group of teachers delineated technology time among students in their classrooms. most of the teachers had created or modified a procedure for students to follow in order to use computers. some of the teachers mentioned that they were attempting to be less genderbiased in their educational technology approaches. the general group of teachers noted some frustration with securing a successful system in providing equal opportunities for all students. one third-grade teacher explained, "i have students coming in and out of my room for a bunch of reasons. when they are not in my room all day, it is hard to fit in time for them to be on the computer." a second-grade teacher claimed, "i have not found a way to guarantee that all students have equal access to the computers." after the intervention sessions were completed, the selected sample group of teachers' views and attitudes regarding gender changed. the general group did not change as dramatically. the selected first-grade teacher explained that there was an increased awareness of how personal attitudes and behaviors affected the children in the class. that first-grade teacher also noted the lack of female images in children's software as well as technology in general. the selected third-grade teacher explained that attempts had made to provide more equitable assistance and support to both the boys and the girls. that teacher also believed that if the school provided more computers, the students would have more access in general. above all, all of the teachers who were involved in the professional development sessions noted a change in attitude toward gender and technology. based on the results of the second round of teacher interviews and questionnaires, the teachers agreed that all students, both male and female, had more equal access to technology than they did before they went through the professional development sessions. the teacher levels of awareness of gender equity were higher than they were before the intervention sessions. some of the participants explained that they modified their selection procedures to be more gender equitable. one first-grade teacher acknowledged that classroom procedures and assistance attempts were meant to be fair, but that teacher could increasing elementary school teachers' awareness of gender inequity / luongo 533 now see how bias had been present in certain ways. the third-grade teacher agreed to use the tips that received during the training when planning for next year. moreover, the teachers discussed how the aggressive nature of the boys as well as other factors can contribute to the unequal distribution of attention to the male students. others found that their procedures were gender equitable and planned on continuing them. in addition to the teacher attitudes, the writer examined the student attitudes towards the teachers who taught them. the writer used the student interview data to support the information that was gathered from the teachers. the writer examined several questions. how did girls and boys view technology? did the students perceive that they all had equal access to the classroom computers? why or why not? to answer these questions, the writer used the student group interview data. the 16 students involved in this study viewed technology in a variety of ways. the preintervention interviews provided a base of what the students knew about technology and its connection to computers. the writer categorized the responses as ones that mentioned the term computer and ones that did not mention the term. during the preintervention interviews, 31% of students mentioned the term computer. when asked about technology, the kindergarteners and first graders were unsure of a definition for it. when asked what technology was, some of the students simply did not know; the others mentioned light. the second graders named some familiar forms of technology, such as computers, lights, cell phones, and other technologies. the third graders also named common forms of technology. the percentage of students who referred to computers increased to 63% during the postintervention interviews. the follow-up interviews showed that the students were more aware of technology and its connection to computers. the majority of the student subjects in all grades acknowledged technology was related to computers. some mentioned cell phones and lights, but the most common response for this section was computer-related. in terms of whether or not they perceived themselves as having equal access to technology in the classroom and computer laboratory, the writer examined their responses to the interview question regarding who had the most access to technology in the classroom. the responses were coded in terms of whether they referred to a specific group or referred to the students being able to access technology in a fair and equitable manner. during the preintervention interview sessions, 31% of the students believed that they received equal access to technology. after the intervention sessions, 50% of the students perceived that they were provided with equal access to technology. although some of the responses discussed other students, many of the replies focused on the teacher and how the teacher used the computer the most in the classroom. implications of findings several implications can be made regarding teacher attitudes and perceptions of gender and technology. in this section, the same topics and questions that were presented in the previous chapter were focused upon: (a) technology access, (b) teacher attitudes, and (c) technology provisions. one finding of the study was that neither the general group nor the selected sample group of teachers involved in the intervention was satisfied with the amount of time given to technology in the classroom or the computer laboratory. both groups of teachers provided various reasons for why they were not satisfied. these reasons included time, space, lack of knowledge, and system requirements. additionally, both the general group and selected international electronic journal of elementary education, 2012, vol.4, issue 3, 519-544 534 sample group of teachers who participated in the intervention showed changes in attitudes and perceptions on general technology usage during the study. although the study did not directly influence the number of computers present or the teacher access to the computer laboratory, both the general group and selected sample group of teachers claimed to be more satisfied with the amount of time they spent using technology at the end of the study. the reasons included (a) general professional development training sessions, (b) a higher number of computers present in the classrooms, and (c) more access to the computer laboratory. the technology coordinator explained that, during the course of the school year, there were several technology improvements that occurred in the school district. these upgrades included the addition of more computers in the classrooms and computer laboratory as well as the purchase of interactive whiteboards. additionally, several technology training sessions for the teaching staff were also held. the writer also found that the selected sample of teachers involved in the intervention believed that all students had more equal access to technology than they did before they had gone through the professional development sessions. mainly, the teachers explained that the intervention sessions made them reconsider their gender biases as well as other biases and reconstruct how they provided student access to the available technology. however, this finding did not isolate gender as the only variable in the increase of student access to technology. the general group and selected sample group of teachers explained that the increase in equity correlated with the fact that the teachers were more confident using technology, thereby, giving them the option to use it more in their teaching methodology. the teachers cited professional development opportunities and more practice opportunities as reasons behind their increased comfort using technology and infusing it into their teaching. the teachers explained that this ability to integrate technology into the curriculum provided more access to all students, not just boys or girls. additionally, the writer found that while many of the students perceived that they did not have equal access to the computers, gender bias was not the main reason behind the student perceptions of their teachers. they claimed the inequality related to areas, such as academic ability, speed in finishing work, and other factors. the student views towards their teacher's attitudes and perceptions did not seem to change after the intervention sessions were implemented. the view was the same for both boys and girls. however, this finding may have been skewed due to the young ages of the children involved in the study. many of these children may not be developmentally prepared to comprehend their teacher's gender biases. the children viewed the access they had to computers and other forms of technology as not being correlated with their teacher's levels of gender bias. the results of the questionnaire distributed at the beginning of the study showed that neither the general teacher group nor the selected sample group involved in the intervention were satisfied with the amount of time and training they received on the topic of technology. the results at the end of the study showed that there was a difference between the general teacher group and the selected sample group involved in the intervention in terms of the changes in attitudes and perceptions of gender and technology. the selected sample group of teachers acknowledged that there were general similarities between the boys and girls in their classes. the first-grade teacher from the selected sample group asserted that both boys and girls were sensitive, inquisitive, anxious, loving, caring, and helpful. they also seemed eager to learn, but lacked patience and control. the secondgrade teacher claimed that boys and girls both "love to come to school. they love to read and be read to. they like to draw pictures." the third-grade teacher claimed that boys and girls were similar in their math ability, but did not comment on their other abilities. increasing elementary school teachers' awareness of gender inequity / luongo 535 conversely, the selected sample group of teachers identified that girls and boys had different qualities. the kindergarten teacher stated that girls tend to be more reserved in their responses while the boys are more outspoken. the first-grade teacher explained that boys tend to be more physical and aggressive. the second-grade teacher thought that the boys were more excited to use the computers than the girls. the third-grade teacher acknowledged that the children may possess equal academic skills, but often show different behaviors when working academically. the selected sample group of teachers expressed some differences in the manner in which boys and girls utilized technology. the first-grade teacher saw that the boys wanted to play games on the computer while the girls enjoyed drawing and writing using a variety of educational software programs. the third-grade teacher observed that the boys tend to use sports games while the girls play games that deal with academics. the third-grade teacher explained that when there are more boys, that teacher tended to help them more frequently with academic tasks. when the selected sample group of teachers was presented, during the second intervention session, with gender equity and technology statistics from earlier studies, they voiced concern about the dates of the study. the teachers discussed how they believed the statistics were outdated because women were then-currently being viewed as more acceptable in the world of technology. the writer discussed the years of the statistics and how things have revolutionized. the teachers agreed with the change, citing the fact that the main computer teacher and the main technology director in the school district were both female. the teachers agreed that females have made much progress in the field of technology. moreover, the teachers predicted that females will become more prominent in the field in the future in the world of technology. as a result of the intervention, the writer found that the selected sample group of teachers recognized to their surprise that they had held certain gender biases that they demonstrated in their classrooms and the computer laboratory. during the study, the attitudes and perceptions of the selected sample group of teachers had changed. initially, these teachers claimed to teach in a gender equitable manner. after the intervention, they recognized and admitted that they did possess some gender bias in regards to technology. with regard to the changes in attitudes and perceptions of gender and technology, there was a distinction between the general teacher group and the selected sample group that was involved in the intervention. the general group did not exhibit any specific changes in attitude regarding gender equity with the study. these teachers seemed to hold the same views of gender and technology before and after the study. their unvarying attitudes could be attributed to the fact that they were not involved in the gender intervention sessions. on the other hand, the selected sample group demonstrated identifiable changes in attitudes and perceptions of gender and technology in the postintervention data collection procedures. the general group's postintervention questionnaire responses were similar to the ones in the preintervention questionnaire. the selected sample group of teachers focused more specifically on gender as a construct in correlation with technology attitudes. in the preintervention observation of the kindergarten class, there were three boys who received most (19) of the 31 responses. these three individuals received teacher assistance and positive responses. the teacher provided responses to 10 of the 11 boys; the teacher provided responses to 3 of the 8 girls. during the postintervention, all of the children except for one girl received responses. there were no students who seemed to receive more attention than others. international electronic journal of elementary education, 2012, vol.4, issue 3, 519-544 536 the preintervention observation of the second-grade class showed that there was 1 boy who received 8 of the 35 responses and 1 girl who received 7 of the 20 responses. these 2 students received teacher assistance and positive responses. the teacher provided responses to all of the boys, but did not respond to 3 of the 10 girls. in the postintervention observation, the teacher responded to all of the students and did not provide more attention to any specific individual. the third-grade preintervention observation produced some strong evidence of teacher change in attitude in terms of gender equity. during the preintervention observation, the teacher only provided two responses in total to the girls. that teacher focused on one boy, in particular, providing six responses. the teacher provided no responses to seven girls and eight boys. after the intervention, the teacher provided responses to all of the students except for one girl. furthermore, the teacher did not provide more attention to any specific individual. the writer concluded that the gender equity intervention was successful with the sample group as it met the objectives of the study. additionally, there was a pattern to the perceptions of the sample group to their own gender biases. the results showed that they were erroneous. the writer concluded that these teachers realized that they did possess some gender biases. the teachers explained that the intervention session activities had contributed to this revelation. during the second intervention sessions, the writer explained that there was a total ratio of 44 responses to girls and 110 responses boys. in response, the teachers offered several reasons behind this discrepancy. the third-grade teacher explained that there were more boys in that room. that teacher also explained the boys in the class were easily distracted, did not pay attention, and were less mature than the girls. moreover, the third-grade teacher claimed that the boys spoke out more and questioned the teacher when they did not understand. the secondgrade teacher added that the boys were often louder and did not follow instructions as well as the girls. similarly, the first-grade teacher claimed that boys were less likely to follow the directions provided by their teacher and, thereby, needed the teacher's assistance while they were working. the selected sample group of teachers discussed what they were presently achieving in their teaching activities that fostered gender equity and what more they could do to support gender equity in their own classrooms and in the computer laboratory. the teachers described procedures, such as setting up a rotating schedule for students to use the computer and using random selection processes of choosing names from a jar. others claimed they do not use pink or blue to depict certain genders. however, one of the teachers admitted to often allowing the girls to be first and created lists of boys and girls for that teacher's personal records. above all, the teachers believed that society plays a huge part in the development of role play. in addition, the teachers expressed interest in expanding their education on both gender equity and technology issues. the teachers expressed interest in additional professional development sessions and learning opportunities as well as ongoing informal discussion between staff members and administrators. however, the writer concluded that there were certain circumstances that could have affected the computer laboratory observation results. the behavior of the boys and girls could have factored into the results of the pretesting and posttesting observations. in both observation periods, the boys seemed more aggressive in their pursuit of help than the girls did. the writer witnessed boys waving their hands while the girls seemed content to figure out their problems on their own. moreover, the teachers involved in the intervention agreed that the boys were more aggressive in many circumstances. increasing elementary school teachers' awareness of gender inequity / luongo 537 the findings suggest that there is a problem in relation to gender equity, education, and technology. particularly, research has shown that the attitudes and behaviors of elementary school teachers regarding technology and gender equity correlate to the success of technology in the classroom. thompson et al. (as cited in tatar & emmanuel, 2001) contended that gender is not considered significant to elementary school teacher training. the implications of this study showed that many of the teachers had not been aware of the problems gender inequality could cause. they had not experienced any gender equity training before the professional development sessions. the male and female participants in the study exhibited certain gender-specific behaviors in the computer laboratory that correlate to the research available on the topic (aauw educational foundation, 2000). the elementary school boys dominate computer use by crowding the girls out. the teachers in the study explained the same phenomenon happening in their classrooms. boys were often more likely to help the teachers with technology in the classroom for a variety of reasons. additionally, the teachers viewed the males as more aggressive and the girls as more passive regarding technology. researchers (gurian & henley, 2001; siann, macleod, glissov & durndell, 1990) agreed that the boys tend to seek out computers whereas the girls often step aside. moreover, this study revealed that gender equity should become systemic and should be built into teacher education programs as well as ongoing teacher professional development. sanders (2002b) contended that this endeavor must be on the agenda of the teacher education profession as well as the college or university that is schooling the preservice teachers. the topic of gender equity should be built into education courses and curricula. finally, the study supported the research that teacher educators need a concise program of instruction as well as materials to establish a reliable means of teaching gender equity. these findings have further implications for the field of education. data from this study implied that teachers who are exposed to gender equity training tend to exhibit more gender equitable behaviors than they did before the training. the data indicated that teachers provide more equitable assistance to their students after being presented with gender equity training. additionally, the writer advocated adopting a proactive stance in regard to gender equity education among faculty and staff members (brusca & canada, 1992). if schools choose to ignore, deny, or view the technological gender gap as a natural state of affairs, the gap will widen. the findings in this study indicated that schools should attempt to create an environment that provides for gender equitable technology opportunities for both the male and female students. simply ignoring the problem will not make it go away. brusca and canada asserted that without intervention, the technological gender gap will only widen. furthermore, the findings showed evidence that when educators assume a proactive stance toward ensuring gender-equitable computer opportunities, their attitudes and teaching behaviors change. school leaders, such as principals and curriculum directors, should be aware of gender issues and matters. these concerns should be an integral part of teacher observations and evaluations. additionally, curriculum developers and directors need to be concerned with gender issues when choosing and developing curriculum. limitations this study had several limitations, delimitations, and constraints. first, the writer was limited in scope due to the sampling procedure (gall et al., 1996). in this study, the writer used a convenience sample of elementary teachers in one school district. this specific sample was international electronic journal of elementary education, 2012, vol.4, issue 3, 519-544 538 located where the writer worked, and was not representative of the entire population. because the study was confined to one small, suburban school district located in northern new jersey, it was delimited to one elementary school in order to focus on the population of kindergarten through third-grade teachers. the data collection method of using a questionnaire presented some limitations (gall et al., 1996). first, some of the general group of teachers did not return the initial questionnaire. some also did not return the concluding questionnaire. moreover, some of the general group did not complete all of the questions in the questionnaire, thereby, threatening the validity of the study. additionally, the writer trusted that the respondents were being honest and candid. often, respondents try to respond with answers that they think the writer wants to hear. additionally, there were situational variables that limited the study outcomes. these conditions included variables, such as lighting, heating, and ventilation (gall et al., 1996). these environmental variables possibly affected how the subjects responded to the interviews and the questionnaires. moreover, the psychological and mental conditions of the subjects influenced the results of data collection. furthermore, the study was delimited in the scope of time required to complete the study. although a longer time period could have facilitated a more accurate longitudinal study, the time range of the study was restricted to the school year. the selected sample of teachers was only available for this duration of time. recommendations based on the findings, conclusions, and implications of this study, the writer has several recommendations for future research and practice. first, additional research should be conducted to better understand the connection between elementary school students, teachers, gender, and technology. research should be conducted using larger and more diverse populations. the writer suggests completing this study in both elementary and secondary school settings to see if the findings are complementary. replication of this study with different samples and populations would confirm results that the intervention strategies lead to teacher awareness of gender issues. moreover, more studies regarding gender and technology should be conducted in different geographic areas. this study was conducted in a suburban school district in the northeastern area of the united states of america. it should be replicated or modified to be conducted in other areas of the country as well as in other nations. it would be interesting to see if gender issues were similar in other areas of the world. do teachers in nations outside the united states of america place as much emphasis on gender? the writer recommends further research regarding gender and technology using elementary school students as subjects. although this study focused on technology in a computer laboratory, a potential study could examine gender, technology, and the classroom. another one could look at gender, technology, and technology at home. it could investigate the connection between gender and home computer use. the writer recommends that studies be conducted asking what teachers think the issues are surrounding gender and education. as sanders (2005) explained, there is a glaring hole in the research on teachers and their point of view. the writer suggests further exploration into the field of gender equity education and preservice teacher training as well as continuing professional development for current teachers. increasing elementary school teachers' awareness of gender inequity / luongo 539 the writer also suggested asking gender equity activists to develop solutions for problems. sanders (2005) asserted that the most "developmental work originates in the activist's belief in their ability to produce programs and materials that teachers will value and that will be effective in increasing female participation in technology" (p. 3). activists may be able to assist educators and other individuals involved in technology education. additionally, curriculum developers should use the research results of this and other genderbased studies to design and develop curriculums that are less gender-biased. particular attention should be placed on technology education, science education, and mathematics education curriculums. lewis (1999) recommended further historical research aimed at telling the story of women in the field of technology. educators and administrators should devote effort and resources into developing less gender-biased instruction. this effort should include additional professional development opportunities for teachers and administrators. these opportunities should include professional development sessions designed for gender equity training. these sessions may include technology training or may simply be directed at common gender issues. the writer also recommends structuring the physical and social environments of computer laboratories and classrooms to enhance gender equitable learning opportunities (brusca & canada, 1992). the physical structure of computing facilities, such as computer laboratories and classrooms, should be gender equitable. brusca and canada explained that many technological areas contain individual and segregated cells and conform to a masculine separation and individuation social style rather than to a feminine social style, which is characterized by personal connections and networks. the writer suggests adjusting computers and other pieces of technology in such a way that allows for more interaction among female students. brusca and canada suggested strategies, such as peer tutoring, team work, and computer networking, to connect people in order to reduce female interpretation of computers as isolating, nonsocial machines. additionally, the writer recommends more computer time for females. because males tend to dominate computer laboratories and computer resources, providing females-only times in computer facilities and females-only computer classes could reduce gender bias in schools (brusca & canada, 1992). software developers should also use the results of gender-based studies to cultivate less gender-biased software programs. inkpen (1997) alleged that many computer games are designed by men for the young male market. moreover, gender biases are found in these games, which often involve violence and use women as objects to be rescued (provenzo, as cited in inkpen). inkpen recommended that additional research be conducted on how to effectively design and use educational multimedia in a learning environment without gender bias. additionally, advertisers should consider gender when promoting technology products. they should aim to have an equal representation of men and women in advertisements. although there are more females in advertisements at the time of this study, they contain images of women in novice or helpless roles. moreover, women are portrayed in a passive manner, suggesting they have limited or no computer ability. above all, the profession should focus more closely on gender equity issues while developing educational resources. this concentration is particularly important for technology teachers and coordinators, many of whom have relatively little formal training in gender education. continued professional development can assist in resolving this inadequacy. international electronic journal of elementary education, 2012, vol.4, issue 3, 519-544 540 . . . references abramson, g. (2006, may). has the gender gap closed?: yes. learning and leading with technology, 33, 6-7. retrieved august 5, 2006, from http://www.iste.org/content/ navigationmenu/publications/ll/llissues/volume_33_2006_2005_/ may_no_8_/33806a.pdf american association of university women educational foundation. (1992). how schools shortchange girls: a study of major findings on girls and education. washington, dc: author and national education association. american association of university women educational foundation. (2000). tech-savvy: educating girls in the new computer age. washington, dc: author. anfara, v. a., brown, k. m., & mangione, t. l. (2002). qualitative analysis on stage: making the research process more public. educational researcher, 31(7), 28-38. best, j. w., & kahn, j. v. (1993). research in education. boston: allyn & bacon. best, r. (1983). we've all got scars: what boys and girls learn in elementary school. bloomington: indiana university. bogo, m., sussman, t., & globerman, j. (2004). the field instructor as a group worker: managing trust and competition in group supervision. journal of social work education, 40(1), 13-26. bravo, m. j. (2003). a psychosocial/educational intervention for decreasing gender stereotypes in technology (doctoral dissertation, university of texas, austin, 2003). dissertation abstracts international, 64 (12), 4352. (umi no. aat 3116265) brown, j., & sheppard, b. (1997). professional development: what do we know and where are we going? retrieved march 8, 2005, from http://www.mun.ca/educ/ faculty/mwatch/win97/pdfinal.htm brusca, f., & canada, k. (1992). the technological gender gap: evidence and recommendations for educators and computer-based instruction designers [electronic version]. educational technology research & development, 39(2), 43-51. campbell, p. b., & sanders, j. s. (1997). uninformed but interested: findings of the national survey on gender equity in preservice education. journal of teacher education, 48(1), 69-75. retrieved jun 5, 2006, from http://jte.sagepub.com/ content/vol48/issue1/ carr-hill, r., & chambers-dixon, p. (2002). a review of methods for monitoring and measuring social inequality, deprivation, and health inequality. retrieved july 27, 2006, from http://www.ihs.ox.ac.uk/sepho/publications/carrhill/ chapman, a. (2002). gender bias in education. retrieved november 17, 2004, from http://www.edchange.org/multicultural/papers/genderbias.html christensen, r., knezek, g., & overall, t. (2005). transition points for the gender gap in computer enjoyment. journal of research on technology in education, 38(1), 23-37. retrieved july 10, 2006, from http://www.iste.org/content/ navigationmenu/ publications/ jrte/ issues/ volume_38/ number_1_fall_2005/ number_1_fall_2005.htm christie, a. a. (1997). using email within a classroom based on feminist pedagogy [electronic version]. journal of research on computing in education, 30(2), 146-176. creswell, j. w. (2003). research design: qualitative, quantitative, and mixed methods approaches (2nd ed.). beverly hills, ca: sage. cuban, l. (2001). teachers' infrequent computer use in classrooms. in j. woodward & l. cuban (eds.), technology, curriculum and professional development: adapting schools to meet the needs of students with disabilities (pp. 121-137). thousand oaks, ca: corwin. increasing elementary school teachers' awareness of gender inequity / luongo 541 delp, d. j. (2002). the effect of professional development training for secondary mathematics teachers concerning nontraditional employment roles for females (doctoral dissertation, university of north texas, 2002). dissertation abstracts international, 63 (09), 3137. (umi no. aat 3065687) dooling, j. o. (1999). a study of gender differences in beliefs toward computer technology and factors which influence these beliefs in grades 4, 5, 6, and 7 (doctoral dissertation, university of hartford, 1999). dissertation abstracts international, 60 (06), 430. (umi no. aat 9936826) ferguson-pabst, d., persichitte, k., lohr, l., & pearman, b. (2003). an analysis of the influence of gender, grade level, and teacher on the selection of mathematics software by intermediate students. information technology in childhood education annual, 1, 5-27. fey, m. h. (n.d.). gender and technology: a question of empowerment. retrieved may 23, 2004, from http://www.geneseo.edu/~balajthy/rwq/rwq12.htm freeman, h., mokros, j., parkes, a., perez, c., & rubin, a. (2000). weaving gender equity into math reform. retrieved march 20, 2005, from http://www.terc.edu/ wge/techsession.html fullan, m. (2002). the change leader. educational leadership, 59(8), 16-20. fullan, m., bertani, a., & quinn, j. (2004). new lessons for district-wide reform. educational leadership, 61(7), 42-46. gall, j. p., borg, w. r., & gall, m. d. (1996). educational research: an introduction (6th ed.). white plains, ny: longman. gipps, c., mccallum, b., & hargreaves, e. (2000). what makes a good primary school teacher: expert classroom strategies. new york: routledgefalmer. glickman, c. d., gordon, s. p., & ross-gordon, j. m. (1995). supervision of instruction: a developmental approach. boston: allyn & bacon. graymill. (1997). graymill, gesa, equity. retrieved march 16, 2005, from http://www.graymill.com/ gunn, c., mcsporran, m., macleod, h. & french, s. (2003). dominant or different?: gender issues in computer supported learning [electronic version]. journal of asynchronous learning networks, 7(1). gurian, m. (1996). the wonder of boys. new york: tarcher/putnam. gurian, m., & henley, p. (2001). boys and girls learn differently. san francisco: jossey-bass. heinich, r., molenda, m., russell, j. d., & smaldino, s. e. (1999). instructional media and technologies for learning. upper saddle river, nj: prentice-hall. hoepfl, m. c. (1997). choosing qualitative research: a primer for technology education researchers. journal of technology education, 9(1), 48-63. hometown locator. (2005). census details. retrieved april 4, 2005, from http://www.hometownlocator.com honey, m., moeller, b., brunner, c., bennett, d., clements, p., & hawkins, j. (1991). girls and design: exploring the questions of technological imagination. in e. rassen & l. iura (eds.), the jossey-bass reader on gender in education (pp. 329-344). san francisco: jossey-bass. inkpen, k. (1997). three important research agendas for educational multimedia: learning, children, and gender. retrieved june 26, 2006, from http://www.edgelab.ca/publications/edmedia97.pdf johns hopkins medical institute. (2000). hopkins research shows nature, not nurture, determines gender. retrieved april 14, 2004, from http://www.hopkinsmedicine.org/press/2000/may/000512.htm kindlon, d., & thompson, m. (1999). thorns among roses: the struggle of young boys in early education. in e. rassen & l. iura (eds.), the jossey-bass reader on gender in education (pp. 153-181). san francisco: jossey-bass. levin, b. b., & barry, s. m. (1997). children's views of technology: the role of age, gender, and school setting. journal of computing in childhood education, 8, 267-290. international electronic journal of elementary education, 2012, vol.4, issue 3, 519-544 542 lewis, t. (1999). research in technology education: some areas of need [electronic version]. journal of technology education, 10(2), 41-56. lindsay, g., & dockrell, j. e. (2004). whose job is it?: parents' concerns about the needs of their children with language problems. journal of special education, 37(4), 225-235. locke, f. l., spirduso, w. w., & silverman, s. j. (1987). proposals that work: a guide for planning dissertations and grant proposals. newbury park, ca: sage. lundeberg, m. a. (1997). you guys are overreacting: teaching prospective teachers about subtle gender bias. journal of teacher education, 48, 55-61. martin, a., & marsh, h. (2005). motivating boys and motivating girls: does teacher gender really make a difference? australian journal of education, 49(3), 320-324. retrieved january 22, 2005, from http://www.accessmylibrary.com/ coms2/summary_0286-11961821_itm matthews, c. e., binkley, w., crisp, a., & gregg, k. (1998). challenging gender bias in fifth grade. educational leadership, 55(4), 54-57. mcgrath, d. (2004). closing the gender gap: girls, technology, fluency, and pbl. learning and leading with technology, 31(6), 28-31. mctaggart, r. (1989). the 16 tenets of participatory action research. retrieved march 8, 2005, from http://www.caledonia.org.uk/par.htm national coalition for equity in education. (n.d.). national coalition for equity in education. santa barbara: university of california, department of education. national organization for women. (1995). concepts and definition of terms used to construct the constitutional equality amendment. retrieved november 28, 2004, from http://www.now.org/issues/economic/cea/concept.html owens, s. l., smothers, b. c., & love, f. e. (2003). are girls victims of gender bias in our nation's schools? journal of instructional psychology, 30, 131-136. perez, c. (2001). weaving gender equity into math reform: equity resource guide. cambridge, ma: education research collaborative/technical education research centers. pollack, w. (1998). real boys: the truth behind the myths. in e. rassen & l. iura (eds.), the jossey-bass reader on gender in education (pp. 88-100). san francisco: jossey-bass. rasmussen, c. (1995). critical issue: ensuring equity and excellence in mathematics. oak brook, il: north central regional educational laboratory. retrieved march 27, 2005, from http://www.ncrel.org/sdrs/areas/issues/content/cntareas/math/ ma100.htm rice, m. (1995). issues surrounding the integration of technology into the k-12 classroom: notes from the field [electronic version]. interpersonal computing and technology: an electronic journal for the 21st century, 3(1), 67-81. rubin, a. (2000). weaving gender equity: technology, math, and equity session. retrieved december 25, 2004, from http://www.terc.edu/wge sadker, d. (1996). where the girls are? retrieved march 15, 2005, from http://www.sadker.org/wherethegirlsare.htm sadker, d. (1999). gender equity: still knocking at the classroom door. educational leadership, 56(7), 22-26. sadker, d. (2001). what is title ix? retrieved march 15, 2005, from http://www.american.edu/sadker/titleix.htm sadker, d., & sadker, m. (1994a). failing at fairness: how america's schools cheat girls. new york: macmillan. increasing elementary school teachers' awareness of gender inequity / luongo 543 sadker, d., & sadker, m. (1994b). the miseducation of boys. in e. rassen & l. iura (eds.), the jossey-bass reader on gender in education (pp. 182-203). san francisco: jossey-bass. sanders, j. (1997). teacher education and gender equity (report no. rr93002015). washington, dc: eric clearinghouse on teaching and teacher education. (eric document reproduction service no. ed408277) sanders, j. (2002a). fairness at the source. retrieved april 1, 2004, from http://www.josanders.com/pdf/fairness_122702.pdf sanders, j. (2002b). something is missing from teacher education: attention to two genders. phi delta kappan, 84, 241-244. sanders, j. (2003a). teaching gender equity in teacher education. education digest, 68(5), 25-29. sanders, j. (2003b). guidelines for a gender equity workshop. retrieved december 22, 2005, from http://www.josanders.com/pdf/ge%20workshop.pdf sanders, j. (2003c). resources. retrieved january 30, 2005, from http://www.josanders.com/resources.html sanders, j. (2005, june). gender and technology in education: what the research tells us. proceedings from the international symposium on women and ict: creating global transformation, baltimore. sanders, j., & campbell, p. b. (2001). using mentors and interdisciplinary teams to genderize teacher education [electronic version]. journal of women and minorities in science and engineering, 7(4), 301313. sanders, j., koch, j., & urso, j. (1997). equity right from the start: instructional activities for teacher educators in mathematics, science, and technology. mahwah, nj: lawrence erlbaum. sanders, j., & tescione, s. m. (2002). gender equity and technology. in j. koch & b. irby (eds.), defining and redefining gender equity in education (pp. 99-115). greenwich, ct: information age. schoolkit international. (2004, july). viewing professional development as change management. retrieved february 27, 2005, from http://www.schoolkit.com/ newsletters/nl_us_04_07/nl_us_04_07.htm shapiro, j., kramer, s., & hunerberg, c. (1981). equal their chances: children's activities for non-sexist learning. englewood cliffs, nj: prentice hall. siann, g., macleod, h., glissov, p., & durndell, a. (1990). the effect of computer use on gender differences in attitude to computers. computers and education, 14(2), 184-191. sprinthall, n. a., sprinthall, r. c., & oja, s. n. (1994). educational psychology: a developmental approach (6th ed.). new york: mcgraw-hill. starr, l. (2000). is technology just for boys? retrieved march 23, 2004, from http://www.educationworld.com/a_issues/chat/chat017.shtml stephenson, c. (2006). has the gender gap closed?: no. learning and leading with technology, 33(8), 67. tatar, m., & emmanuel, g. (2001). teachers' perceptions of their students' gender roles [electronic version]. journal of educational research, 94, 215-224. technical education research centers. (2000). weaving gender equity: workshops. retrieved march 16, 2005, from http://www.terc.edu/wge/rgworkshops.html thompson, r., stephenson-hawk, d., & clark, b. (1999). integrating gender equity and reform: toolkit of curriculum materials. retrieved march 8, 2005, from http:// www.coe.uga.edu/ingear/ thorne, b. (1993). do boys and girls have different cultures? in e. rassen & l. iura (eds.), the jossey-bass reader on gender in education (pp. 125-150). san francisco: jossey-bass. title ix of the education amendments of 1972, 20 u.s.c. sec. 1681. (1972). international electronic journal of elementary education, 2012, vol.4, issue 3, 519-544 544 u.s. census 2000. (2000). census bureau homepage. retrieved october 17, 2004, from http://www.census.gov/ u.s. department of education. (2000). gender equity expert panel: exemplary and promising gender equity programs. retrieved march 15, 2005, from http://www .ed.gov/pubs/genderequity/gender_equity.doc usselman, m., & whiting, d. (2002, june). gender equity professional development for teachers in a summer camp setting. american society for engineering education annual conference & exhibition, georgia institute of technology, atlanta, ga. willis, j. (2001). foundational assumptions for information technology and teacher education. contemporary issues in technology and teacher education, 1(3), 305-320. retrieved march 12, 2005, from http://www.citejournal.org/vol1/iss3/ editorials/article1.htm worthen, b. r., sanders, j. r., & fitzpatrick, j. l. (1997). program evaluation: alternative approaches and practical guidelines (3rd ed.). boston: allyn & bacon. international electronic journal of elementary education, 2016, 8(3) issn:1307-9298 copyright © iejee www.iejee.com dear iejee reader, international electronic journal of elementary education is bringing you a comprehensive issue which covers many important topics in the field of elementary education. dr. berat ahi addresses and discusses the mental models in preschool children’s conceptualization. researchers from finland highlight how supervision supports inclusive teaching practices of the professional teachers. three other researchers study and capture the primary school teachers’ experiences with and opinions on constructive classroom managements. dr. theresa a. coogan decribes and discusses the role of supportive school counselling for the establishment of a middle school career development program in belize. two other researchers investigate the locus of control and personality traits of preschool teachers related their level of self-directed learning readiness. the next paper investigates another important challenge in our time: the sensitivity of students to the natural environment, animals, social problems and cultural heritage. another paper which addresses environmental consciousness is written by dr. laurie james of western washington university, united states. dr. laurie’s research is very concreate, practical oriented and demonstrated how the schools can make substantial gains with regard to minimizing waste in a school setting by reducing, reusing, recycling, and composting waste products by small but conscious strategies. in this issue of the iejee the readers will also find two which address reading related topics. one has turkish and the other has arabic as its point of departure. in the international literature english-based perspectives on reading related issues have a dominating position. the field has a need of reading research which also takes the characteristics of different languages like turkish and arabic. dr. birkan güldenoğlu of ankara university, turkey, investigates the effects of syllableawareness skills on the word-reading performances of students reading in a transparent orthography as the turkish has. on the other hand dr. abdullah m. seraye of king saud university, saudi arabia addresses the impact of different approaches to reading comprehension as the children have arabic as their language. as an editor-in-chief, i consider the mentioned two reading related papers as important contribution which can enrich and expand our understanding of reading in different language environments. dr. ebru melek koç addresses several aspects of in-service training of english language teachers at elementary schools. her reseach reminds us once more the importance of teachers’ learning for students’ learning and school development. dr. angela maria la porte of university of arkansas, ar, united states investigates the efficacy of the arts in a transdisciplinary learning experience for culturally diverse fourth graders. her findings are convincing and inspiring us educators. with her words: http://www.iejee.com/ vi “a cooperative school environment paved the way for student self-confidence and motivation for learning through (1) opportunities for student choice and decision-making and (2) collaborative, inquiry-based, transdisciplinary, project-based learning. regardless of demographics, transdiciplinary learning through the arts challenged and motivated students to think and make decisions in collaboration with others, using and valuing the expertise of peers. regardless of student ethnicity or socioeconomic status, learners felt empowered and enthusiastic about attending school and gained knowledge through inquiry and project-based opportunities. this progressive ideology and practice has the potential to benefit diverse learners in 21st century education.” researchers basal, celen, kaya and boğaz of yildiz technical university, turkey, investigate he illustrations in english course books in a turkish context. the illustrations in the textbooks are the iconic representations of the realities in our lives. what kind of functions do they accomplish and what kind of roles do they have in the teaching-learning processes in the schools is an important area of research, but as much as we know it did not get enough attention from the researchers that it deserves. therefore we appreciate that the four researchers submitted their paper to iejee. it’s an informative paper because they “… investigate the frequency of occurrence and functions of illustrations in english course books. four course books (5th, 6th 7th, and 8th grades) approved by the ministry of national education in turkey were analyzed. in this qualitative research, content analysis was used as the research technique. in the analysis, levin's 1981 typology including five categories in terms of the functions of illustrations (decorational, representational, organizational, interpretational, and transformational) was employed. based on the results, it was found that most of the illustrations in four books were subsumed under the category of representational function. the results also showed that the books were mostly lacking transformational, interpretational, and organizational illustrations. this is considered a serious drawback of the books in terms of content-related illustrations since they make abstract concepts more memorable, enhance the recall of details of texts, and facilitate students’ comprehension. the study also provides recommendations for the use of illustrations in english language course books.” as one of the peer reviewers said “this paper demonstrate the power of iconography” in teaching-learning processes. i want to express my deep gratitude to ulaş yabanova, doctoral candidate mustafa bakir, doctoral candidate hasan tabak , dr. hayriye gül kuruyer, and dr. gökhan özsoy for their valuable coordination of the review process and for dr. turan temur for his tireless efforts for ensuring the academic quality of the publishing process. editor-in-chief kamil özerk, professor of education, university of oslo © 2022 published by kura education & publishing. this is an open access article under the cc byncnd license. (https://creativecommons. org/licenses/by/4.0/) 137 copyright © www.iejee.com issn: 1307-9298 international electronic journal of elementary education december 2022, volume 15, issue 2, 137-144 validation of a scale to assess activities of daily living at home in children and adolescents with autism spectrum disorder marco cossio-bolañosa, rubén vidal-espinozab, fernando alvear-vasquezc, christian de la torre choqued, nicolás vidal-fernandeze, jose sulla-torresf, rodrigo monne de la peñag, rossana gómez-camposh,* abstract introduction daily living skills may affect repetitive behavior patterns in individuals with autism spectrum disorder (asd). this study aimed to validate a scale to assess activities of daily living at home (saadl) in children and adolescents with asd. content validity by expert judgment showed aiken v values between 0.75 and 0.88. the exploratory factor analysis (efa) showed factor loadings (0.62 to 0.89) and high communalities. the % explanation of variance for dimension 1 (personal care) was 51.5% and for dimension 2 (mobility in the home) was 66.3%. the cronbach's reliability was 0.87 and the retest values reflected high values of precision and accuracy. the saadl is valid and reliable and can be used to assess activities of daily living of self-care and mobility at home. activities of daily living (adls) are essential and routine tasks that most young, healthy people can perform without assistance (edemekong et al., 2020). they are characterized by performing fundamental tasks to support participation in school, home, and community settings in diverse populations (james et al. 2014). adls according to the literature are classified as basic and instrumental activities (edemekong et al. 2020; law et al. 2005); "occupational therapy practice framework: domain and process (3rd edition)" 2017). the former refers to activityoriented personal tasks such as: walking independently, ability to feed oneself, selecting and putting on appropriate clothing, ability to bathe, groom oneself, maintain personal hygiene, control bladder function, and ability to go to and from the toilet. meanwhile, instrumental activities have to do with those that require more complex thinking skills. these include organizational activities such as transportation and shopping, managing finances, shopping, food preparation, managing communication, medication management. research currently suggests that many people with autism spectrum disorder (asd) have impairments in daily living keywords: activities of daily living, autistic, validity, reliability, scale. received : 31 october 2022 revised : 24 december 2022 accepted : 26 december 2022 doi : 10.26822/iejee.2023.285 a marco cossio-bolaños, department of physical activity sciences, universidad católica del maule, talca, chile. faculty of education, psychology and sports science,. universidad de huelva. huelva, spain. e-mail: mcossio1972@hotmail.com orcid: https://orcid.org/0000-0001-7230-9996 b rubén vidal-espinoza, universidad católica silva henriquez, santiago, chile. e-mail: rvidal@gmail.com orcid: https://orcid.org/0000-0002-8593-5248 c fernando alvear-vasquez, universidad autónoma de chile, chile e-mail: falvearvasquez@gmail.com orcid: https://orcid.org/0000-0002-9461-1384 d christian de la torre choque, universidad san ignacio de loyola, lima, perú. e-mail: cdelatorre@hotmail.com orcid: https://orcid.org/0000-0002-1560-1445 e nicolás vidal fernández, department of physical activity sciences, universidad católica del maule, talca, chile. e-mail: nfernandez@hotmail.com orcid: https://orcid.org/0000-0002-1447-0949 f jose sulla-torres, universidad católica de santa maría, arequipa, perú. e-mail: josullato@gmail.com orcid: https://orcid.org/0000-0001-5129-430x g rodrigo monne de la peña, department of educational diversity and inclusiveness, universidad católica del maule, talca, chile. e-mail: rmonne@ucm.cl orcid: https://orcid.org/0000-0002-0142-7685 h,* corresponding author: rossana gómez-campos, department of educational diversity and inclusiveness, universidad católica del maule, talca, chile e-mail: rossaunicamp@gmail.com orcid: https://orcid.org/0000-0001-6509-5707 138 december 2022, volume 15, issue 2, 137-144 skills relative to their cognitive skills (bal et al. 2015; farley et al. 2009; marsack-topolewski et al. 2021). therefore the assessment of functional impairment, including different areas of a person's life, such as education, family, social life, work life, leisure and free time are crucial, both for the diagnosis, as well as for the therapeutic approach of young people with asd (rotger et al. 2014). in general, levels of functioning, measures of adaptive behavior, and in particular daily living skills, are the variables that most affect repetitive patterns of behavior in people with asd (farley et al. 2009; vahia, 2013). in fact, most people with asd remain at home with their families until they enter adulthood, so parents provide continuous support to their children to overcome daily living difficulties (adls) (dudley et al. 2019; marsack-topolewski et al. 2021). in addition, in recent months, following the covid 19 pandemic, children, adolescents and adults with asd have been locked in their homes for more than a year, so adls at home could undergo substantial changes. therefore, to assess these daily activities at home it is necessary to have a simple, brief and easy to apply instrument. therefore, this study aimed to validate a scale to assess activities of daily living (adls) within the home of children and adolescents with asd. methodology type and sample a cross-sectional study was designed in chilean children and adolescents with asd. the sample consisted of 32 boys and 5 girls from public educational institutions in the cities of talca, rancagua and santiago (chile). the age range ranged from 4 to 20 years old. the sample selection was non-probabilistic by convenience. all the information about the children and adolescents with asd was collected through their parents, since they were the ones who answered the survey. the average age of the parents was 36.85±7.61 years. the sociodemographic variables of both the young people with asd and their parents are shown in table 1. procedures parents were contacted via telephone in may 2021. once contacted, the objective of the research was explained to them. parents who agreed to participate in the study signed the informed consent. this stage lasted two weeks. children enrolled in state special education schools (preschool, primary and secondary) and those living directly with one or both parents were included. children whose parents did not complete the applied scale and those living with relatives (other than their parents, one or both parents) were excluded. the entire procedure was performed in accordance with the local ethics committee and the helsinki declaration for human beings. validity and reliability of the scale the saadl was validated by expert judgment (content validity) and construct analysis, while reliability was assessed by measures of internal consistency and stability (retest). content validity was assessed by expert judgment, according to the suggestions described by wiersma (2001). six health and education professionals with a minimum of 10 years of professional and research experience in their area were invited to participate as experts. this panel of professionals with extensive experience in asd reviewed appropriate scale items as described by schultz (2005) the saadl was sent to each of the experts by e-mail. they then evaluated the degree of representativeness, relevance, diversity, clarity, simplicity and completeness of each of the items of the elaborated instrument in an index card. the alternatives presented a scoring scale from 1 to 5 points. each expert evaluated the saadl and resubmitted the form for further analysis. in the end, the saadl consisted of 8 questions and can be seen in appendix 1. construct validation was carried out by measuring exploratory factor analysis (efa), allowing the underlying structure of the data to be determined (bollen, 1989). reliability was assessed by internal consistency, calculating cronbach's alpha per question and total scale. on the other hand, stability measures (retest) were also used with a 14-day time interval between both measurements. for the second measurement, nine parents were used, representing 24% of the total sample. statistics the normality of the data was verified by the shapirowilk test. descriptive statistics were analyzed for frequencies, percentages, range, mean (x), standard deviation (sd), skewness, and kurtosis. for content validity, aiken's v (bulger & housner, 2007) was used which evaluates the adequacy of the items to the content validity criteria. higher values of aiken's v ≥ 0.75 were accepted (bulger & housner, 2007). for construct validity, the exploratory factor analysis (efa) fit model was considered by adopting the kaiser meyer-olkin (kmo) sample adequacy criteria, 139 validation of a scale to assess activities of daily living at home / cossio-bolanos, vidal-espinoza, ... & gómez-campos bartlett's sphericity value was considered to establish the relevance of the factor analysis, comparative fit index (cfi) root mean square error of approximation (rmsea). the analysis provided the measure of variance explained, factor loadings, communalities and chi-square approximation. to assess stability measures, the concordance correlation coefficient (ccc) was calculated, using precision (p) and accuracy (a) according to lawrence and lin (1989) approach. in addition, the weighted kappa (cohen, 1968) was calculated to measure the magnitude of agreement between the two scores (test and retest). in all cases, p < 0.05 was adopted. the results were processed and analyzed initially in excel spreadsheets and subsequently in spss 18.0 and med calc 11.1.0, as appropriate. results the sociodemographic variables of the parents and children are shown in table 1. thirty-seven parents were surveyed, corresponding to 51.4% cohabiting families, 40.5% married and 8.1% divorced. the majority of respondents were female (81.1%) and only 18.9% were male. in addition, most of these families lived in urban areas (86.5%) compared to those in rural areas (13.5%). regarding information on children with asd, the majority were males (86.5%) relative to females (13.5%). of the children, 67.7% lived with both parents (67.7%) and 32.4% with their mothers, while 18.9% were only children and 81.1% had one or more siblings. table 1. sociodemographic characteristics of the sample studied. variables fi % parental information gender males 7 18.9 female 30 81.1 marital status married 15 40.5 divorced 3 8.1 cohabitant 19 51.4 area where you live urban 32 86.5 rural 5 13.5 children information sex male 32 86.5 female 5 13.5 lives with parents both parents 25 67.6 mother only 12 32.4 father only 0 0 number of brothers none 7 18.9 1 brother 15 40.5 2 brothers 11 29.7 3 brothers 3 8.1 4 brothers 1 2.7 values for the v of aiken test are presented in table 2. the values for each question varied from 0.75 to 0.92 while the values for the dimensions were between 0.82 and 0.88. for all of the cases, the values obtained from the judges reflected an agreement of 0.75 to 0.88. table 2. content validity of the instrument (saadl) using aiken's v by question and dimension. nº dimensions/ items rp re di cl si co personal care 1 item 1 0.92 0.83 0.83 0.92 0.88 0.88 2 item 2 0.88 0.83 0.75 0.79 0.88 0.83 3 item 3 0.83 0.88 0.96 0.92 0.75 0.92 4 item 4 0.79 0.83 0.79 0.88 0.88 0.88 total 0.85 0.84 0.83 0.88 0.84 0.88 mobility in the house 5 item 5 0.92 0.83 0.96 0.88 0.83 0.83 6 item 6 0.88 0.83 0.83 0.83 0.83 0.83 7 item 7 0.83 0.88 0.79 0.92 0.92 0.92 8 item 8 0.83 0.79 0.71 0.83 0.88 0.88 total 0.86 0.83 0.82 0.86 0.86 0.86 legend: rp: representativeness, re: relevance, di: diversity, cl: clarity, si: simplicity, co: comprehensibility, t: total. the descriptive values of the saadl are shown in table 3. skewness showed values lower than the average (-0.97 to 1.39) while kurtosis ranged from (-1.88 to 2.68). the coefficient of variation in all cases was less than 33% and the values of cronbach's alpha per question ranged from 0.81 to 0.90 and in the total scale it was r = 0.87. table 4 shows the factor loadings and communalities obtained after the efa. the rotated components matrix was obtained through varimax. the model fit gave the following results: cfi (0.903), rmsea (0.057), kmo was 0.80, x2= 165.1 (p = 0.000). factor loadings ranged from 0.62 to 0.89, while communalities were higher than 0.53. overall, the % explanation of variance of dimension 1 (personal care) reflected 51.5% and dimension 2 (mobility in the house) 66.3%. the reliability values analyzed by means of retesting can be seen in table 5. the dri was calculated, obtaining ccc values from 0.87 to 1.0, the values of precision (0.96) and accuracy (0.99) were very high, and even the weighted kappa showed high concordance values (0.74 to 1.0), which guarantee equivalence between both measurements. discussion the results of the study have shown that the scale proposed in this study proved to be valid and reliable for assessing adls in the homes of children and adolescents with asd. 140 december 2022, volume 15, issue 2, 137-144 table 3. descriptive analysis of saadl in children and adolescents with asd. no questions mean sd cv asymmetry kurtosis cronbach 1 in personal care for grooming (bathing, washing, combing hair, etc.) 2.7 0.9 33.3 0.65 -1.51 0.813 2 in personal care for dressing, undressing (change of clothes) 2.89 0.9 32.4 0.23 -1.88 0.9 3 in household chores to organize (clothes, utensils, food, other) 2.3 0.7 28.8 2.03 2.68 0.817 4 in household chores to clean the house (sweeping, mopping, washing clothes, other) 2.43 0.7 29.9 1.39 0.44 0.824 5 in mobility inside the house to move around 3.49 0.7 21 1.07 -0.22 0.812 6 in mobility inside the house to get in and out of the bathtub. 3.14 0.9 29.3 -0.28 -1.8 0.816 7 in mobility within the home for getting up and going to bed 3.43 0.8 23.3 -0.97 -0.7 0.809 8 in mobility inside the house when moving around, going up and down stairs, others. 3.49 0.7 21 -1.08 -0.22 0.863 legend: sd: standard deviation, cv: coefficient of variation table 4. factor loadings and communalities based on exploratory factor analysis. n° questions factor loadings communalities 1 2 1 in personal care for grooming (bathing, washing, combing hair, other) 0.732 0.419 0.71 2 in personal care for dressing, undressing (changing clothes) 0.625 0.546 0.68 3 in household chores to organize (clothes, utensils, food, other) 0.897 0.117 0.82 4 in household chores to clean the house (sweeping, mopping, washing clothes, other) 0.858 0.18 0.77 5 in mobility inside the house to move around -0.073 0.802 0.65 6 in mobility inside the house to get in and out of the bathtub. 0.409 0.66 0.6 7 in mobility within the home for getting up and going to bed 0.29 0.666 0.53 8 in mobility inside the house when moving around, going up and down stairs, others. 0.273 0.771 0.67 legend: 1: personal care, 2: mobility in the house. table 5. values that define the concordance between the values of the test and retest of the saadl. n° questions ccc precision accuracy weighted standard kappa error 1 in personal care for grooming (bathing, washing, combing hair, other) 0.93 0.939 0.99 0.84 0.1 2 in personal care for dressing, undressing (changing clothes) 0.951 0.959 0.993 0.9 0.1 3 in household chores to organize (clothes, utensils, food, other) 1 1 1 1 0 4 in household chores to clean the house (sweeping, mopping, washing clothes, other) 0.878 0.91 0.97 0.74 0.13 5 in mobility inside the house to move around 0.878 0.906 0.97 0 0 6 in mobility inside the house to get in and out of the bathtub. 1 1 1 1 0 7 in mobility within the home for getting up and going to bed 0.905 0.932 0.971 0.85 0.13 8 in mobility inside the house when moving around, going up and down stairs, others. 0.878 0.906 0.97 0.74 0.13 scale total 0.95 0.959 0.99 0.74 0.13 legend: ccc: concordance correlation coefficient 141 validation of a scale to assess activities of daily living at home / cossio-bolanos, vidal-espinoza, ... & gómez-campos two validation techniques (content and construct) were used in this study. first, it was validated by expert judgment, where the results reported by the judges highlight broad homogeneity among the experts' responses. this consensus guarantees the criteria of relevance, diversity, clarity, simplicity and comprehensiveness of the scale as suggested by wiersma (2001). the questions or items of the scale are adequate and exhaustive, allowing to capture the true attributes that measure adls in children and adolescents with asd. in fact, the v aiken values have reflected values above 0.80, which are consistent with other studies (bolivarparedes & villanueva-ruska, 2017; gomez-campos et al. 2021; lagos-luciano et al. 2019). moreover, they exceed the values of 0.70 described in the literature as adequate values (charter, 2003) and even, the interjudge reliability standard is higher than 0.75, which are considered excellent (cicchetti, 1994). secondly, the validation by afe revealed two dimensions, personal care and mobility at home. in addition, the model was stable and met the criteria for goodness-of-fit indices in cfi, rmsea, kmo, % variance explanation as described in the literature (dini et al., 2014; hu & bentler, 1999; schermelleh-engel, moosbrugger, & müller, 2003). on the other hand, the factor loadings obtained in this study were higher than 0.62, while the communalities >0.53, reflecting acceptable values between item and dimension (fabrigar et al. 1999; knekta et al. 2019; maccallum et al. 2001). these values are similar to those reported in recent studies in youth with asd (cassidy et al. 2021; zhou et al. 2017). regarding reliability, data were analyzed by internal consistency and retest. for the first case, cronbach's alpha was used evidencing high reliability values r = 0.87, being similar to other studies conducted in asd populations (breidbord & croudace, 2013; brugha et al. 2020; skuse et al. 2005). in general, the scale proposed here reflects internal consistency among its items according in line with what is suggested in the literature (nunnally, 1994; streiner, 2003), highlighting a minimum of 0.80. in the second case, the retest was used as a measure of stability. the scale was applied to the parents on two occasions with an interval of 14 days, in which it is proposed to consider between 10 and 14 days (terwee et al. 2007). this time interval has allowed reporting concordance between both measures and high levels of precision and accuracy. this shows that the scale presents stability in the scores of both tests, so they remain without substantial changes when measured on different occasions (michalos, 2014). in addition, the values obtained in this study are consistent with research that has evaluated reliability by retesting (berthoz & hill, 2005; dutil et al. 2017; pereira et al. 2008). re-testing (test and re-test) as quality control criteria for scales in general are crucial, especially if the scales are intended to be applied in treatment interventions. this is because instruments are required to be stable in their results, especially in scales that have to do with adls (dutil et al. 2017). in general, this study presented some limitations that deserve to be clarified. a relatively small sample was used, and the scale was applied in covid-19 pandemic time, through google drive. probably, these factors could have affect the results obtained in the study. however, despite this, the validity and reliability techniques have shown consistent results. it is even one of the first studies that seeks to propose a new scale to be used in the asd populations that can serve as a baseline for comparing post-pandemic adls. it is suggested that future studies expand the study sample, apply other validity and reliability techniques to achieve external validity of the scale. conclusion this study concludes that the saadl for children and adolescents with asd is valid and reliable. these psychometric properties warrant its use and regular application to assess daily activities of self-care and mobility in the home in youth with asd. for the ability to perform adls at home can serve to improve safety conditions, greater participation in the home, alleviate family or caregiver overload, and improve their quality of life. disclosure statement the authors reported no potential conflict of interest. references bal, v.h., kim, s.h., cheong, d., & lord, c. (2015). daily living skills in individuals with autism spectrum disorder from 2 to 21 years of age. autism, 19(7), 774-784. https://doi.org/10.1177/1362361315575840 berthoz, s., & hill, e.l. (2005). the validity of using selfreports to assess emotion regulation abilities in adults with autism spectrum disorder. european psychiatry, 20(3), 291-298. https://doi. org/10.1016/j.eurpsy.2004.06.013 bolivar-paredes, e., & villanueva-ruska, a. (2017). validation and reliability of the aq-27 questionnaire of stigmatizing attitudes towards patients with schizophrenia in a general hospital 2015. revista de neuro-psiquiatría, 80(3), 165-171. http://dx.doi.org/10.20453/rnp. v80i3.3153 bollen, k.a. (1989). structural equations with latent variables (vol. 210): john wiley & sons. 142 december 2022, volume 15, issue 2, 137-144 breidbord, j., & croudace, t.j. (2013). reliability generalization for childhood autism rating scale. journal of autism and developmental disorders, 43(12), 2855-2865. https://doi.org/10.1007/s10803013-1832-9 brugha, t., tyrer, f., leaver, a., lewis, s., seaton, s., morgan, z., . . . van rensburg, k. (2020). testing adults by questionnaire for social and communication disorders, including autism spectrum disorders, in an adult mental health service population. international journal of methods in psychiatric research, 29(1), e1814. https://doi.org/10.1002/mpr.1814 bulger, s.m., & housner, l.d. (2007). modified delphi investigation of exercise science in physical education teacher education. journal of teaching in physical education, 26(1), 57-80. https://doi.org/10.1080/02701367.2014.930087 cassidy, s.a., bradley, l., cogger-ward, h., & rodgers, j. (2021). development and validation of the suicidal behaviours questionnaire-autism spectrum conditions in a community sample of autistic, possibly autistic and non-autistic adults. molecular autism, 12(1), 1-22. https://doi. org/10.1186/s13229-021-00449-3 cicchetti, d.v. (1994). guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. psychological assessment, 6(4), 284. https://doi. org/10.1037/1040-3590.6.4.284 cohen, j. (1968). weighted kappa: nominal scale agreement provision for scaled disagreement or partial credit. psychological bulletin, 70(4), 213. https://doi.org/10.1037/h0026256 charter, r.a. (2003). a breakdown of reliability coefficients by test type and reliability method, and the clinical implications of low reliability. the journal of general psychology, 130(3), 290304. https://doi.org/10.1080/00221300309601160 dini, a.p., alves, d.f. d. s., oliveira, h.c., & guirardello, e. d. b. (2014). validity and reliability of a pediatric patient classification instrument. revista latinoamericana de enfermagem, 22, 598-603. https:// doi.org/10.1590/0104-1169.3575.2457 dudley, k.m., klinger, m.r., meyer, a., powell, p., & klinger, l.g. (2019). understanding service usage and needs for adults with asd: the importance of living situation. journal of autism and developmental disorders, 49(2), 556-568. https:// doi.org/10.1007/s10803-018-3729-0 dutil, é., bottari, c., & auger, c. (2017). test-retest reliability of a measure of independence in everyday activities: the adl profile. occupational therapy international, 2017. https://doi.org/10.1155/2017/3014579 edemekong, p.f., bomgaars, d.l., sukumaran, s., & levy, s.b. (2020). activities of daily living: statpearls publishing, treasure island (fl).avalaible from: https://www.ncbi.nlm.nih.gov/books/ nbk470404/ fabrigar, l.r., wegener, d.t., maccallum, r.c., & strahan, e.j. (1999). evaluating the use of exploratory factor analysis in psychological research. psychological methods, 4(3), 272. https://doi.org/10.1037/1082-989x.4.3.272 farley, m.a., mcmahon, w.m., fombonne, e., jenson, w.r., miller, j., gardner, m., . . . ritvo, r.a. (2009). twenty-year outcome for individuals with autism and average or near-average cognitive abilities. autism research, 2(2), 109-118. https:// doi.org/10.1002/aur.69 gomez-campos, r., vidal-espinoza, r., castelli correia de campos, l.f., lee-andruske, c., sulla-torres, j., cornejo-valderrama, c., . . . urra-albornoz, c. (2021). validation of a self-perceived adaptive behaviors scale in older chilean women and percentiles for evaluation. international journal of environmental research and public health, 18(2), 731. https://doi.org/10.3390/ijerph18020731 hu, l.t., & bentler, p.m. (1999). cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. structural equation modeling: a multidisciplinary journal, 6(1), 1-55. https://doi. org/10.1080/10705519909540118 james, s., ziviani, j., & boyd, r. (2014). a systematic review of activities of daily living measures for children and adolescents with cerebral palsy. developmental medicine & child neurology, 56(3), 233-244. https://doi.org/10.1111/dmcn.12226 knekta, e., runyon, c., & eddy, s. (2019). one size doesn’t fit all: using factor analysis to gather validity evidence when using surveys in your research. cbe—life sciences education, 18(1), rm1. https:// doi.org/10.1187/cbe.18-04-0064 lagos-luciano, j., andruske, c.l., cossio-bolaños, m.a., sulla-torres, j., coyla-idme, l., luarterocha, c., & gómez-campos, r. (2019). proposed standards for assessing language skills of adolescent students: language skills of adolescent students. international electronic journal of elementary education, 12(2), 223-232. 143 validation of a scale to assess activities of daily living at home / cossio-bolanos, vidal-espinoza, ... & gómez-campos law, m., dunn, w., & baum, c. (2005). measuring participation. measuring occupational performance: supporting best practice in occupational therapy, 107-126. lawrence, i., & lin, k. (1989). a concordance correlation coefficient to evaluate reproducibility. biometrics, 255-268. https://doi.org/ 10.2307/2532051 maccallum, r.c., widaman, k.f., preacher, k.j., & hong, s. (2001). sample size in factor analysis: the role of model error. multivariate behavioral research, 36(4), 611-637. https:/doi.org/10.1207/ s15327906mbr3604_06 marsack-topolewski, c.n., samuel, p.s., & tarraf, w. (2021). empirical evaluation of the association between daily living skills of adults with autism and parental caregiver burden. plos one, 16(1), e0244844. https://doi.org/10.1371/journal. pone.0244844 michalos, a. c. (2014). encyclopedia of quality of life and well-being research: springer netherlands dordrecht.https://doi.org/10.1007/978-94-0070753-5_3001 nunnally, j. c. (1994). psychometric theory 3e: tata mcgraw-hill education. occupational therapy practice framework: domain and process (3rd edition). (2017). american journal of occupational therapy, 68(supplement_1), s1s48. doi:10.5014/ajot.2014.682006 pereira, a., riesgo, r.s., & wagner, m.b. (2008). childhood autism: translation and validation of the childhood autism rating scale for use in brazil. jornal de pediatria, 84, 487-494. https:// doi.org/10.2223/jped.1828 rotger, s., richarte, v., nogueira, m., corrales, m., bosch, r., vidal, r., . . . goikolea, j. m. (2014). functioning assessment short test (fast): validity and reliability in adults with attention-deficit/ hyperactivity disorder. european archives of psychiatry and clinical neuroscience, 264(8), 719-727. https://doi.org/10.1007/s00406-0140501-0 schermelleh-engel, k., moosbrugger, h., & müller, h. (2003). evaluating the fit of structural equation models: tests of significance and descriptive goodness-of-fit measures. methods of psychological research online, 8(2), 23-74. schultz, k., whitney, d.j. (2005). measurement theory in action: thousand oaks, ca: sage. skuse, d. h., mandy, w. p., & scourfield, j. (2005). measuring autistic traits: heritability, reliability and validity of the social and communication disorders checklist. the british journal of psychiatry, 187(6), 568-572. https://doi. org/10.1192/bjp.187.6.568 streiner, d.l. (2003). starting at the beginning: an introduction to coefficient alpha and internal consistency. journal of personality assessment, 80(1), 99-103. https://doi.org/10.1207/ s15327752jpa8001_18 terwee, c.b., bot, s.d., de boer, m.r., van der windt, d.a., knol, d.l., dekker, j., . . . de vet, h.c. (2007). quality criteria were proposed for measurement properties of health status questionnaires. journal of clinical epidemiology, 60(1), 34-42. https://doi.org/10.1016/j.jclinepi.2006.03.012 vahia, v.n. (2013). diagnostic and statistical manual of mental disorders 5: a quick glance. indian journal of psychiatry, 55(3), 220. https://doi. org/10.4103/0019-5545.117131 wiersma, l.d. (2001). conceptualization and development of the sources of enjoyment in youth sport questionnaire. measurement in physical education and exercise science, 5(3), 153-177. https://doi.org/10.1207/ s15327841mpee0503_3 zhou, h., zhang, l., luo, x., wu, l., zou, x., xia, k., . . . jiang, y.h. (2017). modifying the autism spectrum rating scale (6–18 years) to a chinese context: an exploratory factor analysis. neuroscience bulletin, 33(2), 175-182. https://doi. org/10.1007/s12264-017-0104-7 144 december 2022, volume 15, issue 2, 137-144 appendix 1 scale to assess activities of daily living at home in children and adolescents with autism spectrum disorder part 1. sociodemographic data of parents: date of birth: ____/____/_____ marital status: married ( ) single ( ) widowed ( ) divorced ( ) area where you live: urban ( ) rural ( ) part 2. sociodemographic data of the child, adolescent, young person. date of birth: ____/____/ ______ sex: male ( ) female ( ). lives with: father ( ) mother ( ) both ( ) other relatives ( ). has brothers: 1 ( ) 2 ( ) 3 ( ) >4 ( ). n questions independently with supervision with assistance (support) with full support 1 in personal care for grooming (bathing, washing, combing hair, shaving) 4 3 2 1 2 in personal care to get dressed and undressed 4 3 2 1 3 in household chores for shopping (clothes, food...) 4 3 2 1 4 in household chores to clean the house (sweeping, mopping, etc.) 4 3 2 1 5 in mobility within the home to move around 4 3 2 1 6 in mobility inside the house to get in and out of the bathtub. 4 3 2 1 7 in mobility inside the house for getting up and going to bed 4 3 2 1 8 in mobility outdoors to go up and down stairs 4 3 2 1 international electronic journal of elementary education, 2016, 8(3), 481-506 short vowels versus word familiarity in the reading comprehension of arab readers: a revisited issue abdullah m. seraye  king saud university, saudi arabia received: 19 july 2015 / revised: 31 october 2015 / accepted: 2 november 2015 abstract arab readers, both beginning and advanced, are encouraged to read and accustomed to unvowelized and undiacriticized texts. previous literature claimed that the presence of short vowels in the text would facilitate the reading comprehension of both beginning and advanced arab readers. however, with a claimed strict controlling procedure, different results emerged, revealing that the only variable that affected the reading process of arab adult skilled readers was word frequency, and its effect was limited to the time load of the reading process; this result raised the question of whether the neutral role of short vowels in the text reading process of experienced arab readers would be maintained for less experienced readers, as represented by fourth graders, or whether word frequency would be the only variable that plays a role in their reading process. in experiment, 1,141 fourth-grade students were randomly assigned to 5 reading conditions: plain, only shaddah, short vowels plus shaddah, only short vowels, and finally the wrong short vowels plus shaddah. in experiment 2, 38 participants from the same population were assigned to a fully vowelized and diacriticized reading condition. each participant was asked to read two texts, of high and low frequency words and then given recall and multiple-choice tests. in general, the multivariate analysis showed that the only manipulated variable that was found to affect their reading process in terms of reading time load and, to some degree, reading comprehension was word frequency, although its effect was marginal. accordingly, pedagogical recommendations and future research were proposed. keywords: arabic short vowels, arabic beginning readers, reading comprehension, arabic orthography. introduction reading comprehension, both process and product, is affected by many factors that are related to the text, the reader, and the interaction between the text and the reader (kendeou, muis, & fulton, 2010). indeed, those factors can also be classified as either internal or external factors. internal factors are considered to be factors that are related to the text per se, and thus they are considered to be textual, such as the words embedded in  abdullah m. seraye, king saud university, college of education, dep. curriculum & instruction, building 15, 2nd flr office, po box 2458, riyadh 11451 saudi arabia, phone (+966) 467-4611, email: aalseraye@ksu.edu.sa international electronic journal of elementary education vol.8, issue 3, 481-506, 2016 482 the text in terms of frequency and abstractness (e.g., ryder & hughes, 1985), syntactic structures, genre (e.g., meyer & freedle, 1984), and text organizing structure (e.g., meyer, 1975). external factors, on the other hand, are considered to be exterior factors that are related to the reader, such as his/her prior knowledge of the topic (bransford & johnson, 1972; kendeou, rapp, & van den broek, 2004, 2007; kientch, 1988), reading skills, schemata and beliefs (kardash & howell, 2000; kendeou et al., 2010), interest, and knowledge of and skills in using cognitive and metacognitive strategies (e.g., zwaan & brown, 1996). one internal factor that is exclusive to texts written in arabic is its orthography, which permits a dual representation of its scripts: a shallow, transparent orthography, in which the appropriate short vowels (dhammah ُ , fatha َ , and kasrah ) and diacritics (shaddah _ and skun ْ_ ) are supplement with the consonants in the text, and a deep orthography, in which only the consonants are presented and no appropriate short vowels or diacritics are provided (abu-raiba, 1995; mahmoud, 1980; seraye, 2004). arabic is a semitic language written from right to left with an alphabetic-principlebased writing system that represents both consonants and short vowels, including the diacritics, separately and voluntarily. the arabic writing system visually consists of two types of symbols: consonants that represent, to a large extent, the trilateral/quadrilateral root of words, and tiny visual signs that are affixed to the consonants and take the shapes of diacritics. however, by analyzing the script symbols in terms of functions, we could divide the arabic writing system symbols into four types. (1) there are 26 consonants, including the three long vowels that represent both a consonant and a vowel under some conditions: “alif: ا ,” “waaw: و ,” and “yaa’: (2) ”. ي there are three short vowels, “fathah,” “dhammah,” and “kasrah,” that mark the equivalent short vowels in english, a, u, i, and take the forms of diacritics, and the following shapes: ‘ َـ ‘ ’,ـ,’ and ‘ ـ ‘ respectively. (3) diacritics are very small visual signs that are superscripted to the consonants or the letters in a word: skun is represented with the symbol ‘ ْ_’ to indicate that the consonant is vowelless; shaddah is represented by ‘ _ ’ and is used to indicate a doubled consonant “geminated,” and maddah is represented with the symbol ‘ _~ ’ and is placed over the consonant ‘ ا ’ to indicate the combination of two consonants, alif ‘ ا ’ and hamza ‘ ء .’ finally, (4) the caseending markings are very small visual symbols that take the following shapes “ ْ “, “ “, “ َ “, “ “, to indicate syntactically different cases, including the nominative, genitive, and accusative cases. the short vowels might be doubled to indicate nunation, which takes the following shapes: ـ ,ـ, and ـ (bateson, 1967; campbell, 1997; mahmoud, 1979). the voluntariness of representing short vowels and diacritics in arabic print that characterizes arabic orthography has resulted in a dual representation of its print: a transparent orthography in which the appropriate diacritics are supplemented with the consonants, as can be seen in traditional and young children’s texts; and a deep orthography in which only the consonants and no appropriate diacritics are presented, as can be seen in public print, newspapers, and books written for adults. this uniqueness of arabic orthography of being of dual representation has attracted some researchers to evaluate its effect on the process of reading arabic at all textual levels—word recognition, sentence understanding (specifically parsing), and text integration—using different populations and different types of materials and methodologies (see for example abu-rabia, 1995-2001; ibrahim, 2013; ibrahim, eviatar, & aharon peretz, 2002; seraye, 2004). subsequently, researchers also examined word recognition, sentence parsing, and text reading and comprehension to assess the extent to which the departure of a writing system from representing speech, as can be realized in the absence of short vowels/pointings and diacritics from script, might influence the reading process (chitiri, 1991; seraye, 2004; shimron, 1993). such investigations would short vowels versus word familiarity / seraye 483 help researchers in the field of reading to arrive at a universal explanation of the “blueprint of the reader” (perfetti, 1999) through integrating other alphabetical writing systems and unique orthographies in the equation, constructing a reading model of the arabic reading process. finally, they would also help arabic curriculum designers and policy makers to suggest appropriate reading instruction methods and reading scripts for school pupils. one of the major concerns over the role of those missing tiny short vowels and diacritics from arab students’ print/textbooks was its effect on students’ reading comprehension, a concern that drew the attention of some arab researchers to investigate to what extent the absence of short vowels (and diacritics) from arab reading materials would affect students’ reading process while they attempt to integrate the text in order to understand it. abu-rabia (1999) conducted two experiments to compare the effect of short vowels on the reading comprehension of two different populations: second graders (given the label “the beginning readers”) and sixth graders (given the label “the advanced readers”). in experiment 1, 74 sixth graders, aged 12 to 12 ½, were divided randomly into two groups: in one group, the participants individually and silently read a vowelized short story taken from their “basic reader,” and answered 10 multiple-choice vowelized questions. in the other group, the participants individually and silently read the unvowelized version of the same text and answered 10 multiple-choice unvowelized questions. in a within-subject design, the researcher conducted the same procedure by asking 71 second graders, aged 7 to 8, to read two different narrative texts, one vowelized and the other unvowelized, and then answer seven multiple-choice vowelized questions, in two sessions with two days elapsing between them. the results showed that both the beginning readers and the advanced ones benefited from the presence of short vowels in the texts, which facilitated their reading comprehension, as can be realized from the significant difference between the means. the second graders on average scored higher with vowelized texts (m= 6.34, sd= 1.58) than with unvowelized ones (m= 5.46, sd= 2.00) with a maximum score of 7 for both tests, and the sixth graders on average scored higher with the vowelized text (m= 7.20, sd= 1.70) compared with the unvowelized one (m= 6.10, sd= 2.22) with a maximum score of 10 for both tests. according to abu-rabia (1999), this result can be explained in terms of the role of short vowels in providing the text with phonological information that “affects working memory in processing text information in such a way that, if information is also phonologically coded in working memory, this will maintain that information longer during reading, which facilitates reading comprehension” (p. 100). in another study (abu-rabia, 2001), three tasks were designed to investigate the role of short vowels, pointings (the counterpart of short vowels in hebrew orthography), and context on the reading process of arab adults as they read arabic and hebrew reading materials: words, short paragraphs, and short story texts. in one task, 65 native arab adults, who were university students aged 22 to 30 years and were proficient in arabic and hebrew, were assessed on their reading comprehension in relation to the presence and absence of both short vowels and pointings as they read short story texts written in arabic and hebrew. there were two different short stories: one written in arabic and another written in hebrew. from each story, two versions were constructed to present the reading condition: one was presented fully vowelized for the case of arabic, or fully pointed for the case of hebrew, and the other was presented fully unvowelized or unpointed, respectively. although it is not clearly stated, it seems that the participants were divided into two groups, and each group received one reading condition. one group individually and silently read the vowelized arabic text and the pointed hebrew text, and the other group international electronic journal of elementary education vol.8, issue 3, 481-506, 2016 484 individually and silently read the unvowelized arabic text and the unpointed hebrew one, then they answered six multiple-choice questions (with 6 as the ultimate score). the results demonstrated that the participants on average performed better with the fully vowelized and pointed texts than with the ones presented without short vowels or pointings (m= 4.51, sd= 1.20 for the vowelized arabic text, and m= 4.10, sd= 1.56 for the unvowelized arabic text; m= 2.43, sd= 1.39 for the vowelized/pointed hebrew text, and m= 2.27, sd= 1.16 for the unvowelized/unpointed hebrew text). abu-raiba (2001) attributed these results to the presence of short vowels and pointings in the texts, which provided “additional phonological information” (p. 52) and subsequently helped in understanding the text. this finding of a positive effect of short vowels on the reading comprehension of arab adults does not just support abu-rabia’s (1999) previous study in showing a consistent result, but also empowers the role of short vowels in facilitating the reading comprehension process of arabic readers regardless of their reading skill. not just beginning readers but also experienced ones (adults) benefitted from the presence of short vowels in the texts they read. however, examining the mean values for both studies shows that the less experienced readers, second graders, benefitted much more from the presence of short vowels in the texts; they reached comparatively higher maximum scores than the experienced skilled readers, sixth graders and adults. in the case of hebrew texts, the positive effect of short vowels (pointings) on the reading comprehension of adults as they read a hebrew text was in agreement with a previous study by shimron and sivan (1994), which showed that providing a hebrew text with short vowels/pointings helped the adults comprehend the text to some degree better than they comprehended an unvowelized/unpointed text (m= 1.75, sd= 0.44 for correct answers for the vowelized text; m= 1.42, sd= 0.72 for the unvowelized text). however, reading comprehension, as can be assessed by the load reflected in the reading time it took the participants to read the texts, showed that on average it took the participants the same amount of time to read the vowelized/pointed and unvowelized/unpointed hebrew texts (m= 68.8 s, sd= 31.3 s for the unvowelized text; m= 69.0 s, sd= 30.4 s for the unvowelized text). it can be concluded from abu-rabia’s (1999, 2001) studies that short vowels (i use his term roughly here, because presenting the text fully vowelized does not mean providing the text with only short vowels but means providing it with other diacritics that also represent speech: shaddah, skun, and case markings) had an effect on the reading comprehension of arab readers as they read a text silently for comprehension, and that was consistent regardless of the type of population involved in the process: beginning readers with less experience or advanced ones with more experience. further, the effect was noticed on the product of the reading comprehension process as was assessed by the multiple-choice test. other variables for assessing the reading comprehension process, as can be realized by the reading time it takes the subjects to read a text, were not incorporated in the two studies, although they reflect the reading comprehension process. thus, more than one drawback can be observed in abu-rabia’s (1999, 2001) studies. these drawbacks are related to the methodology applied, including the dissimilarity and type of texts used (e.g., the length of his experimental texts were equalized on the number of words, and not on the number of morphemes where they should due to the affixation feature of arabic; nor they were equalized in terms of word frequency, syntactic structure, or genre) and their familiarity to the participants, especially in the 1999 study. the author also failed to manipulate the short vowels to the degree that their effect alone would be isolated, because the author included shaddah and skun as short vowels in arabic. further, short vowels versus word familiarity / seraye 485 he included even the case endings, tiny signs take the shape of short vowels and the diacritic skun, which function in relation to the syntactic structure of the sentence. the measurement applied was insensitive (only a multiple-choice test was used, not a recall test), and finally the difference in the results was overestimated and the author did not point out to the slight difference between the means. in the 1999 study, the means were 7.20 out of 10 for the vowelized condition and 6.10 out of 10 for the unvowelized condition, a difference of 1.10, and 6.34 out of 7 for the vowelized condition and 5.46 out of 7 for the unvowelized condition, a 0.88 difference. it should be noted that the measurement scale involved one point for each correct answer, and therefore a 1.10-unit difference and a 0.88-unit difference between the means were equivalent to differences of 1.10 and 0.88 correct answers, respectively. thus, the effects of short vowels on the reading comprehension of both advanced and beginning readers should have taken into account the sizeable difference between the means and the measurement scales employed in the study. for the 2001 study, it was found that the participants performed better with the vowelized texts and with the pointed texts than with the texts that were presented plain— that is, without short vowels or pointings. however, examining the means between each pair of reading conditions shows the same phenomenon found in the 1999 study; only a very slight difference between the means was found. for the arabic texts, the participants scored 4.51 on average for the vowelized arabic text and 4.10 for the unvowelized arabic text, a difference of 0.41; for the hebrew texts, the participants on average scored 2.43 for the pointed hebrew text and 2.27 for the unpointed text, a difference of 0.16, which equals a 0.16 difference in correct answers. because the measurement scale involved one point for each correct answer, the 0.41-unit difference and the 0.16-unit difference between the means were equivalent to differences of 0.41 and 0.16 correct answers, respectively. thus, interpreting the effects found for the short vowels and pointings on the reading comprehension of arab adults should have taken into account the sizeable difference between the means and the measurement scales employed in the study. accordingly, attributing this difference in comprehension performance to the short vowels per se is still questionable under these conditions, as abu-rabia’s studies did not control for the short vowels to the degree that would isolate their effect accurately, nor did they control for the texts used in terms of being identical, as he used two uncontrolled-for texts. furthermore, he included other diacritics, such as the shaddah sign, ـ, as short vowel signs, which means that “the representation of the short vowels was not scientifically and experimentally manipulated to the degree that the extraneous variables were controlled” (seraye, 2004, 56). therefore, it was necessary to take such realities into account to reassess the role of short vowels in the reading comprehension of adult arabs who were exposed only to unvowelized texts and to determine the degree to which such exposure would affect their reading process as they read vowelized texts versus unvowelized texts. in experiment 1 of a three-experiment study, seraye (2004) study responded to such concern and assumed that, once we control for the reading materials used, the procedure, and the reading condition, the homographic phenomenon that characterizes the unvowelized arabic text would not affect people’s reading comprehension, nor would the presence of short vowels. he reasoned that arab adults (considered to be experienced readers) would use their language experience, particularly the morphological stem of the arabic root of the words, their trilateral/quadrilateral root, and the linguistic textual context, in understanding the text. thus, seraye (2004) used a matching procedure to construct matched texts on all textual levels. in terms of short vowels, he differentiated between full consonant representation and full “morphological short vowel” representation in order to exclude confounding effects of other diacritics (e.g., shaddah, international electronic journal of elementary education vol.8, issue 3, 481-506, 2016 486 case-ending markings, and skun) and thus to determine the role of short vowels in comprehension. further, he built a frequency effect into the design because word frequency could covariate with the vowelization effect. he also used a retelling procedure (recall), which is considered a better indicator of readers’ performance on texts, in addition to a multiple-choice test (lipson & wixson, 1997). for the purpose of the study, two texts were constructed, one high-frequency text (hf) and one low-frequency text (lf), by using a matching procedure to guarantee the identicality between the texts, except in word frequency. the two texts, one hf and one lf, were of 504 words and 834 morphemes each and served as the main texts in the experiment. five versions of each text were constructed to represent the reading conditions: in the first condition the text was given plain (no short vowels or shaddah), in the second condition the text was provided with only shaddah, in the third condition the text was provided with short vowels plus shaddah, in the fourth condition the text was provided with short vowels minus shaddah, and finally in the fifth condition the text was provided with wrong short vowels plus shaddah. the author included the only-shaddah and only-vowels conditions in his design for control purposes. the participants were 104 native arabic speakers aged 19 to 40. they were randomly divided into five groups and then assigned randomly to the five reading conditions. thus, in each reading condition, the participant silently read two texts, hf and lf, in two sessions separated by 14–20 days. the order of passage presentation was rotated to counterbalance reading materials and reading conditions within each group. the first group read the plain texts (no short vowels or shaddah were provided); the second group read the texts with only shaddah (only shaddah was provided with the text); the third group read the texts vowelized with shaddah (both short vowels and shaddah were provided); the fourth group read the vowelized texts (only short vowels were provided), and the fifth group read the texts wrongly vowelized with shaddah (both short vowels and shaddah were provided on the wrong positions in the words, which would lead, if they were read with the consonants, to phonemic distortion and not to graphemic distortion). after each reading, the participant was asked to recall what he read and then respond to 10 multiple-choice questions. there are three dependent variables collected: the reading time it took the participant to complete the text, measured in milliseconds; the propositions recalled after the reading task and the correct answers on the multiple-choice test; and two independent variables, the vowelization conditions and the text types, hf versus lf. the results showed that for the reading time data, a significant main effect was found for text type (hf vs. lf), but not for reading condition. further, there was no significant interaction between text type and reading condition. as a result, it did not matter which reading condition the participant was in; it always took him longer to read the lowfrequency text than the high-frequency text. on average, it took the participant 206.32 s to read the lf text and 194.13 s to read the hf text. for the number of propositions from the recall test data, no significant main effects for text type or reading condition were found. further, the results did not show any significant interaction between text and reading condition. thus, it did not matter which text the participants read or which reading condition they were in; their performance was on average the same. there was a 1.5-unit difference in recall between the marginal means for the lf and hf texts (30.83 and 29.31, respectively), with the note that the measurement scale involved one point for each meaningful proposition, and therefore a 1.5-unit difference was equivalent to a difference of 1.5 propositions. for the number of correct responses as measured by the multiple-choice test, the analysis revealed exactly the same result that was obtained from analyzing the data of the recall test. that is, no significant main effects for reading condition or text type were short vowels versus word familiarity / seraye 487 found, nor was there a reading condition × text type interaction. thus, it did not matter which reading condition the participants were in or which text they read; their performance was on average the same. in fact, the difference between the marginal means for reading condition and the difference between the marginal means for text was a fractional difference (only a 0.1 difference between the hf text marginal mean and the lf text marginal mean: 7.62 and 7.52, respectively). note that the measurement scale involved one point for each correct response, with an ultimate score of 10 points. in general, the results showed that the presence or absence of short vowels and diacritics in combination does not affect the reading process comprehension of skilled adult arabic readers. in sum, the results demonstrated that the only variable that affects the reading process of skilled adult arabic readers is word frequency. those findings of seraye (2004) did not contribute positively to the subsequent investigations conducted by abu-rabia and others. for example, in a descriptive comparison study of dyslexics of grade 8 (n= 29) and matching normal readers on both reading performance (n= 29 of grade 6) and chronological age who had similar general ability performance (n= 31 of grade 8), abu-rabia and abu-rahmoun (2012) compared the performance of the three groups on multiple tasks—phonological, orthographic, spelling, and reading comprehension tasks—under two reading conditions: vowelized and unvowelized. on the reading comprehension task, the participants were asked to read two different informational texts (only controlled and equalized on the number of lines), one fully vowelized and the other presented plain, which were chosen from their basal reader for grade 8. the analysis revealed that all groups’ participants benefitted from the presence of short vowels; that is, the participants performed better with the vowelized text (m= 6.44 vs. 5.33 for dyslexics, 7.66 vs. 4.83 for normal readers of matching reading level, and 9.48 vs. 8.32 for normal readers of matching chronological age, for the vowelized and unvowelized texts, respectively). furthermore, the two normal groups comprehended the vowelized text much better than the dyslexic group. however, the size difference between the means, particularly for the dyslexics and the group of normal readers matching the dyslexics on age variable, was still a small difference, 1.11 and 1.16 units. in line with abu-rabia’s perspectives on the role of short vowels, abu-hamour et al. (2013) administered two tasks adopted from the curriculum-based management assessment in order to ensure its applicability in evaluating the reading performance of arabic fifth graders. there were 131 fifth graders (divided into two groups: 89 of skilled reading ability, and 42 with struggling reading ability) and two tasks: oral reading fluency and silent reading comprehension. in the second task, which was designed to assess the reading comprehension of fifth graders in relation to the presence and absence of short vowels, the participants were given a text of 300 words with reading conditions (plain, partially vowelized, and fully vowelized) with every seventh word deleted and were asked to silently read the passage within three minutes and “restore” the right words from the three alternatives provided after every deleted word. thus, the number of correctly restored words was the dependent variable in the study (42 were the maximum score). thus, every participant was asked to read three texts under the three reading conditions and restore the deleted words from the three alternative options. although the skilled readers scored higher than the struggling readers, both skilled and struggling readers performed better with the vowelized texts (m= 24.19, sd= 5.64 for skilled readers on the vowelized text and m= 7.88, sd= 3.82 for the plain text; m= 14.19, sd= 5.51, m= 3.76, sd= 2.80 for struggling readers on the vowelized and plain texts, respectively. their justification for these results is that, unlike the situation with the plain text, vowelizing the text makes it transparent and reserves the attention needed for comprehension. international electronic journal of elementary education vol.8, issue 3, 481-506, 2016 488 from the overall results, two factors were found to play a role in the arabic reading process for reading time and reading comprehension: short vowels, as abu-rabia (1999; 2001), abu-rabia and abu-rahmoun (2012), and abu-hamour et al. (2013) observed; and word frequency, as observed by seraye (2004). therefore, the question is whether the effect of short vowels on reading comprehension, as found by abu-rabia (1999) and abuhamour et al. (2013), holds with the correct categorization of less experienced readers, as represented by fourth graders—who are starting to read across the curriculum and for learning, and whose textbooks are written in a deep orthography, versus with second graders, who are still learning to read, or fifth graders, who were more advanced readers, when the appropriate experimental control is taken. alternatively, should we find word frequency, as found in seraye’s (2004) study on arab adults, to be the only variable that affects both populations’ reading processes, with the more experienced readers represented by arab adults (seraye, 2004) and the less experienced readers represented by fourth graders in the current study? therefore, a combination effect of both short vowels and word frequency on the reading comprehension process of fourth graders as they read connected texts is investigated. such an investigation should accomplish the following aims: it should illustrate the role of short vowels, diacritics, and word frequency in the reading comprehension process of the less experienced readers, as represented by fourth graders, and subsequently tap into the results on how beginning readers’ print and textbooks should be presented. second, the results should help in building a model of the reading process in arabic orthography by determining the factors that affect the process. therefore, the goal of the current study is to assess the effect of short vowels per se, the word frequency per se, and the short vowels and word frequency in combination on the reading process of fourth graders’ reading time and reading comprehension, as they read a connected text. therefore, two main questions were raised to represent the two main independent variables found to affect the reading process of arabic texts: short vowels and word frequency. since reading comprehension can be assessed as a process (reading time) and a product (correct responses), two sub-questions under each main question were constructed to assess the effect of short vowels and word frequency on the reading comprehension of skilled children arab readers, as represented by fourth-grade students. according to the previous observations, presenting the short vowels or the short vowels and diacritics in combination within an expository text should not affect the reading comprehension of fourth graders, as can be measured by the number of propositions they recall or the correct answers they score. the assumption is justified by the fact that skilled readers will use their linguistic knowledge, their knowledge of the trilateral/quadrilateral root that characterizes arabic morphology, and the text context to compensate for the missing short vowels and diacritics from print (abu-rabia, 2002; seraye, 2004; abu-rabia & abu-rahmoun, 2012). however, their reading process, in terms of the time they need to read the text, will be affected by the presence and absence of short vowels and diacritics: fourth graders should take more time to read a plain text than a vowelized one due to the heterophonic homographic phenomenon in arabic words that, when starting a sentence, might garden-path the reader to the degree that it would force him/her to reread the sentence in order to choose the right word form (for more detail, see experiment 2 in seraye, 2004). further, it is assumed that word frequency might affect their reading time, but not their reading comprehension. inserting 15–25% of the low-frequency words in the text should not affect their reading comprehension (ryder & hughes, 1985; seraye, 2004), because the fourth graders would exploit their knowledge of the morphological roots of arabic words, trilateral/quadrilateral roots, and the text context in constructing a textual short vowels versus word familiarity / seraye 489 representation of the text they read. however, along with reading time, word frequency was found to affect arab adult readers—the experienced readers (seraye, 2004); subsequently, it is also expected to affect the less experienced readers, the fourth graders. 1) do short vowels play a psychological role in the reading comprehension process of skilled arab children as represented by fourth graders? 1a) is there a significant difference in the reading comprehension process, as measured by the reading time, of skilled fourth graders when reading a vowelized versus an unvowelized text? 1b) is there a significant difference in the reading comprehension product, as measured by the multiple-choice and recall tests, of skilled fourth graders when reading a vowelized versus an unvowelized text? 2) does word frequency play a psychological role in the reading comprehension process of skilled arab children, as represented by fourth graders? 2a) is there a significant difference in the reading comprehension process, as measured by reading time, of skilled fourth graders when reading a high-frequency versus a low-frequency text? 2b) is there a significant difference in the reading comprehension product, as measured by the multiple-choice and recall tests, of skilled arab children when reading a high-frequency versus a low-frequency text? method experiment 1 participants. one hundred and forty fourth-grade native arab male students were chosen from three elementary public schools in riyadh, saudi arabia. their ages ranged between 9 and 10. they were offered r20 as compensation for their participation. all of them had normal vision and reported no learning or reading difficulties. the students’ reading assessment scores from the first term were initially used to select suitable participants for the study, and other criteria—preand posttest procedures—were used to screen the participants, in terms of their reading proficiency level. only those who had 40 or more correct readings from the 50-word list were included in the study. further, a post-criteria judgment (reading a short passage) was administered in the second session for each participant to ensure that only those who expressed reading fluency were included in the experiment. initially, 147 students were selected and then randomly divided into five groups, and then randomly into five reading conditions. consent and admission were officially taken before administering the experiments. out of the 147 participants who took part in the study, 6 participants were excluded from the study data due to either the post-criteria procedure, which revealed that they were under grade level in reading, or to their moving from the schools during the experiment materials. the use of controlled texts instead of familiar and different texts was necessary to address the questionable controlling issues observed in previous studies; thus, two long expository arabic texts were constructed for experiment 1: one of high frequency (hf) and the other of low frequency (lf). to ensure equality between the two texts, a highfrequency text was constructed, and its low-frequency counterpart was created using a matching process. other than the word frequency, the two texts were equalized on all aspects, including sentence structure, length, neighboring word size, and even on the number of words and morphemes, due to the affixation feature of the arabic morphological system. in terms of the test’s organizing structure, the events spots, pronoun names, and identities of the characters were replaced with other event spots, international electronic journal of elementary education vol.8, issue 3, 481-506, 2016 490 pronoun names, and identities (see seraye (2004) for more details on the matching technique). thus, there were two texts of high and low frequency, respectively (each text contained 166 words and 320 morphemes). sixteen words from the hf text (10% of the words in the text) were replaced with their low-frequency synonyms. five versions of each text were created to control the short vowels and diacritics, as well as isolate the effects of the short vowels per se: one version was left plain—that is, unvowelized; the second was fully consonantally presented by providing the text with only the diacritic shaddah, _ ); in the third version, the text was presented with both short vowels and shaddah; with the fourth version, the text was supplemented with short vowels only and without any diacritics; finally, for a controlling procedure, the text in the fifth version was provided with the wrong short vowels and shaddah so that, if ignored and not assembled with the consonants, the text would resemble the plain text that the arab readers were exposed to (see table 1). table 1. reading conditions reading condition arabic script 1 plain وديةعبدالعزيز يعيش هو وعائلته في قرية من ... السع 2 fully consonantly: with shaddah only ودي ةعبدالعزيز يعيش هو وعائلته في قرية من ... الس ع 3 short vowels with shaddah ن ... السَّع يز يَع يش ه و َوَعائ لَته ف ي قَريَة م يَّةَعبدالعَز ود 4 only short vowels ن ... الَسع يز يَع يش ه و َوَعائ لَته ف ي قَريَة م يَةَعبدالعَز ود 5 wrongly vowelized and shaddah َّع ائ لَته ف ي ق ري ة َمن ... الس يز ي عَيش ه و َوع يَّةع بدالعَز َود further, a short passage of 100 words, written in both shallow and deep orthography, and a fifty-word list were constructed to assess the participants’ reading proficiency level for the inclusion and exclusion criteria. the texts and word list were assessed and judged by experts in the arabic teaching field, and by primary-grade reading teachers. measures. three types of data were collected: the reading time in which the participant read the text, in milliseconds; and reading comprehension, as assessed by the recalling test based on the number of propositions recalled and by the correct answers based on the 10item multiple-choice test. both tests were judged by experts in the arabic teaching field, and by primary grade reading teachers for their content validity, format, accuracy, etc. both tests were piloted before the actual experiment. the recall test’s reliability was examined using rater judgment, and the correlation coefficient was found to be r = 0.97. for the multiple-choice test, the alpha value was 0.67 for the hf test and 0.65 for the lf test. further, a congruent validity was adopted, and the pearson correlation coefficients between the recall and the multiple-choice tests were found to be significant (r = 0.70 for the hf test and r = 0.69 for the lf test). design. a split-plot factorial mixed 5 x 2 design was adopted for the current study (kirk, 1982), with one between-subject factor (the five reading conditions) and one withinsubject factor (the text type: hf vs. lf), in order to determine the role of each independent factor per se and in combination on the fourth graders’ reading comprehension as they read a connected text. counterbalance procedures were conducted between texts within every group in order to avoid the order effect. procedure. as explained before, previous studies do not to control for the carry-over effect from having the two texts read simultaneously, or from having a short time elapse short vowels versus word familiarity / seraye 491 between the reading sessions. in one of abu-rabia (1999) experiments, for example, the task was given in two sessions with two days elapsing between them. carry-over effect is possible in such a situation. therefore, in order for the current study to have a controlled procedure, the carry-over and order effects were taken into account through counterbalancing and 14-20-day time interval between the experimental sessions (seraye, 2004). thus, in experiment 1, the participants were asked to read the two texts (the hf and lf texts) in two separate sessions, at intervals between 14 to 20 days. the tests were given in an empty, quiet room in the participants’ schools. in session one, the participant was asked to read one of the texts for reading comprehension (either hf or lf) silently and at his pace; he was asked to read the other text in the second session. directly following each reading, the participant was asked to recall the text he just read (prompts were used to prevent the participant from being selective in his recalling). the time it took each participant to complete the reading was calculated in milliseconds, and his recall was recorded. after the recalling task, the participant was given the multiple-choice test and instructed to avoid any guessing responses as well as to answer the questions according to what he read. the same procedure was followed for the second session, except that the participant was given a short passage at the end of the session to read orally and accurately, in order to double check the participant’s reading skill, which would then be included or excluded from the data analysis. however, an additional procedure was taken for the group that was asked to read the wrongly vowelized and shaddah text. these participants were informed about the wrong short vowels and shaddah prior to reading. the order of text presentation was rotated within every group to counterbalance the texts and reading conditions. both experiments were conducted and administered by the researcher. results. in order to answer the two questions and sub-questions proposed (a and b), three dependent variables were collected: reading time, as measured in milliseconds; the number of propositions, as assessed by the recall test; and finally, the number of correct answers, as assessed by the multiple-choice test. a multivariate analysis (a two-way repeated-measures analysis of variance) was adopted to determine whether there was a main effect for the short vowels per se, a main effect for the word frequency per se, or an interaction between the two independent variables, short vowels and word frequency, on the reading comprehension of the fourth graders. reading time for questions 1a and 2a on the reading time data (“is there a significant difference in the reading process, as measured by reading time, among fourth graders when reading a vowelized versus an unvowelized text, and when reading a high-frequency versus lowfrequency text?”), the analysis on the reading time data revealed a significant main effect for text type only (hf vs. lf), f(1, 135) = 5.59, p = .019, p²= 0.04, and not for reading condition f(4, 135) = 0.277, p = .893, nor there was a significant interaction between the text type and reading condition, f(4, 135) = 0.878, p = .479. however, the post hoc analysis conducted on each pair, using tukey and scheffe tests, did not show any significant differences between the compared pairs. subsequently, despite the reading condition the participant was in, he would always take more time to read the low-frequency text than the high-frequency text. the participants spent 160.60 seconds reading the lf text and 151.53 seconds reading the hf text (see table 2). further, the pair means in table 2 for the five reading conditions show that the participants in reading condition 3—compared with the other groups, in which short vowels and shaddah were provided with the text— spent less time, on average, reading the two texts (146.07 and 148.68 milliseconds for the hf and lf texts, respectively). however, both groups, the plain one and the vowelized one, spent on average the same time to read the hf text (146.87 and 146.07 respectively). this international electronic journal of elementary education vol.8, issue 3, 481-506, 2016 492 result is inconsistent with the proposed assumption that beginning readers should require more time to read a plain text than to a vowelized text due to the heterophonic homographic words in arabic, particularly with the garden-path sentences in the text. the partial eta squared value (p²) was calculated as a measure of effect size for the text type variable, the word frequency. its value of 0.04 is considered to be of small to medium effect size (kotrlik & williams, 2003). it indicates that 4 % of the total variance in the dependent variable, the reading time, is attributed to the manipulated word frequency. table 2. cell and marginal means on the reading time test by reading condition and text group reading condition text high freq. low freq. m sd m sd margina l 1 plain: no short vowels or shaddah 146.87 69.49 156.33 73.67 151.60 2 fully consonantly: with shaddah only 150.96 79.38 173.32 111.70 162.14 3 vowelized: short vowels and shaddah 146.07 76.63 148.68 67.77 147.38 4 vowelized: short vowels, with no shaddah 162.46 87.20 169.64 79.23 166.05 5 wrongly vowelized: wrong short vowels and shaddah 151.62 78.01 154.92 82.01 153.27 marginal means 151.53 77.32 160.60 83.45 156.06 reading comprehension for questions 1b and 2b on the recalling test data (“is there a significant difference in the reading comprehension, as measured by the number of propositions, of fourth graders when reading a vowelized versus an unvowelized text, and when reading a high-frequency versus a low-frequency text?”), the analysis on the number of propositions in general did not reveal any significant results: there were no main effects for text type, f(1, 128) = .022, p = .883, or reading condition, f(4, 128) = 0.371, p = .829, nor there was a significant interaction between text type and reading condition, f(4, 128) = 0.508, p = .730. regardless of the text type and reading condition, the participants on average performed the same in recalling the texts’ propositions. in fact, the differences between the marginal means for the text type and between the pair means for every reading condition, as shown in table 3, are very small, when we take into account that the measurement scale adopted in the study involved one point for each meaningful proposition recalled. therefore, a 0.12 difference between the marginal means of the hf and lf texts (14.46 and 14.58, respectively) and the roughly one-proposition difference between the pair means for every reading condition (table 3) are equivalent to a difference of 0.12, and one proposition, respectively. short vowels versus word familiarity / seraye 493 table 3. cell and marginal means on the recall test by reading condition and text reading condition text high freq. low freq. m sd m sd marginal 1 13.90 9.97 13.74 10.54 13.82 2 14.89 8.28 15.19 9.45 15.04 3 15.48 7.65 14.44 6.30 14.96 4 14.44 9.61 14.63 8.89 14.54 5 13.60 9.23 14.92 8.57 14.26 marginal means 14.46 8.95 14.58 8.75 14.52 for questions 1b and 2b on the multiple-choice data (“are there any significant differences in the reading comprehension, as measured by the number of correct answers, of fourth graders when they read a vowelized versus an unvowelized text, and when they read a high-frequency versus low-frequency text?”), the analysis on the number of correct answers data revealed a significant main effect for text type (hf vs. lf) f(1, 134)= 21.80, p= .000, p²= 0.14, but not for reading condition f(4, 134) = 0.28, p= .89, nor for the interaction between text type and reading condition, f(4, 134)= 0.61, p= .65 (table 4). examining the marginal means for the hf and lf texts shows that the participants on average scored higher on the hf text, with a mean= 6.44, versus a mean= 5.56 for the lf text. however, even with the reported large effect size value of 0.14 (kotrlik & williams, 2003), the marginal means shows only a 0.88 unit difference, which is a very slight difference when the measurement scale adopted is taken into account. the current study adopted a measurement scale that involves one point for each correct answer; thus, a 0.88 unit difference is equivalent to a difference of 0.88 correct answer points. however, the post hoc analysis conducted on each pair, using tukey and scheffe tests, did not show any significant differences between the pairs of comparison. the partial eta squared value (p²) of 0.14 indicates that 14 % of the total variance in the dependent variable, number of correct answers, is accounted for by the manipulated word frequency variable. international electronic journal of elementary education vol.8, issue 3, 481-506, 2016 494 table 4. cell and marginal means on the multiple-choice test by reading condition and text reading condition text high freq. low freq. m sd m sd marginal 1 6.81 2.52 5.42 2.17 6.11 2 6.43 2.53 5.79 2.60 6.11 3 6.60 2.22 5.70 2.20 6.15 4 6.29 2.32 5.32 2.55 5.80 5 6.08 2.17 5.58 2.10 5.83 marginal means 6.44 2.35 5.56 2.33 6.00 general results two main findings were revealed by the analysis: one is that vowelization is neither attributed positively nor negatively to the reading process of beginning readers, as represented by the fourth-grade students. that is, the supplemented vowelization did not affect their reading process for comprehension, as was measured by the time it took the fourth grader to read the text, or by the number of propositions recalled and the correct responses scored. additionally, the only factor that played a role in the reading process of beginning readers was word frequency. its role was observed with the reading time, and with the reading comprehension, as assessed by the correct responses scored, although its effect might not be practically affected due to the observed sizeable difference between the means and the measurement scales employed in the study. furthermore, insignificant results were observed with the other comprehension assessment procedure—the recall test—which did not show such a difference. such an effect of word frequency on the reading comprehension process of beginning arabic readers indicates that students’ reading experience, as reflected by word familiarity, in terms of its frequency, was obviously involved in such a process. this subsequently drew the researcher’s attention to investigate the role of reading experience among the beginning readers, in terms of the type of print exposure those readers were familiar with. that is, the extent to which the familiarity with the print type (text representation) that the beginning arabic readers were exposed to would affect the way they build a mental representation of the text. as was earlier laid out, beginning arabic readers’ performance did not differ according to the type of text representation, in terms of vowelization and diacriticizing. their performance was the same, on average, whether they read a plain text or a vowelized text. however, in order for the researcher to isolate the effectiveness of short vowels per se, a strict sound controlling procedure ought to be followed, even if this procedure would lead the researcher to include unusual reading conditions, in which the texts are presented in a way that beginning arabic readers might not be used to. for example, in one of the reading conditions, the text was supplemented with only short vowels, and no diacritics, such as shaddah or skun, were included. this raised a concern over the manipulation followed in experiment 1 for control purposes: whether the manipulation of short vowels and shaddah (the third reading condition in the design) without the skun diacritic was insufficient to exclude the role of short vowels in the reading process of beginning readers, especially when we know that the term “vowelizing” (which should mean “providing the short vowels versus word familiarity / seraye 495 text with short vowels only”), in a repeated uncontrolled procedure, would add skun (which means void of any short vowels) as part of the vowelization. thus, adding the skun and shaddah diacritics along with the short vowels to the consonants when vowelizing a text would help to assess whether the arab children’s reading process, particularly reading comprehension, is a habitual act that is directed, not only by the presence and absence of short vowels but rather by the diacritics, which represent both the presence and absence of the missing short vowels, the diacritics, skun, and shaddah, from the arabic orthography. in other words, would the reading comprehension process of beginning readers be affected by how the text is printed and exposed to those children, rather than claiming that the effect should be attributed to short vowels alone? therefore, a follow-up experiment in which the vowelization was manipulated differently was necessary in order to respond to such a concern. the experiment added another reading condition in which the text’s consonants were fully vowelized and discretized (to adhere to how previous studies controlled the reading conditions). accordingly, a reading condition in which the text is presented in a familiar form that beginning readers of fourth grade have experienced should help in determining the extent to which reading experience is the operating factor in their reading comprehension process. therefore, a reading condition in which the text was provided with short vowels, shaddah, and skun was constructed and tested. experiment 2 participants. a different sample of 38 fourth graders drawn from exactly the same population in experiment 1 participated voluntarily in this experiment. only the data of 25 participants were analyzed due to students moving or transferring, or attendance circumstances. they were offered r 20 as compensation for their participation. none of them had ever participated in a similar study. materials. the same methodology adopted in experiment 1 was applied in experiment 2, in terms of text formats, measures, design and procedure, with exceptions only in how the two texts, hf and lf, were presented. that is, the same texts were supplemented with short vowels, shaddah, and skun (fully vowelized and diacriticizing). the vowelization and diacriticizing of the two texts were assessed and judged by a team of arabic experts and reading teachers in primary grades (table 5). table 5. reading conditions 6 fully vowelized and diacriticized َّْن ... الس ْيش ه َو َوَعائ لَت ه ف ْي قَْريَة م ْيز يَع يَّة َعْبد اْلعَز ع ْود results. as can be seen in table 6, the participants spent 133.76 seconds on average reading the high-frequency text and 134.56 reading the low-frequency text; note the small difference between the two means (0.8 milliseconds difference). however, incorporating the current data of the fully vowelized and diacriticized reading condition with the previous reading conditions and reanalyzing the data show the following results: there were no significant main effects were found for text type, f(1, 160) = .817, p = .37, or for reading condition f(5, 160) = 0.62, p = .68, nor there was a significant interaction between text type and reading condition, f(5, 160) = 0.79, p = .56. subsequently, the participants’ performance, in terms of their reading time for the two texts, was on average the same. however, observing the marginal means and the mean values for both texts (hf and lf) for all groups together and for group 6 individually shows that the participants, in general, took longer to read the low-frequency text than the high-frequency text (marginal means: 148.62 and 156.24, respectively; these values are after summing up the values for all reading conditions). further, the participants in reading condition 6 (group 6) took less time on average to read both texts, compared with the other groups (133.76 seconds to international electronic journal of elementary education vol.8, issue 3, 481-506, 2016 496 read the high-frequency text and 134.56 to read the low-frequency text; see tables 2 and 6). table 6. cell and marginal means on the reading time test by reading condition and text for group 6 reading condition text high freq. low freq. m sd m sd marginal 6 133.76 51.34 134.56 52.05 134.16 recall test as can be seen in table 7, the participants on average recalled 16.88 propositions in the high-frequency text, and 15.72 propositions in the low-frequency text; note the small difference between the two means and the measurement unit in the scale (only 1.16 propositions difference between the means). however, incorporating this data of the fully vowelized and diacriticized reading condition with the previous reading conditions and reanalyzing the data show the following results: no significant main effects were found for text type, f(1, 156)= 0.03, p= 0.87, or for reading condition, f(5, 156)= 0.29, p= 0.92, nor there was a significant interaction between text type and reading condition, f(5, 156)= 0.40, p= 0.85. subsequently, their recalling was not affected by the type of text they read or the reading condition they were in; their performances on average were the same. the marginal means (table 7) show that the groups in general scored the same (marginal means: 14.87 and 14.77 for hf and lf, respectively; these values are after summing up the values for all reading conditions), with a 0.09 difference, which means a 0.09 unit difference equivalent to a difference of 0.09 propositions. however, the data analysis shows that the participants in group 6 recalled, to some degree, more propositions in both texts than the participants in the other reading conditions (16.88 in the high-frequency text and 15.72 in the low-frequency text); this was particularly noticeable with the low-frequency text. however, there was only a 1.16 unit difference between the two marginal means for the lf and hf texts, which equals a 1.16 unit difference. table 7. cell and marginal means on the recall test by reading condition and text for group 6 reading condition text high freq. low freq. m sd m sd marginal 6 16.88 9.31 15.72 9.43 16.30 short vowels versus word familiarity / seraye 497 the multiple-choice test the mean values of correct answers showed that the participants on average scored higher on the hf text than on the lf text (6.84 for the hf text and 6.48 for the low frequency text); note the very small difference between the two means (the 0.36 difference is equivalent to a difference of 0.36 correct answer points). however, incorporating this data of the fully vowelized and diacriticized reading condition with the previous reading conditions and reanalyzing the data show the following results: a significant main effect was found for text type, f(1, 159) = 19.91, p= .000, p²= 0.111, but not for reading condition f(5, 159)= 0.59, p= .71. further, there was no significant interaction between text type and reading condition, f(5, 159)= 0.76, p= .58. examining the means shows that the participants on average scored higher on the hf text than on the lf text (marginal means: 6.51 and 5.72, respectively; these values are after summing up the values for all reading conditions). note that there was only a 0.79 unit difference between the marginal means for the lf and hf texts, and that the measurement scale involved one point for each correct answer; therefore, a 0.79 unit difference was equivalent to a difference of 0.79 correct answer points. however, conducting a post hoc analysis on every pair using both scheffe and tukey tests did not show any significant differences between the compared pairs. that is, the participants did well on the highfrequency text, regardless of the reading condition they were in and which text they read first. further, generally speaking, the participants in group 6—the reading condition in which the participants read a fully vowelized and diacriticized text—scored higher, to some degree. this was particularly noticeable with the low-frequency text, compared with the other groups (6.84 and 6.48 for the hf and lf texts, respectively), but still with very close means, given the measurement unit of the scale, in which each correct answer was given one point (table 8). the reported partial eta squared value, p², is considered to be a value of a medium effect size (kotrlik & williams, 2003), and indicates that 11 % of the total variance in the dependent variable, number of correct answers, is attributed to word frequency variable. table 8. cell and marginal means on the multiple-choice test by reading condition and text for group 6 reading condition text high freq. low freq. m sd m sd marginal 6 6.84 2.29 6.48 2.14 6.66 general results despite the non-significant results observed over the reading time variable after adding group 6’s data to the analysis, the marginal means between the text, and the pair of means for each group did show noticeable differences in the amount of reading time between the texts (see tables 2 and 6). fourth graders always took more time to read texts in which 10% of its words were of low-frequency. thus, we could say that word frequency affects the reading process of the less experienced and skilled arab readers (beginning readers), as reflected in the reading time it takes them to read a text. further, the supplemented vowelization did not affect their reading process for comprehension, as was measured by the time it took the fourth grader to read the text, or by the number of propositions international electronic journal of elementary education vol.8, issue 3, 481-506, 2016 498 recalled and the correct responses scored. the statistically nonsignificant effect of the reading condition variable on the three dependent variables, reading time, number of propositions recalled, and correct responses scored, was trivial. the partial eta squared value, p², was found to be between 0.011 and 0.008, which means that the reading condition variable accounts for 1% to 0.8 % of the variance of those dependent variables. additionally, the only factor that played a role in the reading process of beginning readers was word frequency. its role was observed with the reading time, and with the reading comprehension, as assessed by the correct responses scored. although its effect might not be practically affected due to the observed sizeable difference between the means and the measurement scales employed in the study, a percentage proportion of 11 to 14 % of variance in the number of correct answers is explained by the manipulated word frequency factor. however, giving beginning arabic readers a plain text, with 10% percentage of its vocabulary of low frequency, did not affect their recalling of the text’s propositions, nor did it affect severely their understanding of the text, as was observed from the close mean values. another observation worth mentioning is that it took the less experienced arabic readers less time to read texts that were both vowelized and diacriticized, as can be observed in group 6, which was asked to read the versions of texts with short vowels, shaddah, and skun together; this result is not in agreement with a previous study by alfahid (2000), which claimed (from examining the reading rates of some individual participants in his study) that adding short vowels and diacritics to the consonants prolongs the reading process of arab adults as they read fully diacriticized text. further, it was noticed that group 6 had smaller differences between the pairs of means (the high and low-frequency texts) for each reading condition (133.76 for hf; 134.56 for lf). can we say that the expected effect of the word frequency in processing a text diminishes when the orthography of the texts is transparent? oppositely, when the orthography of the texts presented is plain or partially presented, does the effect of word frequency become larger? further, the reading process of the low-frequency texts becomes much longer when the orthographies of the texts only present consonants, have only short vowels presented, or have incorrect vowels presented (see table 2). the readers took 173.32 seconds to read the only consonantly presented text; 169.64 seconds to read the texts with only short vowels presented; and 154.92 seconds to read the wrongly vowelized text. therefore, can we say that children’s arabic reading learning and reading experience were accustomed to either plain or fully vowelized and diacriticized texts (sight words), and that they were affected by the experience of their reading exposition to print, which they brought to the reading conditions? this finding is in accordance with a previous study (seraye, 2004) conducted on arab adult readers when reading connected texts, which showed that the only variable that played a role in their reading processing as reflected in the reading time it takes them to read a text was word frequency. however, in terms of reading comprehension, as can be realized by the products of their reading process, their performance in recalling more information from the text and getting the correct responses, showed, on the surface, conflicting results. however, examining the analysis closely resolves such a conflict. the analysis reveals that the reading comprehension of beginning arabic readers was not affected by missing short vowels from text, nor it was affected by the low frequency: their performance, on average, was always the same. this conclusion is more obvious when the beginning arabic readers were asked to recall as many propositions, as they could. however, this was not always the case. in the multiple choice test, the result showed that short vowels versus word familiarity / seraye 499 beginning arabic readers’ reading comprehension was correlated with word frequency, and not with the presence or absence of short vowels. on average, they scored higher with the high-frequency texts. that is, they comprehended high-frequency texts better than the low-frequency ones. however, the differences between every pair of means for all groups were still very small. further, the post hoc analyses conducted on those pairs of means did not reveal any significant differences between the pairs of means. furthermore, even when accepting the small differences found between the means (bowing to the reported large effect size), the average scores of 6.51 for the high-frequency text and 5.72 for the low-frequency one are far from the maximum score of 10 in the multiple-choice test, which indicates that their performance is obviously average, if not quite weak. in conclusion, the current data does not support attributing the high scores in the multiple choice data to the presence of short vowels (even to the short vowels and diacritics in combination). this finding is not in accordance with previous studies, such as abu-rabia’s (1999) and (2001) studies, as well as shimron and sivan (1994) study, which all claimed—except with caution, as did shimron and sivan’s study in their comments on the results—that short vowels have a positive effect on the reading comprehension of both adults and children (as represented by second graders), although they neither manipulated the short vowels to a degree that should isolate their effect, nor did they reinterpret, cautiously, the slight difference found between the means. such a conflict can be explained in terms of the controlling procedures of the reading conditions and the reading texts used, which did not take into account that short vowels and diacritics are different forms that represent different aspects of the orthography, nor did it take into account the involvement of word frequency in text reading difficulty and thus the necessity of having similar texts (as can be achieved by using a matching procedure) in their experimental design. further, even after accepting their experimental design, as mentioned before, the differences between the means reported for the groups that read the vowelized text versus those that read the unvowelized one are very small, once we take the measurement unit into account. in fact, some of the researchers (shimron & sivan, 1994) stated explicitly that the difference was nearly significant, since the p = 0.05. on the other hand, abu-hamour’s (2013) study showed that fifth-grade students, both poor and skilled, performed better with the vowelized text, and the benefit was higher for the skilled readers. the difference between the means was large enough to attribute such a difference to the dependent variable (24.19 for the vowelized text vs. 7.88 for the unvowelized one). however, there were still some questionable observations in the analysis. for example, the skilled students scored 24.19 on average, compared with 7.88; however, the score range was from 18 to 43, an indication that that 24.19 is still far from the maximum score; in fact, it is almost close to the minimum score (18). second, the researchers manipulated the diacritics to be part of the short vowels and considered the fully vowelized text to be supplemented with short vowels and diacritics. in fact, they even considered the case markings to be part of this vowelization. therefore, attributing the positive results to the presence of short vowels is neither accurate nor valid. the researchers did not isolate the role of short vowels accurately. further, the authors used a within-subject design and different texts, and word frequency was not manipulated, despite the fact that word frequency is implicated in the text-reading process (seraye, 2004). general discussion the only variable implicated in the less experienced/skilled readers’ reading process, as represented by fourth graders, was word frequency, which was obvious in the reading time and reading comprehension product data as measured by the number of correct answers. indeed, the effect of word frequency was also found with the more experienced international electronic journal of elementary education vol.8, issue 3, 481-506, 2016 500 readers, as represented by arab adult readers at the graduate and postgraduate academic levels (seraye, 2004). the opaqueness and depth of the texts, when presented without suitable short vowels and diacritics, did not affect the children’s reading process, as reflected in the time load needed to process the texts, nor did it interact with the type of text they read, in terms of word frequency. rahbari and senechal (2009) arrived at the conclusion that “it is not the orthographic depth of a language, but it is the reading skills, or more precisely the experience with reading words, as well as a task demand that affect processes used to read” (p. 523). previous literature, in general, has demonstrated that sensitivity to word frequency was found in tasks such as word naming and word recognition. high-frequency words are read more quickly than low-frequency words, which has been a consistent finding for different orthographies that both differ from and resemble arabic orthography (seraye, 2004, for arabic; raman & baluch for turkish, 2001; baluch, 1996, for persian). in fact, well-established research on eye movement has shown that the fixation durations for high-frequency words while reading a sentence or passage are shorter than the fixation durations for low-frequency words; this was documented for both adults and children readers (e.g., joseph, nation, & liversedge, 2013; rayner et al., 2006; rayner & duffy, 1986; rayner, sereno, & raney, 1996; sereno & rayner, 2000). similarly, predictable words take short duration times to fixate, compared to their non-predictable counterparts (e.g., balota, pollatsek, & rayner, 1985; miellet, sparrow, & sereno, 2007; rayner & well, 1996). thus, these higher time fixations for low-frequency words are assumed to add more time to the process of reading the text and subsequently prolong its reading time process. this effect was more salient due to the controlling procedure for word frequency as a variable adopted in the current study by intentionally inserting low-frequency counterpart words in one of the versions of the text. therefore, we can say that reader’s experience, as reflected in his/her lexical representation of the word—word familiarity, specifically—plays a major role in the reading process of the less experienced arabic readers, as it did with the more experienced arabic readers (e.g., seraye, 2004). such an effect should not be surprising, since individual words and their characteristics were found to play a central role in the reading process of different orthographies (see for example the three experimental studies by seraye, 2004, for arabic; and rahbari and senechal, 2009, for persian). the ease and speed in accessing the mental lexicon from memory during reading affects the reader’s reading fluency and comprehension (perfetti, 2007). this finding can be justified by the explanation given by the so called restrictedinteractive theory proposed by perfetti (1994), which presents—in the researcher’s opinion—a suitable framework for explaining the effects of the reader’s experience, as reflected in word familiarity, on the reading processing of texts. perfetti (1994) stated that “learning to read is the acquisition of increasing numbers of orthographically addressable words (quantity acquisition) and the alteration of individual representations along quality dimensions: specificity and redundancy” (p. 857). his theory is based on two principles: specificity (an increase in the number of position-correct specific letters in a representation) and redundancy (the increasing establishment of redundant phonemic representations), which would lead to a quality word representation in a reader's mind and subsequently to a resource-cheap reading. as noted by perfetti (1994), “[a]s individual words become fully specified and redundant, they move from the functional lexicon, which allows reading, to the autonomous lexicon, which allows resource-cheap short vowels versus word familiarity / seraye 501 reading” (p. 857). in macleod and kampe’s (1996) words, “automaticity is a direct function of experience” (p. 132). however, the effect of word frequency on the reading process was found to be equally present, regardless of the boundary of exposure to print, whether limited, as was the case with fourth graders, versus unlimited, as was the case with skilled adult readers at the undergraduate and graduate academic levels. for reading comprehension, as assessed by the two tests measuring the number of propositions and the correct answers, the absence and presence of short vowels and diacritics did not implicate the reading comprehension process of the less experienced readers, as represented by the fourth graders. further, their reading comprehension, although found to be significantly correlated with word frequency, was not severely affected, as can be seen from the closeness of the mean values. such a finding can be attributed to the fact that beginning arabic readers use their knowledge of the morphological aspects of the language in comprehending the text; indeed, they exploit their knowledge of the trilateral/quadrilateral root of arabic words that is needed to comprehend the text. a study by badry (1982) arrived at the conclusion that arab moroccan children from ages 3 to 6 “are aware of the underlying morphological root in their spoken language, and this awareness was reflected in the production stage of their acquisition” (seraye, 2004, p. 83). further, in their comparison study between dyslexics and normal readers in sixth and eighth grades, abu rabia and abu-rahmoun (2012) reached to the conclusion that, while reading, both dyslexic and normal readers rely heavily on the morphological aspects of the arabic language—particularly the root. this reliance on the roots of arabic words appears more clearly with the reading materials that were presented unvowelized, particularly observed with the normal arab readers (abu rabia & abu-rahmoun, 2012). their conclusive statement is that, “[r]oots of words are the key to initial lexical access” (abu rabia & abu-rahmoun, 2012, p. 1265). furthermore, well-documented research in languages with alphabetic-based writing systems revealed that children in fifth grade and up employ morphological information to infer the meanings of unfamiliar and low-frequency words while reading (mccutchen & logan, 2011). therefore, by combining the findings in the current investigation with those from previous research, we can reach a conclusive statement: in the absence of short vowels and diacritics from print, beginning readers still can compensate for such an absence by relying on their linguistic knowledge, particularly the morphological roots of arabic words, and the textual context to understand the text. limitations the current study’s findings should be restricted to only fourth graders, particularly male fourth graders population, whose reading levels are at the fourth-grade instruction reading level, and should not be generalized to fourth graders in general. further, the interpretation should be restricted to students who have mastered the reading skill and just started to read for learning across the curriculum. therefore, the roles of vowelization (short vowels plus diacritics) and word frequency in the reading process of less-skilled fourth graders should be assessed for two reasons: to help determine the degree to which exposing such readers to plain reading materials would affect their reading comprehension, and subsequently, whether they would be able to compensate for such an effect by exploiting their knowledge of the morphological roots of arabic language. additional caution should be taken into account, regarding the finding of no explicit additional benefits regarding the presence of short vowels and diacritics for the reading comprehension of arabic readers, both experienced and less experienced readers. the international electronic journal of elementary education vol.8, issue 3, 481-506, 2016 502 benefits should be interpreted in the context of silent-mode reading tasks and not be extended to other reading modes such as reading accuracy tasks. recommendations the current study targeted beginning readers, as represented by skilled fourth-grade male students; thus, replicating the same procedure for non-skilled readers is recommended. further, due to the segregated nature of the education system in saudi arabia, replicating the same procedure for female fourth graders of skilled and non-skilled reading levels is encouraged. additionally, according to my aforementioned explanation, the small subscripts and superscripts in written arabic words do not represent only short vowels, but also diacritics (such as skun, shaddah, maddah) as well as case ending markers that take the shape of short vowels and skun. therefore, the term “fully vowelized”, as reported in the current and previous studies, needs to be given a new meaning when investigating its effect. that is, the role of short vowels needs not be investigated separately from the other diacritics included in the fully vowelized reading condition. • • • references abu-hamour, b., al-hmouz, h., & kenana, m. (2013). the effect of short vowelization on curriculumbased measurement of reading fluency and comprehension in arabic. australian journal of learning difficulties, 18, 181-197, doi:10.1080/19404158.2013.852980. abu-rabia, s. (1995). learning to read in arabic: reading, syntactic, orthographic, and working memory skills in normally achieving and poor arabic readers. reading psychology: an international quarterly, 16, 351-394, doi: 10.1080/0270271950160401. abu-rabia, s., & siegel, l. (1995). different orthographies different context effects: the effects of arabic sentence context in skilled and poor readers. reading psychology: an international quarterly, 16, 1-19, doi: 10.1080/0270271950160101. abu-rabia, s. (1996). the role of vowels and context in the reading of highly skilled native arabic readers. journal of psycholinguistic research, 25, 629-641, doi: 10.1007/bf01712413. abu-rabia, s. (1997a). the need for cross-cultural considerations in reading theory: the effects of arabic sentence context in skilled and poor readers. journal of research in reading, 20, 137-147, doi: 10.1111/1467-9817.00026. abu-rabia, s. (1997b). reading in arabic orthography: the effect of vowels and context on reading accuracy of poor and skilled native arabic readers in reading paragraphs, sentences, and isolated words. journal of psycholinguistic research, 26, 465-482, doi: 10.1023/a:1025034220924. abu-rabia, s. (1997c). reading in arabic orthography: the effect of vowels and context on reading accuracy of poor and skilled native arabic readers. reading and writing: an interdisciplinary journal, 9, 65-78, doi: 10.1023/a:1007962408827. abu-rabia, s. (1998). reading arabic texts: effects of text type, reader type and vowelization. reading and writing: an interdisciplinary journal, 10, 105-119, doi: 10.1023/a:1007906222227. abu-rabia, s. (1999). the effect of arabic vowels on the reading comprehension of secondand sixth-grade native arab children. journal of psycholinguistic research, 28, 93-101, doi: 10.1023/a:1023291620997. abu-rabia, s. (2001). the role of vowels in reading semitic scripts: data from arabic and hebrew. reading and writing: an interdisciplinary journal, 14, 39-59, doi: 10.1023/a:1008147606320. short vowels versus word familiarity / seraye 503 abu-rabia, s. (2002). reading in a root-based morphology language: the case of arabic. journal of research in reading, 25, 299-309, doi: 10.1111/1467-9817.00177. abu-rabia, s., & abu-rahmoun, n. (2012). the role of phonology and morphology in the development of basic reading skills of dyslexic and normal native arabic readers. creative education, 3, 1259-1268, doi: 10.4236/ce.2012.37185. al-fahd, j. m. (2000). the goodman psycholinguistic model of english reading and its applicability to semitic languages. unpublished doctoral dissertation, university of arizona, arizona, tucson. badry, f. (1982). the centrality of root in semitic lexical derivation: evidence from children’s acquisition of moroccan arabic (papers and reports on child language development, no. 21. 915). berkeley, california: stanford university. (eric document reproduction service no. ed 222046). balota, d. a., pollatsek, a., & rayner, k. (1985). the interaction of contextual constraints and parafoveal visual information in reading. cognitive psychology, 17, 364–390, doi:10.1016/00100285(85)90013-1. baluch, b. (1996). word frequency effects in naming in experienced and previously experienced adult readers of persian. reading and writing: an interdisciplinary journal , 8, doi: 433-441, 10.1007/bf00404004. campbell, g. (1997). handbook of scripts and alphabets. london: routledge. bransford & johnson chitiri, h. (1991). the influence of language and writing system characteristics on the reading process. unpublished dissertation, university of toronto, toronto. ibrahim, r. (2013). reading in arabic: new evidence for the role of vowel signs. creative education, 4, 248-253, doi: 10.4236/ce.2013.44036. ibrahim, r., eviatar, z., & aharon peretz, j. (2002). the characteristics of the arabic orthography slow its cognitive processing. neuropsycholgy, 16, 322-326, doi: 10.1037/0894-4105.16.3.322. joseph, h., nation, k., & liversedge, s. (2013). using eye movement to investigate word frequency effects in children’s sentence reading. school psychology review, 41(2), 207-222. kardash, c.m. & howell, k.l. (2000). effects of epistemological beliefs and topic-specific beliefs on undergraduates’ cognitive and strategic processing of dual-positional text. journal of educational psychology, 92, 524–535, doi:10.1037/0022-0663.92.3.524. kendeou, p., muis, k., & fulton, s. (2010). reader and text factors in reading comprehension processes. journal of research in reading, 1-19, doi: 10.1111/j.1467-9817.2010.01436.x. kendeou, p., rapp, d., & van den broek, p. (2004). the influence of reader’s prior knowledge on text comprehension and learning from text. in r. nata (ed.), progress in education, (vol. 13, pp. 189209). new york: nova science publishers inc. kendeou, p. & van den broek, p. (2007). the effects of prior knowledge and text structure on comprehension processes during reading of scientific texts. memory and cognition, 35(7), 1567– 1577. kientch, w. (1988). the role of knowledge in discourse comprehension: a construction-integration model. psychological review, 95, 163–182, doi:org/10.1037/0033-295x.95.2.163. kirk, r. e. (1982). experimental design: procedures for the behavior sciences. pacific grove, ca: brook/cole publishing company. kotrlik, j. w. & williams, h. a. (2003). the incorporation of effect size in information technology, learning, and performance research. information technology, learning, and performance journal, 21(1), 1-7. lipson, m. y., & wixson, k. k. (1997). assessment & instruction of reading and writing disability: an interactive approach. ny: longman. http://dx.doi.org/10.4236/ce.2012.37185 http://dx.doi.org/10.1016/0010-0285(85)90013-1 http://dx.doi.org/10.1016/0010-0285(85)90013-1 international electronic journal of elementary education vol.8, issue 3, 481-506, 2016 504 mahmoud, y. (1979). the arabic writing system and the sociolinguistics of orthographic reform. unpublished doctoral dissertation, georgetown university, washington, d.c. mahmoud, y. (1979). on the reform of the arabic writing system. journal of reading, 23, 727-729. macleod, c. m., & kampe, k. e. (1996). word frequency effects on recall, recognition, and word fragment completion tests. journal of experimental psychology: learning, memory, and cognition, 22(1), 132-142. mccutchen, d., & logan, b. (2011). inside incidental word learning: children’s strategies use of morphological information to infer word meanings. reading research quarterly, 46, 334-349, doi: 10.1002/rrq.003. meyer, b.j.f. (1975). the organization of prose and its effects on memory. amsterdam: north-holland publishing company. meyer, b.j.f. & freedle, r.o. (1984). effects of discourse type on recall. american educational research journal, 21, 121–143, doi: 10.3102/00028312021001121. miellet, s., sparrow, l., & sereno, s. (2007). word frequency and predictability effects in reading french: an evaluation of the e-z reader model. psychonomic bulletin & review, 14, 762-769, doi: 10.3758/bf03196834. perfetti, c. (1994). psycholinguistics and reading ability. in m. a. gernsbacher (ed.), handbook of psycholinguistics (pp. 849-894). san diego: academic press. perfetti, c. (1999). comprehending written language: a blueprint of the reader. in p. hagoort & c. brown (eds.), the neurocognition of language (pp. 167-208). oxford university press. perfetti, c. (2007). reading ability: lexical quality to comprehension. scientific studies of reading, 11, 357-383, doi: 10.1080/10888430701530730. rahbari, n., & senechal, m. (2009). lexical and nonlexical processes in the skilled reading and spelling of persian. reading and writing: an interdisciplinary journal, 22, 511-530, doi: 10.1007/s11145-008-9122-1. raman, i., & baluch, b. (2001). semantic effects as a function of reading skill in word naming of a transparent orthography. reading and writing: an interdisciplinary journal, 14, 599–614, doi: 10.1023/a:1012004729180. ryder, r. j., & hughes, m. (1985). the effect on text comprehension of word frequency. journal of educational research, 78, 286-291, doi: 10.1080/00220671.1985.10885617. rayner, k., & duffy, s. a. (1986). lexical complexity and fixation times in reading: effects of word frequency, verb complexity, and lexical ambiguity. memory & cognition, 14, 191–201, doi: 10.3758/bf03197692. rayner, k., reichle, e., stroud, m., williams, c., & pollatsek, a. (2006). the effect of word frequency, word predictability, and font difficulty on the eye movements of young and older readers. psychology and aging, 21, 448-465, doi: 10.1037/0882-7974.21.3.448. rayner, k., & sereno, s. (1994). eye movements in reading: psycholinguistic studies. in m. a. gernsbacher (ed.), handbook of psycholinguistics (pp. 57-81). san diego: academic press. rayner, k., sereno, s. c., & raney, g. e. (1996). eye movement control in reading: a comparison of two types of models. journal of experimental psychology: human perception & performance, 22, 1188-1200, doi: 10.1037/0096-1523.22.5.1188. rayner, k., & well, a. d. (1996). effects of contextual constraint on eye movements in reading: a further examination. psychonomic bulletin & review, 3, 504–509, doi: 10.3758/bf03214555. rayner, k., well, a., pollatsek, a., & bertera, j. (1982). the availability of useful information to the right of fixation in reading. perception & psychophysics, 31, 537-550, doi: 10.3758/bf03204186. short vowels versus word familiarity / seraye 505 seraye, a. m. (2004). the role of short vowels and context in the reading of arabic, comprehension, and word recognition of highly skilled reader. unpublished doctoral dissertation, university of pittsburgh, pennsylvania, pittsburgh. sereno, s.c., & rayner, k. (2000). spelling-sound regularity effects on eye fixations in reading. perception & psychophysics, 62, 402-409, doi: 10.3758/bf03205559. shimron, j. (1993). the role of vowels in reading: a review of studies of english and hebrew. psychological bulletin, 114, 52-67, doi: 10.1037/0033-2909.114.1.52. shimron, j., & sivan, t. (1994). reading proficiency and orthography: evidence form hebrew and english. language learning, 44, 5-27, doi: 10.1111/j.1467-1770.1994.tb01447.x. zwaan, r.a. & brown, c.m. (1996). the influence of language proficiency and comprehension skill on situation-model construction. discourse processes, 21, 289–327, doi: 10.1080/01638539609544960. http://psycnet.apa.org/doi/10.1037/0033-2909.114.1.52 international electronic journal of elementary education vol.8, issue 3, 481-506, 2016 506 www.iejee.com this page is intentionally left blank http://www.iejee.com/ international electronic journal of elementary education, 2013, 5(2), 109-124. issn:1307-9298 copyright © iejee www.iejee.com student academic performance outcomes of a classroom physical activity intervention: a pilot study heather erwin  university of kentucky, usa alicia fedewa university of kentucky, usa soyeon ahn university of miami, usa received: 4 june 2012 / revised: 15 november 2012 / accepted: 11 february 2013 abstract physical activity is beneficial to children’s health, yet academic pressures limit opportunities for students throughout the school day. the purpose of this study was to determine the effect of a classroom pa intervention on student academic performance outcomes. intervention participants (n=15) received daily pa breaks. reading and mathematics fluency, pa, grades, and standardized test scores were collected. effects of the intervention were examined using mixed-design anovas. intervention students had significantly higher reading fluency and mathematics scores postintervention and higher means for standardized reading and mathematics scores as well as grades. short bouts of pa are important for improving cbm math and reading fluency scores. classroom teachers should be encouraged to devote time during academic learning to incorporate pa. keywords: curricular intervention, academic achievement, child health, curriculum-based measurement introduction throughout the last three decades, children have become increasingly more sedentary given the changes in our modernized environment (centers for disease control and prevention [cdc], 2009; stevens, to, stevenson & lochbaum, 2008). schools have been identified as  heather erwin, university of kentucky, heather erwin, ph.d. dept. of kinesiology & health promotion university of kentucky 115 seaton center lexington, ky 40506-0219 office: (859)-257-5311 fax: (859)-323-1090 http://www.iejee.com/ student academic performance outcomes of a classroom physical activity intervention / erwin, fedewa & ahn 110 locations in which physical activity (pa) promotion should occur (pate, davis, robinson, stone, mckenzie, & young, 2006). no child left behind legislation has led to budget cuts and increased pressure for schools to increase standardized test scores, thereby leaving schools to reduce or even eliminate programs that could enhance pa in children (chomitz, slining, mcgowan, mitchell, dawson, & hacker, 2009; coe, pivarnik, womack, reeves, & malina, 2006; sibley & etnier, 2003). during school hours, the decrease of pa through limited time spent in physical education class or recess breaks contributes to the significant increase of sedentary behaviors in children. fewer children walk or ride their bicycles to school, and pa is increasingly being replaced with television watching, time spent on the internet, and the ubiquitous playing of video games (cdc, 2009; stevens et al., 2008; world health organization [who], 2009). experts recommend that children engage in 60 minutes or more of moderate to vigorous pa per day (strong, malina, blimkie, daniels, dishman, gutin…trudeau, 2005), yet studies have found that only 42% of children ages 6-11 years obtain this goal (troiano, berrigan, dodd, masse, tilert, & mcdowell, 2008). when addressing health outcomes, typically the physical benefits are discussed; however, participating in physical activities has also shown a significant and positive effect on children’s cognitive functioning (fedewa & ahn, 2011; trudeau & shephard, 2010) and academic outcomes, with no detrimental effects to learning when time is taken away from instruction (sibley & etnier, 2003). researchers theorize that children receive cognitive benefits from participating in pa through a number of mediating processes (basch, 2010; trudeau & shephard, 2010). in a review of the literature, trudeau and shephard (2010) identified physiological influences such as greater arousal and enhanced levels of neurotrophins that stimulate neural connections in the hippocampus or learning center of children’s brains. further, additional psychosocial influences were also found in the literature, including an increased level of self-esteem and connectedness in schools, likely enhancing children’s ability to learn (trudeau & shephard, 2010). research attempting to identify the mediating relationships between children’s levels of pa and cognitive outcomes are limited by methodology employed in most of the studies (see fedewa & ahn, 2011), and thus the specific causal pathways between pa and children’s cognitions have yet to be identified. to date, most of the research examining the academic and cognitive effects of children’s pa has been measured through traditional, standardized tests or grades. although helpful in assessing the long-term effects of pa interventions on children’s cognitive outcomes, these traditional measures are not useful in assessing short-term gains or improvement as a result of the intervention (bricker, yovanoff, capt, & allen, 2003; pretti-frontczak, 2002). given that pa interventions are not typically implemented over long durations of time (i.e., greater than one academic year), it is likely that effects of these interventions may be missed due to the measurements used to assess academic or cognitive gains (see macy, bricker, & squires, 2005). curriculum-based measurements (cbms) one way of assessing academic gains over short periods of time is through the use of cbms. cbms are research-based assessments used in schools to ascertain student achievement on basic skills such as reading, math, writing, or spelling. in response to the limitations of traditional, standardized tests, cbms were developed in the 1970s as a means of monitoring children’s response to an intervention (see reschly, busch, betts, deno & long, 2009). these measures are well known and utilized by many teachers, school psychologists and other school personnel, as they are sensitive to small growth over time, are inexpensive, and translate into targeted goals for student achievement (macy et al., 2005; reschly et al., 2009). in a recent meta-analysis, reschly and colleagues (2009) were able to demonstrate the strong international electronic journal of elementary education vol.5, issue 2, 109-124,2013 111 predictive validity (r = .67) of a particular type of cbm — oral reading fluency measures — on children’s future reading achievement and high-stakes standardized assessments. the cumulative evidence over the past three decades has been remarkable for these measures given the relatively minimal resources in terms of cost and administration time. as pressures for high stakes testing increase and the time children spend engaged in pa decreases, considerable evidence is needed to demonstrate the effectiveness of classroombased interventions that promote pa during the school day. yet despite this need, there is a dearth of research assessing the effectiveness of classroom-based pa interventions on children’s learning outcomes. these types of interventions have, however, been shown to significantly increase student pa levels and intensity in the classroom (cardon, de clercq, de bourdeaudhuij, & breithecker, 2004; erwin, abel, beighle, & beets, 2009; erwin, beighle, morgan, & noland, 2011; gibson, smith, dubose, greene, bailey, williams…donnelly, 2008; liu, hu, ma, cui, pan, chang, et al., 2007; mahar, murphy, rowe, golden, shields, & raedeke, 2006; stewart, dennison, kohl, & doyle, 2004), as well as result in enhanced health outcomes such as improved bmi (liu et al., 2007), decreased back/neck pain (cardon et al., 2004), increased bone strength (macdonald, kontulainen, khan, & mckay, 2007; macdonald, kontulainen, beck, khan, & mckay, 2008), and noise reduction in the classroom. all of these positive outcomes result in an increased ability to concentrate (norlander, moas, & archer, 2005). in the handful of studies assessing the impact of classroom-based pa on children’s academic performance, a number of benefits have been found. in particular, students have improved their behaviors (maeda & randall, 2003; mahar et al., 2006), concentration (lowden, powney, davidson, & james, 2001; norlander et al., 2005), recognition and memory (della valle, dunn, dunn, geisert, sinatra, & zenhausern, 1986), and reading and mathematical skills (fredericks, kokot, & krog, 2006; uhrich & swalm, 2007) from physical activities performed in the classroom setting. embedded within the need to establish effective pa classroom interventions are measures that are sensitive to incremental changes in students’ academic growth. cbms will not only allow for progress monitoring but also assesses students on content in which they are being exposed through their instruction. by using measures that detect small changes in academic growth, it may be possible to more accurately detect whether pa is exerting a positive effect on children’s rate of learning or ability to retain material. thus, the purpose of the current pilot study was twofold. first, the study aimed to evaluate whether implementing curricular pa positively influenced children’s reading and mathematics achievement. second, the relationship of cbms with other standardized measures and grades used in assessing children’s reading and mathematics achievement will be measured in order to examine its potential for further use as an academic assessment tool in monitoring the effectiveness of pa interventions. because cbms have not been used before as a tool for measuring the impact of curricular pa on children’s academic outcomes, the present study serves as a pilot in investigating these questions. student academic performance outcomes of a classroom physical activity intervention / erwin, fedewa & ahn 112 methods participants participants included 29 3rd grade students (mage = 8.87, sd = .54) from one southeastern elementary school (two classrooms). students were assigned to intervention (n = 16) and control (n = 13) conditions via a quasi-experimental design (by homeroom class) over a 20week intervention period. one classroom served as the treatment, while another classroom served as the control. procedures were approved by the lead author’s institutional review board, and all parents/guardians signed an informed consent form, while all child participants completed an assent form to participate. instruments measurements of reading fluency, mathematics aptitude, grades, standardized test scores, classroom behavior, and school day pa were collected for all participants. two cbms: reading and mathematics fluency. specifically, curriculum-based reading fluency and mathematics measures are short progress measures designed to assess children’s reading and mathematical fluency (stecker & lembke, 2005). criterion validity coefficients for curriculum-based measurements are .80-.90 for reading and .between .60-.80 for mathematics (foegen, jiban, & deno, 2007; jitendra, sczesniak, & deatline-buchman, 2005). the oral reading fluency measures consisted of three reading passages wherein the child would read aloud for one minute, with the examiner recording the number of words correctly read for each passage. the median score out of the three reading passages was used for the child’s oral reading fluency score at each of the three time points. for mathematical fluency, grade-appropriate mathematical problems consisting of addition, subtraction, and basic multiplication were given on a classwide level to the students every two weeks. the students were given one minute to complete as many problems as they could with the number of correct responses used as their mathematical fluency score for each of the three time points. the psychometric properties of these instruments are described in the results section. grades. each classroom teacher also provided student grades for reading and mathematics at each of the three designated time points throughout the school year (december 2009, march 2010, may 2010). these were recorded as percentages (out of 100). standardized test scores. a number of different standardized tests were administered at different points throughout the school year. at the beginning and end of the school year, students took the test of primary reading outcomes (t-pro), which assesses phonics, vocabulary, comprehension, and research skills, as well as standardized testing and reporting (star) reading tests (r=0.93; http://www.cde.ca.gov/ta/tg/sr/technicalrpts.asp), which coincide with the accelerated reader program. at three time points (august 2009, december 2009, and march 2010), the students completed the discovery education assessment which assesses reading/language arts and mathematics. the outputs rate the students at levels, which are determined by the number of correct responses. these levels were recorded as: novice = 1, apprentice = 2, proficient = 3, and distinguished = 4. physical activity. to measure school day pa, participants wore a pedometer (walk4life, ls 2500, plainfield, il) for five consecutive school days, which is consistent with recommendations of monitoring periods for this age of children (vincent & pangrazi, 2002). this pedometer brand and model has been found to produce reliable and valid scores when used with children (beets, patton, & edwards, 2005). http://www.cde.ca.gov/ta/tg/sr/technicalrpts.asp international electronic journal of elementary education vol.5, issue 2, 109-124,2013 113 procedures curriculum-based reading and mathematics fluency. during the baseline week (september, 2010), trained researchers administered the reading fluency probes and each classroom teacher administered the mathematics assessments for all students. the same procedures were followed once every two weeks using different forms (alternate passages and worksheets validated for the purposes of alternate use) of the reading and math standardized assessments designed to measure small progress over time (stecker & lembke, 2005). physical activity. to prevent reactivity with the pedometers, participants were given the opportunity to handle the pedometer, open it, and practice applying and removing it from their waistband prior to data collection. on the first day of data collection, each participant was assigned a pedometer to be used for the duration of the study. upon entering the classroom first thing in the morning, students were instructed to wear the pedometer on their waistband for the entire school day. immediately prior to dismissal, students returned their pedometer to the assigned bin. their data were recorded on a data sheet and reset for use the next day. this occurred during five days of baseline and one random day per week during the intervention. the classroom teacher of the intervention group led pa breaks for 20+ minutes per day. she maintained a log of all pa breaks she provided including the name and nature of the pa break as well as the duration and time period. each integrated pa break related to the math and reading content that was currently being taught. she participated in a 30-minute classroom pa training provided by an expert in classroom-based pa. the training took place prior to baseline data collection. during the training, the definition of pa, the importance of pa in the classroom, and the connection between pa and academic performance were presented. additionally, managing children in pa settings and instructional means for presenting activity breaks to the students were emphasized. the intervention teacher was provided with promoting physical activity and health in the classroom activity break cards (pangrazi, beighle, & pangrazi, 2009) and other web resources for classroom physical activities (i.e., energizers, pe central). in addition to the training and resources, the year prior to implementation of the intervention, the intervention classroom teacher took two graduate courses related to pa promotion with youth and teaching effectiveness in pa settings. the courses each addressed classroom pa breaks. the classroom teacher of the control group did not provide these pa breaks to her students. in lieu of the pa breaks, students in the control group continued with traditional, inseat learning of the content. this included teacher-directed instruction, individual student seatwork, and partner or group work at desks. all students had the same amount of time allotted for physical education (two 30-minute classes per week) and recess (one 30-minute session per day). data analysis the validity of curriculum-based measurement (research question 1) was addressed by examining the extent to which a particular test (i.e., cbm) correlates with previously validated measures (i.e., standardized test scores and teacher-reported grades). therefore, scores from cbm’s, standardized test scores, and teacher reported grades were correlated and compared separately for reading and mathematics achievement. of nine repeated cbm measures, only scores at baseline, time 5, and time 8—which were collected at the same time points as standardized test scores and teacher grades of mathematics and reading—were correlated with the other two measures such that differences in the number of repeated measures were student academic performance outcomes of a classroom physical activity intervention / erwin, fedewa & ahn 114 controlled and further students’ performance on different measures were compared concurrently. next, the intervention effects of pa on mathematical and reading performances (research question 2) were examined, using a series of mixed-design anovas. to control for differences in the number of repeated scores, the authors chose three cbm scores at baseline, time 5, and time 8. therefore, for reading and mathematics achievement, two sets of mixeddesign anova—using time and measures as within-subject factors and the type of intervention as a between-subject factor—were performed. results validity of curriculum-based measurement the concurrent validity of the curriculum-based measurement was evaluated based on intercorrelations among all three measures (i.e., cbm, standardized test scores, and teacher ratings of students’ grades) on reading and mathematics, separately. table 1 and table 2 show correlations among scores from cbm, standardized test scores, and teachers’ reported grades for the control group in the upper diagonal of the matrix and for the treatment group in the lower diagonal of the correlation matrix for mathematics and reading, respectively. international electronic journal of elementary education vol.5, issue 2, 109-124,2013 115 table 1. correlations among cbm, standardized test scores, and grades on mathematics cbm time1 cbm time2 cbm time3 test time1 test time2 test time3 grades time1 grades time2 grades time3 cbm: time1 1 .359 .843** .423 .585* .581* .420 .216 .544* cbm: time2 .895** 1 .424 .311 .526 .165 .019 -.254 .167 cbm: time3 .883** .753** 1 .483 .548* .280 .505 .107 .666** test: time1 .544 .464 .579* 1 .782** .352 .002 .582* .394 test: time2 .646* .685* .690* .686* 1 .402 .463 -.013 .632* test: time3 .741** .842** .722** .561 .851** 1 -.277 .337 .067 grades: time1 .370 .306 .130 .516 .219 .327 1 .070 .670** grades: time2 .386 .422 .273 .661* .708** .498 .508 1 .011 grades: time3 .445 .235 .379 .697** .488 .336 .647* .408 1 note. ** p < .01; * p < .05 student academic performance outcomes of a classroom physical activity intervention / erwin, fedewa & ahn 116 table 2. correlations among cbm, standardized test scores, and grades on reading cbm time1 cbm time2 cbm time3 test time1 test time2 test time3 grades time1 grades time2 grades time3 cbm: time1 1 .359 .843** .423 .585* .581* .420 .216 .544* cbm: time2 .895** 1 .424 .311 .526 .165 .019 -.254 .167 cbm: time3 .883** .753** 1 .483 .548* .280 .505 .107 .666** test: time1 .544 .464 .579* 1 .782** .352 .002 .582* .394 test: time2 .646* .685* .690* .686* 1 .402 .463 -.013 .632* test: time3 .741** .842** .722** .561 .851** 1 -.277 .337 .067 grades: time1 .370 .306 .130 .516 .219 .327 1 .070 .670** grades: time2 .386 .422 .273 .661* .708** .498 .508 1 .011 grades: time3 .445 .235 .379 .697** .488 .336 .647* .408 1 note. ** p < .01; * p < .05 international electronic journal of elementary education vol.5, issue 2, 109-124,2013 117 as shown in the shaded areas of table 1, mathematics scores from cbm had small to large correlations with standardized test scores on mathematics for both control and intervention groups. however, the correlations between cbm scores and grades were small and insignificant. as shown in the shaded areas of table 2, reading scores from cbm, standardized scores, and grades were correlated with a small to large magnitude. patterns of correlations among three measures on reading were similar between intervention and control groups, showing lower correlations between cbm scores and grades, yet higher correlations between cbm scores and standardized test scores. intervention effect on mathematics achievement a preliminary analysis was first performed to determine whether any preexisting differences on mathematics scores existed between control and intervention groups. results from three sets of independent t-tests showed that the intervention group was not statistically different from the control group on cbm scores (t(27) = -.87, p = .39), standardized test scores (t(25) = .24, p = .81), or teacher’s reporting of students’ grades (t(25)=-2.52, p = .05), indicating no statistically significant pre-existing differences at the baseline measures of mathematics between the two groups. mauchly’s tests indicated that the assumption of sphericity was violated for the main effects of measure, χ2(2) = 10.94, p = .004, and interaction effect between measure and time, χ2(9) = 23.58, p = .005, but not for the main effect of time (χ2(2) = 2.43, p = .30). therefore, degrees of freedom were corrected using the huynh-feldt estimates of sphericity (gamst, myers, & guarino, 2008) for measure (ε = .78) and interaction between measure and time (ε = .77). as shown in table 3, a mixed-design anova showed that the main effects of time (f(2, 44) = 15.52, p < .01, partial 2  = .41), measures (f(1.56, 34.36) = 2716.32, p < .01, partial 2  =.99), and intervention (f(1, 22) = 7.49, p = .01, partial 2  =.25) were statistically significant. further, two-way interactions between time and measure (f(3.08, 67.84) = 8.67, p < .01, partial 2  = .28) and three-way interactions among time, measure, and intervention (f(3.08, 67.84) = 6.49, p < .01, partial 2  = .23) were statistically significant. because a higher-order interaction supersedes lower-order effects (gamst, myers, & guarino, 2008), follow-up tests were performed to further investigate the three-way interaction among time, measure, and intervention in detail. tests of simple effects showed a significant two-way interaction effect between time and intervention for cbm scores (f(2,26) = 10.31, p <.01), but not for standardized test scores (f (2,21) = 2.63, p = .10) or teachers’ reported grades (f(2,23) = 1.59, p = .23) . as shown in figure 1, the intervention group (m = 24.56, sd = 2.21) scored significantly higher on cbm scores than the control group (m = 13.69, sd = 2.45) at time 3 (mdiff = 10.87, p = .003), but not time 1 (mdiff = 2.75, p = .39) or time 2 (mdiff = 2.16, p = .49). table 3. results from mixed-design anova on mathematics source ss df ms f p partial 2  time 563.18 2 281.59 15.52 <.01 .41 time * intervention 76.95 2 38.48 2.12 .13 .09 error (time) 798.31 44 18.14 measure 330133.93 1.56 211367.38 2716.32 <.01 .99 student academic performance outcomes of a classroom physical activity intervention / erwin, fedewa & ahn 118 measure * intervention 341.37 1.56 218.56 2.81 .09 .11 table 3 (continue). results from mixed-design anova on mathematics source ss df ms f p partial 2  error (measure) 2673.81 34.36 77.81 time * measure 432.63 3.08 140.30 8.67 <.01 .28 time * measure * intervention 323.74 3.08 104.99 6.49 <.01 .23 error (time * measure) 1097.19 67.84 16.17 intervention 872.02 1 872.02 7.49 .01 0.25 error 2562.85 22 116.49 figure 1. mathematics across time by intervention groups intervention effect on reading achievement results from three sets of an independent t-test indicated no pre-existing differences between control and intervention groups on all three measures of reading achievement, t(27) = -1.48, p = .15 for the cbm scores, t(26) = -.97, p = .34 for standardized test scores, or t(25) = 1.39, p = .18 teacher’s rating of students’ grades. mauchly’s tests indicated that the assumption of sphericity had been violated for the main effects of measure, χ2(2) = 80.06, p < .01, and interaction effect between measure and time, χ2(9) = 48.35, p < .01, but not for the main effect of time (χ2(2) = .22, p = .90). therefore, degrees of freedom were corrected using the huynh-feldt estimates of sphericity (gamst, cbm standardized test scores teachers’ reported grades international electronic journal of elementary education vol.5, issue 2, 109-124,2013 119 myers, & guarino, 2008) for both measure (ε = .53) and interaction between measure and time ( ε = .67). a mixed-design anova showed statistically significant main effects of time (f(2, 40) = 14.39, p < .01, partial 2  = .42), intervention (f(1, 22) = 353.51, p < .01, partial 2  = .95) and measures (f(1.06, 21.23) = 95.27, p < .01, partial 2  =.83) as well as two-way interactions between time and measure, f(2.67, 53.29) = 7.66, p < .01, partial 2  =.28. the significant two-way interaction between time and measure was examined by testing the simple effects of measures at each time point. pairwise comparisons using a bonferroni adjustment indicated that the cbm scores were statistically higher than standardized test scores for all three time points (mdiff = 79.46, p <.01 for time 1; mdiff = 87.41, p <.01 for time 2; mdiff = 92.46, p <.01 for time 3). similarly, students scored higher on standardized test scores than teachers’ reported grades for all three time points (mdiff = 88.50, p <.01 for time 1; mdiff = 90.73, p <.01 for time 2; mdiff = 91.91, p <.01 for time 3). however, no differences were found between cbm scores and teachers’ reported grades for any of the three time points. table 4. results from mixed-design anova on reading source ss df ms f p partial 2  time 1015.12 2 507.56 14.39 <.01 .42 time * intervention 11.40 2 5.70 0.16 .85 .01 error (time) 1410.59 40 35.26 measure 344423.48 1.06 324414.61 95.27 .<.01 .83 measure * intervention 7829.28 1.06 7374.45 2.17 .16 .10 error (measure) 72308.34 21.23 3405.38 time * measure 1015.48 2.66 381.11 7.66 <.01 .28 time * measure * intervention 11.02 2.66 4.14 0.08 .96 .00 error (time * measure) 2649.72 53.29 49.72 intervention 757396.55 1.00 757396.55 353.51 <.01 0.95 error 5912.75 1.00 5912.75 2.76 .11 0.12 cbm standardized test scores student academic performance outcomes of a classroom physical activity intervention / erwin, fedewa & ahn 120 figure 2. reading across time by intervention groups discussion the present study sought to evaluate the potential effectiveness of implementing curricular pa on children’s reading and mathematics achievement. second, the authors examined the validity of curriculum-based measures with other standardized measures and grades in assessing children’s reading and mathematics achievement. each of these questions will be discussed with respect to the findings of the current study as well as implications for classroom teachers. the results of the current study suggest that curricular pa had a significantly positive effect on children’s cbm reading and mathematics scores. given the short increments of time in which these measures were administered, it is likely that cbms were better able to pick up small increments of growth in children’s achievement than were standardized test scores. the results of the additional pa on children’s reading and mathematics scores that was implemented in the treatment group confirm the general body of research in this area, suggesting that pa may enhance children’s cognitive outcomes (fedewa & ahn, 2011; sibley & etnier, 2003; trudeau & shephard, 2010). when examining mathematics scores, in particular, cbm scores for the control group peaked at time 2 and dropped at time 3, whereas the intervention group continued to improve upon their outcomes. with regard to standardized test scores, both groups scored similarly at time 1; the control group dropped at time 2, and both groups improved at time 3. again, the intervention group showed a consistent trend of improvement. teachers’ reported grades showed a jump at time 2 and a slight drop at time 3 for both groups. these trends suggest that pa enhanced learning for those students in the intervention group. for reading, students in both groups showed improvement from time 1 to time 3 on cbm scores and teachers’ reported grades. standardized test scores for reading peaked at time 2 and dropped for both groups at time 3; however, the control group demonstrated a greater drop in scores than the intervention group. thus, the pa intervention appeared to be more beneficial for mathematics. one possible explanation is that the pa breaks may have been more geared towards mathematics content thus leading to greater improvements in that area. the last hypothesis examined whether cbms are valid measures of assessing students’ achievement over time in comparison to standardized test scores and teacher grades. as mentioned earlier, although standardized test scores may be helpful in assessing the longterm effects of pa interventions on children’s cognitive outcomes, these traditional measures are not useful in assessing short-term gains or improvement as a result of the intervention (bricker et al., 2003; pretti-frontczak, 2002). in the vast majority of pa intervention research, teachers’ reported grades international electronic journal of elementary education vol.5, issue 2, 109-124,2013 121 durations of curricular interventions are not typically implemented for longer than one academic year, thus necessitating a measure that can capture small increments of achievement growth. in the present study, it was hypothesized that cbms would provide a more accurate indicator of student academic progress than standardized test scores given the short 20 week duration of the study. it was discovered that students in the treatment group had significantly higher scores in reading and mathematics when assessed by cbms, but that this difference did not reach significance when compared by standardized test scores or teacher grades. further, the reading and mathematics cbms were moderately to largely correlated with the standardized test scores while teacher grades were not correlated with either cbm or standardized test scores. these are very promising findings, as the inclusion of cbm assessments in measuring the effects of curricular pa interventions should be strongly considered. cbm assessments are short, accurate, and reliable measures that have been used to assess student academic progress for over four decades (reschly et al., 2009). perhaps by using standardized test scores or teacher grades as indicators of student academic progress, the beneficial effects of pa interventions have been missed in the literature. the current study provides preliminary evidence for the benefits of using cbms in measuring students’ academic growth as a result of curricular pa interventions. in conclusion, allotting 20+ minutes per day to provide curricular-based pa breaks to students does not appear to detract from student performance outcomes, behavior or pa levels. in fact, reading and math scores (as measured by cbms) significantly improved, while pa levels showed a trend of increasing due to this type of intervention. elementary teachers should be encouraged to incorporate pa during their lessons in the classroom setting due to the multiple positive student outcomes. . . . acknowledgements the authors would like to thank lydia childress and her students for participating in this research project. we enjoyed her passion for doing physical activity with her students. heather erwin received her ph.d. in pedagogical kinesiology, from the university of illinois at urbana-champaign in 2006. she is an associate professor at the university of kentucky. her research interests focus on school-based physical activity promotion for youth. alicia fedewa received her ph.d. in school psychology, from the michigan state university. she is an associate professor at the university of kentucky. her research interests focus on school-based physical activity. soyeon ahn received her ph.d. in measurement and quantitative methods in the department of counseling, educational psychology and special education at michigan state university in 2008. she joined the eps faculty in august, 2008. ahn’s current research focuses on the application of the existing data analytic techniques in resolving the complicated data issues in meta-analysis (research synthesis). references ahamed, y., macdonald, h., reed, k., naylor, p. j., liu-ambrose, t., & mckay, h. (2007). school-based physical activity does not compromise children's academic performance. medicine and science in sports and exercise, 39(2), 371-376. student academic performance outcomes of a classroom physical activity intervention / erwin, fedewa & ahn 122 barros, r. m., silver, e. j., & stein, r. e. k. (2009). school recess and group classroom behavior. pediatrics, 123, 431-436. basch, c.e. (2010). healthier students are better learners: a missing link in school reforms to close the achievement gap. research review (6), 1-107. beets, m. w., patton, m. m., & edwards, s. (2005). the accuracy of pedometer steps and time during walking in children. medicine and science in sports and exercise, 37(3), 513-520. bricker, d., yovanoff, p., capt, b., & allen, d. (2003). use of a curriculum-based measure to corroborate eligibility decisions. journal of early intervention, 26, 20-30. cardon, g., de clercq, d., de bourdeaudhuij, i., & breithecker, d. (2004). sitting habits in elementary schoolchildren: a traditional versus a "moving school". patient education and counseling, 54(2), 133142. centers for disease control and prevention. (2010). association between school-based physical activity, including physical education, and academic performance. atlanta, ga: author. centers for disease control and prevention. (2009). physical activity for everyone. retrieved march 23, 2012 from http://www.cdc.gov/physicalactivity/everyone/guidelines/index.html chomitz, v. r., slining, m. m., mcgowan, r. j., mitchell, s. e., dawson, g. f., & hacker, k. a. (2009). is there a relationship between physical fitness and academic achievement? positive results from public school children in the northeastern united states. journal of school health, 79, 30-37. coe, d. p., pivarnik, j. m., womack, c. j., reeves, m. j., & malina, r. m. (2006). effect of physical education and activity levels on academic achievement in children. medicine and science in sports and exercise, 38(8), 1515-1519. della valle, j., dunn, k., dunn, r., geisert, g., sinatra, r., & zenhausern, r. (1986). the effects of matching and mismatching students' mobility preferences on recognition and memory tasks. journal of educational research, 79(5), 267-272. erwin, h. e., abel, m. g., beighle, a., & beets, m. w. (2009). promoting children's health through physically active math classes: a pilot study. health promotion practice, 12(2), 244-251. erwin, h. e., beighle, a., morgan, c. f., & noland, m. p. (2011). effect of a low-cost, teacher-directed classroom intervention on elementary students' physical activity. journal of school health, 81, 455461. fedewa, a.l., & ahn, s. (2011). the effects of physical activity and physical fitness on children’s cognitive outcomes: a meta-analysis. research quarterly for exercise and sport, 82(3), 521-535. foegen, a., jiban, c., & deno, s. (2007). progress monitoring measures in mathematics: a review of the literature. journal of special education, 41(2), 121-139. fredericks, c. r., kokot, s. k., & krog, s. (2006). using a developmental movement programme to enhance academic skills in grade 1 learners. south african journal for research in sport, physical education, and recreation, 28(1), 29-42. gamst, g., myers, l. s., & guarino, a. j. (2008). analysis of variance designs: a conceptual and computational approach with spss and sas. new york, ny: cambridge university press. gibson, c. a., smith, b. k., dubose, k. d., greene, j. l., bailey, b. w., williams, s. l., et al. (2008). physical activity across the curriculum: year one process evaluation results. international journal of behavioral nutrition and physical activity, 5, 36. jitendra, a. k., sczesniak, e., & deatline-buchman, a. (2005). an exploratory validation of curriculumbased mathematical word problem solving tasks as indicators of mathematics proficiency for third graders. school psychology review, 34, 358–371. international electronic journal of elementary education vol.5, issue 2, 109-124,2013 123 liu, a. l., hu, x. q., ma, g. s., cui, z. h., pan, y. p., chang, s. y., et al. (2007). report on childhood obesity in china (6) evaluation of a classroom-based physical activity promotion program. biomedical environmental science, 20(1), 19-23. lowden, k., powney, j., davidson, j., & james, c. (2001). the class moves! pilot in scotland and wales: university of glasgow. macdonald, h. m., coupe, d. m. l., kontulainen, s. a., & mckay, h. a. (2007). a school-based physical activity intervention positively affects change in imax in preand early pubertal boys. journal of bone and mineral research, 22, s12-s12. macdonald, h. m., kontulainen, s. a., petit, m. a., beck, t. j., khan, k. m., & mckay, h. a. (2008). does a novel school-based physical activity model benefit femoral neck bone strength in preand early pubertal children? osteoporosis international, 19(10), 1445-1456. macy, m.g., bricker, d.d., & squires, j.k. (2005). validity and reliability of a curriculumbased assessment approach to determine eligibility for part c services. journal of early intervention, 28(1), 1-16. maeda, j. k., & randall, l. m. (2003). can academic success come from five minutes of physical activity? brock education journal, 13(1), 14-22. mahar, m. t., murphy, s. k., rowe, d. a., golden, j., shields, a. t., & raedeke, t. d. (2006). effects of a classroom-based program on physical activity and on-task behavior. medicine and science in sports and exercise, 38(12), 2086-2094. national center for health statistics. (2009). health, united states, 2008. hyattsville, md: u.s. department of health and human services. norlander, t., moas, l., & archer, t. (2005). noise and stress in primary and secondary school children: noise reduction and increased concentration ability through a short but regular exercise and relaxation program. school effectiveness and school improvement, 16(1), 91-99. ogden, c. l., yanovski, s. z., carroll, m. d., & flegal, k. m. (2007). the epidemiology of obesity. gastroenerology, 132, 2807-2102. pangrazi r,, beighle a,, & pangrazi d. (2009). promoting physical activity and health in the classroom. san francisco, ca: pearson benjamin cummings. pate, r.r., davis, m. g., robinson, t. n., stone, e. j., mckenzie, t. l., & young, j. c. (2006). promoting physical activity in children and youth: a leadership role for schools. circulation, 114, 1214-1224. pretti-frontczak, k.l. (2002). using curriculum-based measures to promote a linked system approach. assessment for effective intervention, 27(4), 15-21. reschly, a.l., busch, t.w., betts, j., deno, s.l., & long, j.d. (2009). curriculum-based measurement oral reading as an indicator of reading achievement: a meta-analysis of the correlational evidence. journal of school psychology, 47, 427-469. sibley, b. a., & etnier, j. l. (2003). the relationship between physical activity and cognition in children: a meta-analysis. pediatric exercise science, 15, 243-256. stecker, p. m., & lembke, e. s. (2005). advanced applications of cbm in reading: instructional decision making manual. retrieved march 20, 2010 from http://www.studentprogress.org/library/training/ cbmmath/advancedreading/advrdgmanualformattedsept29.pdf stevens, t. a., to. y., stevenson, s. j., & lochbaum, m. r. (2008). the importance of physical activity and physical education in the prediction of academic achievement. journal of sport behavior, 31, 368-388. retrieved march 20, 2012 from http://search.ebscohost.com.ezproxy.uky.edu/login.aspx?direc t=true&db=aph&an=35130781&site=ehost-live&scope=site student academic performance outcomes of a classroom physical activity intervention / erwin, fedewa & ahn 124 stewart, j. a., dennison, d. a., kohl, h. w., & doyle, j. a. (2004). exercise level and energy expenditure in the take 10! in-class physical activity program. journal of school health, 74(10), 397-400. strong, w. b., malina, r. m., blimkie, c. j. r., daniels, s. r., dishman, r. k., gutin, b….trudeau, f. (2005). evidence based physical activity for school-age youth. journal of pediatrics, 146, 732-737. troiano, r. p., berrigan, d., dodd, k. w., masse, l. c., tilert, t., & mcdowell, m. (2008). physical activity in the united states measured by accelerometer. measurement and science in sports and exercise, 40, 181-188. trudeau, f. & shephard, r.j. (2010). relationships of physical activity to brain health and the academic performance of schoolchildren. american journal of lifestyle medicine, 4, 138-150. uhrich, t. a., swalm, r. l. (2007). a pilot study of a possible effect from a motor task on reading performance. perceptual and motor skills, 104, 1035-1041 vincent, s. d., & pangrazi, r. p. (2002). determining baseline physical activity levels in children. medicine and science in sports and exercise, 14, 432-441. world health organization. (2009). global strategy on diet, physical activity and health. retrieved march 10, 2012 from http://www.who.int/dietphysicalactivity/childhood/en/ international electronic journal of elementary education, september 2016, 9(1), 109-123. issn:1307-9298 copyright © iejee www.iejee.com emotioncy: a potential measure of readability reza pishghadam a * hannaneh abbasnejad a a ferdowsi university of mashhad, iran received: 29 july 2016 / revised: 20 september 2016 / accepted: 24 september 2016 abstract given the deficiencies of readability formulae as reliable tools for measuring text readability in educational settings, this study aims to offer a new measure to improve the current methods of testing the readability levels of texts through the incorporation of the newly-developed concept of emotioncy. to this end, a group of 221 students were selected from the 2nd to 4th grades of an elementary school. they were given a passage from a 5th grade textbook whose readability was measured by flesch and fog indices. the students were asked to fill out an 8-item emotioncy scale along with a difficulty-finding item. one-way anova and correlational analysis were used to analyze the data. findings indicated that learners’ comprehension increases as a result of an increase in their emotioncy levels. in the end, the results were discussed and some suggestions were made to employ emotioncy in measuring readability. keywords: readability, emotioncy, exvolvement, avolvement. introduction the twentieth century was marked with the advent of readability formulae, which made a considerable difference in the design of textbook materials in educational settings. since comprehension is the most fundamental target of written materials, many textbook writers and test designers regarded these formulae as appropriate and objective measures for matching readers` language abilities to text difficulty levels (e.g., el-haddad, spooner, faruqi, denney-wilson, & harris, 2016; instone, 2011; raj, sharma, singh, & goel, 2016). however, despite their popularity, these formulae received a lot of criticism (pichert & elam, 1985; schriver, 2000). researchers realized that, by focusing on linguistic factors, these formulae disregard human interests, background knowledge, and motivation, and hence fail to provide an exact estimate of human comprehensibility (schriver, 2000). to transcend the linguistic measure of readability and to take the emotional factors into account, it seems that the newly designed concept of emotioncy can shed more light on the nature of readability (pishghadam, tabatabaeyan & navari, 2013). emotioncy is defined as the emotions evoked by the senses from which we receive inputs (pishghadam, jajarmi, & shayesteh, 2016). pishghadam (2015) is of the view that senserelated emotions can affect our comprehension. in fact, senses and emotions are so * corresponding author: reza pishghadam, professor of language education, professor by courtesy of educational psychology, ferdowsi university of mashhad, iran. email: pishghadam@um.ac.ir http://www.iejee.com/ international electronic journal of elementaryeducationvol.9, issue1, 109-123, september2016 110 intertwined that separating them from each other can hamper our understanding of the true nature of learning. emotioncy, ranging from avolvement (null) and exvolvement (auditory, visual, and kinesthetic) to involvement (inner and arch), is believed to be tied to learning (pishghadam, 2015). as pishghadam and shayesteh (in press) have shown, individuals with high levels of emotioncy for words in their first language can learn similar words in a second language faster. therefore, emotioncy with a focus on the integration of senses and emotions has the potential to be utilized as a measure of comprehension. given the significance of readability in devising textbooks and reliable tests (oakland & lane, 2004) and considering the shortcomings of readability formulae (davison & kantor, 1982; lenzner, 2014), it is our belief that emotioncy can be employed as a complementary source of measuring readability. in sum, the major objective of this study is to propose a new measure of readability, which takes the readers` background knowledge and emotions into account. thus, it is hypothesized that by having emotioncy measures along with the results of readability formulae, one can have a more accurate estimate of the comprehensibility of a text. theoretical framework readability formulae interest in the readability of texts emerged in the 20th century (davison & bolt, 1981), when the most popular readability formulae were created and published (e.g., dale & chall, 1948; flesch, 1951; fry, 1968; gunning, 1968; spache, 1953). readability has been defined as the level of ease with which one can understand and comprehend a piece of writing (richards, platt, & platt, 1992). since these formulae were built upon two quantitative variables, namely word length/frequency and sentence length, they were considered to be objective measures of language difficulty (lenzner, 2014). therefore, they were widely used by writers, textbook publishers, and test reviewers as reliable indices to match the difficulty level of a text to readers' language abilities (dreyer, 1984; oakland & lane, 2004; sattler, 2002; zakaluk & samuels, 1988). not long after the introduction of readability formulae, their reliability and validity were questioned by researchers (e.g., rygiel, 1982; wheeler & sherman, 1983). first and foremost, it is safe to state that the very features upon which the formulae were written are faulty. regarding word length, which is the basic component of fog, fre and fkg formulae (flesch, 1948; flesch, 1979; gunning, 1968), it is commonly conceived that the number of polysyllabic words makes a text difficult to perceive, a statement proven wrong on more than one occasion (bailin & grafstein, 2001; dreyer, 1984). since derivation is the most common way of word coinage in english (yule, 2014), there are numerous derived words that hint to their meanings because they are made up of affixes whose meanings are already known to most readers. for instance, it is easier to understand the meaning of the term unemployment than to comprehend the meaning of the word dearth. the case is also true when it comes to compound words. usually, because readers know the meaning of word parts, they can easily guess the meaning of the overall word, e.g., playground (lenzner, 2014). defects of considering word length as a decisive factor were moderated by substituting word frequency for word length in some other formulae such as those of spache (1953) and dale-chall (1948). this concept was taken into practice by creating word lists for the most frequent words and marking the words which do not appear on the list as “rare” (lenzner, 2014). this was also criticized since chall and dall (1995) were to extend their word lists over the years. these word lists also take into account words such as cobbler and washtub which are not used by youngsters today (bailin & grafstein, 2001) and ignore the most frequently used words such as internet and download which have been developed in recent emotioncy: a potential measure of readability / pishghadam & abbasnejad 111 decades. therefore, the most crucial problem of word lists seems to be their negligence toward the fact that different sociocultural groups may have different vocabulary repertoires over years (lenzner, 2014). as for sentence length, the presupposition underlying almost all readability formulae, the longer a sentence is, the more cumbersome the process of analyzing, and therefore the more difficult the comprehension will be, is also thoroughly undermined by davison (1981) and proved to be incorrect in some cases (davison & kantor, 1982; dreyer, 1984). furthermore, readability formulae have been sharply criticized for their heavy reliance on quantitative factors (e.g., vocabulary and syntax) and their disregard of qualitative factors (e.g., idea density and conceptual difficulty). it seems that the factors influencing text readability are not only text-oriented, but they are also reader-oriented. background knowledge, reading fluency, motivation, and engagement are regarded as the most significant factors of the latter nature (oakland & lane, 2004). basically, readability formulae are completely heedless of the interactive nature of the reading process (meade & smith, 1991), and therefore do not correlate with the psycholinguistic model of reading (zamanian & heidary, 2012). some studies emphasized personal interest or the purpose of the reader as a potential source of comprehensibility (davison & kantor, 1982; dreyer, 1984; fry, 1975; klare, 1976; meade & smith, 1991; schriver, 1989). meade and smith (1991) illustrated the point by providing an example of hospital patients recovering after receiving kidney transplants who, as a result of having recently gone through an operation, might be more interested in reading medical instructions provided by the practitioners compared to other people. as another case in point, fry (1975) pointed to students who were deeply fascinated by reading texts about animals when they were 3 years old but had a shift of interest toward texts written on the boy-girl relationship issues during puberty. given that the formulae are often criticized, even called unreliable and deceptive (pichert & elam, 1985; schriver, 2000), it seems that there is no choice but to focus on teacher/writer’s personal decisions, and regard them as the authority figure in determining text difficulty as well as taking measures to facilitate the process of reading and comprehension for students/readers (davison & kantor, 1982; dreyer, 1984; instone, 2011). this approach is, however, very subjective and rather controversial. emotioncy inspired by greenspan’s (1992) developmental, individual-differences, relationship-based (dir) model of first language acquisition which places emphasis on the affective domain of human behavior, pishghadam, tabatabaeyan, et al. (2013) introduced emotioncy as the missing link in shaping human cognition. believing that words are probably acquired in a cultural milieu and not in isolation, pishghadam, et al. (2016) stated that people might have different degrees of emotioncy towards varied items of a language based on their sensory experiences. in order to elucidate the concept, pishghadam (2015) proposed a hierarchical model for different kinds of emotioncy (figure 1). international electronic journal of elementaryeducationvol.9, issue1, 109-123, september2016 112 figure 1. emotioncy levels (adapted from “emotioncy in language education: from exvolvement to involvement”, by r. pishghadam, 2015, october, paper presented at the 2nd conference of interdisciplinary approaches to language teaching, literature, and translation studies. iran, mashhad). pishghadam, et al. (2016) proclaimed that based on individuals' sensory experiences, they might have different types of emotioncies, namely: null, auditory, visual, kinesthetic, inner, and arch (see table 1). moreover, as depicted in figure 1, pishghadam (2015) categorized learners into three groups of avolved (no emotional experience), exvolved (auditory, visual, and kinesthetic emotioncies), and involved (inner and arch emotioncies). in the same vein, pishghadam, adamson, and shayesteh (2013) introduced emotion-based language instruction (ebli) as a new perspective on bilingual education, highlighting the crucial role of emotional involvement in molding individuals’ worldviews. they claimed that words evoking higher levels of emotioncy in learners are acquired faster than ones they have no or less emotional experience with. pishghadam, et al. (2016) later proposed sensory relativism as an expansion of linguistic relativism, claiming that reality can change within and across individuals, based on the sensory inputs they receive. drawing upon this principle, pishghadam, adamson, et al. (2013) suggested that individual’s already-possessed knowledge should be considered to be of primary importance in language instruction and comprehension. pishghadam, adamson, et al. (2013) cast more light on the idea by listing two aspects for each concept: word and world, both necessary in order for the thorough understanding of a concept. mcnamara, ozuru, and floyd, (2011) also highlighted the significance of world knowledge as a mediating gap between readers’ current state of knowledge and the kind of knowledge demanded by the text. this is exactly why some researchers underscored the significance of relating new concepts to already existing concepts in the mind in order for more profound comprehensibility to be achieved (brown, 2000; shrum & glisan, 1994; van den broek, kendeou, lousberg, & visser, 2011). giving more salience to the idea of background knowledge in interpreting the world in piaget’s schema theory (1926), pishghadam, adamson, et al. (2013) switched the focus from prior knowledge to prior emotions, stating emotioncy: a potential measure of readability / pishghadam & abbasnejad 113 that it is the already established emotions toward a particular concept which facilitate individuals' understanding of the world. in contrast to krashen’s input hypothesis (1982), in which comprehensible input is defined as the one which is one step beyond learner’s preexisting knowledge (i+1), pishghadam, adamson, et al. (2013) firmly laid down the idea that it is the input emotioncy level that should supersede the learner’s current state in order for better comprehension to be achieved. in this light, without establishing a firm emotional relationship with the text, the reader would not be able to grasp the meaning (pishghadam, & shayesteh, in press). pishghadam, adamson, et al. (2013) elaborated on the concept by providing an example of an iranian girl who has never experienced words such as “bar”, “drinking” and “wine” in her real life. they believe that this girl would have more difficulty perceiving such words in a written text in comparison with someone whose cultural background includes these concepts. table 1. emotioncy kinds type experience null emotioncy when an individual has not heard about, seen, or experienced an object or a concept. auditory emotioncy when an individual has merely heard about a word/concept. visual emotioncy when an individual has both heard about and seen the item. kinesthetic emotioncy when an individual has touched, worked, or played with the real object. inner emotioncy when an individual has directly experienced the word/concept. arch emotioncy when an individual has done research to get additional information. (adapted from “conceptualizing sensory relativism in light of emotioncy: a movement beyond linguistic relativism”, by r. pishghadam, h. jajarmi, and sh. shayesteh, 2016, international journal of society, culture & language. copyright 2015 by ijscl). in a similar vein, people coming from various regional, religious, and socio-cultural backgrounds would have quite different vocabulary repertoires (lenzner, 2014). consequently, they would have a better comprehension of terms associated with their socio-economic status (pishghadam, et al., 2016). in order to delineate the point, pishghadam, baghaei, and seyednozadi (in press) provided an example within the geographical boundaries of iran, stating that people from western parts of the country have a better familiarity with words such as snow and mountain compared to those living in northern parts, who are more emotionally engaged with words such as rain and jungle. this view emphasizing localization highlights the significance of emotionalization in figuring out the world (pishghadam, et al., in press). therefore, emotionalization should definitely be taken into account when it comes to language teaching and testing (pishghadam, et al., in press). each sense stimulates a specific part of the brain (wagner, et al., 1998); readers who are emotionally engaged in the text; therefore, put less burden on their working memories and so would have less cognitive overloads. consequently, it would be easier for them to process the input and analyze the meaning (pishghadam, 2016b). on the whole, due to the fact that readability formulae were proved unreliable, even invalid in some cases, the current study attempts to introduce a new criterion for measuring the readability of a text. given that “emotional engagement provides meaningfulness” (pishghadam & shayesteh, in press, p.4), emotioncy may be a helpful cognitive joint to invest on. with that in mind, this study intends to answer the following research questions: 1. do readability formulae measure the difficulty levels of texts accurately? international electronic journal of elementaryeducationvol.9, issue1, 109-123, september2016 114 2. are there any significant differences among different groups of students with respect to emotioncy and difficulty levels? method participants the sample used in this study comprised 221 school boys, 8 (n= 66), 9 (n= 70) and 10 (n= 85) years of age (2nd, 3rd & 4th grades), studying in an elementary school in mashhad, iran. they were middle class learners who were selected through convenience sampling. they engaged in the study as a result of their enthusiasm and their parents’ and teachers’ permission. moreover, due to the fact that schools in iran are gender-segregated, the participants were only male ones. therefore, great care was taken to select a text which seems to be neutral to both genders. instruments to conduct this study, the following instruments were utilized by researchers: a sample of a reading passage chosen from the persian reading textbook of iranian students studying at the 5th grade (appendix 1) and an eight-item emotioncy scale (appendix 2) plus a likerttype item to determine the text difficulty level based on students' opinions (appendix 3). the reading passage was selected in a way to include both concrete words and abstract entities at the same time. it should be kept in mind that since the educational system in iran is centralized, that is to say, the government is absolutely in charge and textbooks are written to be taught to all intended students in the whole country, designing appropriate textbooks is of high importance. that is why, we selected a text from the persian book of the 5th graders and designed an 8-item emotioncy scale to check students’ familiarity with the words in the paragraph. the metric (figure 2) was comprised of two scales: frequency and emotion. the frequency aspect ranged from "a little" to "a lot". it aimed to measure the amount of individuals' exposure to a particular concept or word through his/her senses, be it visual, auditory, kinaesthetic, inner (doing it), or arch (conducting research on it). the emotion aspect of the metric measured the valence of emotions ranging from extremely negative to extremely positive. the emotioncy score ranged from 0 to 50. the students were supposed to rate the words based on whether they had just heard about it (1 point), they had heard about and seen it (2 points), they had heard, seen, and touched it (3 points), they had heard, seen, touched, and done it (4 points), and they had done research on it (5 points). then, the total emotioncy scores were calculated by the following formula: emotioncy = sense (frequency + emotion) now imagine a student who has expressed his emotioncy for the word sea as follows: i have gone to sea and swum in it (sense score: 4) many times (frequency score: 5) and i feel extremely positive about sea (emotion score: 5). his total emotioncy score would be 4(5+5) =40, which shows that the student is completely involved with the notion of sea. emotioncy: a potential measure of readability / pishghadam & abbasnejad 115 figure 2. a metric for measuring emotioncy (adapted from “emotioncy, extraversion and anxiety in willingness to communicate in english” by r. pishghadam, 2016a, may, paper presented at the 5th international conference on language, education and innovation. england: london). procedure in order to determine the difficulty level of the text, flesch new reading formula modified by dayyani (as cited in soltani & koosha, 2015) and gunning fog index of readability validated for persian texts by maftoon and daghigh (2001) were utilized. then, the first paragraph of the text was selected as a representative of the whole text and eight words (radio, newspaper, magazine, star, sea, ocean, report, and result) were selected to be measured in the emotioncy scale. since the participants were children, the research procedure was first introduced to them through examples. the passage was intended for the 5th grade students and marked respectively fairly difficult and extremely difficult by flesch and fog indices of readability. the reason behind choosing students from lower grades was to have a group of students who would surely have found it difficult to comprehend the text in accordance with the prediction made by the formulae. they were asked to read the first paragraph of the text and determine its difficulty level from 1=extremely easy to 5=extremely difficult. immediately after that, the emotioncy scale was distributed and explicated to them thoroughly. they were asked to take the emotioncy scale. finally, cronbach`s alpha, one-way anova, post hoc scheffe tests, and pearson productmoment correlation were employed to analyse the data by using spss 20 software. results the following table illustrates the difficulty level of the text based on flesch and fog indices of readability. table 2. scores obtained from two readability formulae formula score interpretation flesch fog 52.6 27.5 fairly difficult extremely difficult as depicted in table 2, the selected passage was considered hard for students of the 5th grade of the elementary school based on the utilized readability formulae. what was international electronic journal of elementaryeducationvol.9, issue1, 109-123, september2016 116 claimed by readability formulae was challenged since the level of difficulty was not consistent for all three groups. table 3 summarizes the results obtained from analysing the emotioncy scale and the difficulty level of the text. moreover, the reliability of the emotioncy scale was .89 for all the participants involved as calculated by cronbach’s alpha; and kolmogorov smirnov test was used to ensure the data was distributed normally. table 3. descriptive statistics grades n frequency emotion emotioncy difficulty mean sd mean sd mean sd mean sd 2 66 10.06 2.6 12.12 1.6 22.18 2.9 3.9 1.2 3 70 17.70 2.1 12.19 1.7 24.89 3.3 3.1 .98 4 85 15.20 3.4 13.16 2.1 28.36 3.5 2.2 .31 as table 4 suggests, the claim made by readability formulae with regard to the difficulty level of the text was relatively true. moreover, with respect to the level of text difficulty, the differences among all three groups were statistically significant (f=2.6, p< .05). it was rated difficult (mean= 3.9) for students of the 2nd grade, but average in difficulty (mean= 3.1) for students of the 3rd grade, and finally marked easy (mean= 2.2) for the 4th grade students. it seems that difficulty is a relative concept which is determined by other factors. with regard to the emotioncy scale, as illustrated in table 4, emotioncy differences among all of the three groups were statistically significant (f=7.7, p< .05). this means that students of the 4th grade had higher levels of emotioncy (mean= 28.36) than the students in the 3rd grade (mean= 24.89) and both groups demonstrated higher levels of emotioncy than the 2nd grade students (mean= 22.18). considering the frequency aspect of emotioncy, the differences among all groups were found to be significant (f=.3.4, p< .05), while the emotion differences were not statistically significant (f= 1.1, p> .05). in fact, as the results of post hoc scheffe tests revealed, as students` age increases, their sense experiences also increase in a way that leads to higher levels of emotioncy. table 4. anova and post hoc scheffe tests for all variables variables df f p level post hoc scheffe frequency 218 3.4 .03 4>3>2 emotion 218 1.1 .09 non-significant emotioncy 218 7.7 .00 4>3>2 difficulty level 218 2.6 .04 2>3>4 moreover, the results of correlational analyses revealed that, except for emotion (r= -.11, p>.05), emotioncy (r= -.44, p<.05) and frequency (r= -.31, p<.05) are statically correlated with text difficulty (see table 5). the results are in line with those of anova analyses. table 5. correlations between all variables variable frequency emotion emotioncy difficultly level -.31** -.11 -.44** ** p< .05 emotioncy: a potential measure of readability / pishghadam & abbasnejad 117 discussion establishing reliable criteria for measuring the difficulty level of a text and matching it to readers' ability is one of the basic requirements of educational settings. having that in mind, in this study, we first sought to inspect the practical function of readability formulae as long trustworthy measures of comprehensibility. we then tried to examine whether the newly designed notion of emotioncy can act as a reliable measure for readability. the outcomes of the study indicated that although readability formulae have been utilized in educational milieu for a long time, they are not as dependable as expected. this suggests that readers’ comprehension of a text might not necessarily correlate with the obtained results from readability formulae. as the results of this study indicate, the text marked as fairly difficult and extremely difficult for a class of readers by specific formulae (in the case of this study, flesch, 1979 and fog, 1968) might be easy for not only the target group, but also for lower level readers, that is, 4th graders. the results show that difficulty is not something that is set firmly in the text through textual factors, but something that is inconsistent from a group of learners to the other. this left the researchers with the idea that other factors which are likely to be human-oriented ones are probably in charge of determining text difficulty. since these factors are totally neglected in readability formulae, the need for opening a new vista of comprehension tools which take into account reader oriented factors is deeply felt. moreover, it was found that students were emotionally exvolved or even involved towards certain concepts/words in the passage. this shows that students were already familiar with the concepts raised in the passage. in addition, there was a significant increase in emotioncy levels when students transcended in age (4th graders’ emotioncy levels > 3rd graders’ > 2nd graders’), which indicates that the more frequently they are exposed to a certain concept, the higher their emotioncy levels, and therefore comprehension will be. it seems that difficulty is a relative concept which is relativized by the amount of emotional engagements. on the contrary to what ausubel’s (1965) notion of relatability claims, it is not only the cognitive connections which tie individuals to words in a passage, but also the emotional dependency on the text itself. that is to say, individuals’ interests, their background knowledge, and motivation, noted among paramount factors disregarded in readability formulae (klare, 1976; schriver, 1989), are the basic requirements of emotioncy which tries to approach readers' minds through their hearts (pishghadam, adamson, et al., 2013). one rationale for an input marked as difficult for the fifth graders to be fairly understandable for students of the 4th grade might be their emotional relationship with the concepts discussed in the passage. information is stored in working memory and is transferred to permanent memory only when it finds a peg to stand on (stevick, 1996). according to pishghadam (2016b), one likely justification may be that working memory is less entangled with processing data when a firm emotional background is established. the result is less cognitive overload, and consequently a more desirable comprehension rate. should this be the case, it would no longer be surprising that some students could easily understand what was marked as hard for them through forming an emotional rapport with the text. by the same token, diverse socioeconomic capitals provide individuals with different sensory experiences. as pishghadam and shayesteh (in press) claim, learners bring with themselves their social and cultural capitals, hence their learning and comprehension of the concept in question may largely differ in accordance with their socio-economic status. that is, there always exists an emotioncy gap within learners of different socio-economic status (pishghadam, 2016a), which leaves us with the idea that an emotioncy gap may generate a comprehension gap among learners. since emotioncy has a dynamic nature and can fluctuate over years, that is to say, an avolved student may become emotionally involved over time (pishghadam, 2016a), it is recommended for teachers to take steps to compensate international electronic journal of elementaryeducationvol.9, issue1, 109-123, september2016 118 for the shortage of comprehensibility by bridging the gap prior to the initiation of teaching by providing emotional experience for learners. derrida, in his deconstruction theory, also highlights the dynamic nature of readers' comprehension of text by claiming that there are “moments of meaning that give way, inevitably to more meanings” (tyson, 2006, p. 259), the fleeting nature of meaning in this quotation can be best justified by the dynamic nature of emotioncy. all in all, although further studies are required to solidify the practicality and generalizability of the current study, this study intends to bring to light the missing piece of readability measures by opening a new window to the notion of emotioncy. the obtained results can have broad implications for educational administers in writing textbooks and reviewing tests, as well as designing syllabi. firstly, for educators to remember the fact that readability is a relative concept which is relativized by readers’ emotional backgrounds and readability formulae per se are not credible anchors to rest upon. they must therefore take into account emotioncy scales alongside the readability formulae. secondly, another possible implication is that textbooks should be localized on the basis of learners' cultural emotional capitals. as it would be impractical to change and reshape all textbooks, another recommendation is for educators to incorporate a diverse emotional load in the textbooks so that all learners can eventually have an adequate and equal share of understanding. thirdly, teachers should be cognizant of the fact that they are responsible for bridging the gap conceived by varied emotional status among learners. they should come up with strategies to make up for that. finally, textbook writers should contemplate the forms they come up with to convey a particular concept and to make it congruent with the readers’ current state of emotioncy, to select words whose world is already experienced by the reader. on the whole, the issue of adapting comprehensibility of what is written with what one would comprehend is still of paramount concern. this study serves as the merging point of the reader and the writer, where emotioncy of form, concept, or situation (manifested in the mind of the reader) espouses readability formulae (manifested in the work of the author). nonetheless, readers should bear in mind that the present study has its own restrictions. first, the sample used in this study comprised only the male middle class students; another study is needed to be done to take females and other socio-economic statuses into account. second, since this study was the first study of its own kind to measure readability through emotioncy, generalization of the findings is not recommended. in fact, the new criterion for readability can serve as a suggestive model which calls for further empirical support to determine the levels of text difficulty for different groups of individuals. • • • references ausubel, d. (1965). a cognitive structure view of word and concept meaning. in r. c. anderson & d. ausubel (eds.). readings in the psychology of cognition (pp. 103-115). new york: holt, rinehart and winston. bailin, a., & grafstein, a. (2001). the linguistic assumptions underlying readability formulae: a critique. language & communication, 21(3), 285-301. brown, h. d. (2000). principles of language learning and teaching. ny: longman. chall, j. s., & dale, e. (1995). readability revisited: the new dale-chall readability formula. cambridge, ma: brookline books. emotioncy: a potential measure of readability / pishghadam & abbasnejad 119 dale, e., & chall, j. s. (1948). a formula for predicting readability: instructions. educational research bulletin, 27, 37-54. davison, a. (1981). readability--appraising text difficulty. reading education report no. 24. davison, a., & kantor, r. n. (1982). on the failure of readability formulas to define readable texts: a case study from adaptations. reading research quarterly, 17, 187-209. davison, a., & bolt, b. (1986). readability, and questions of textbook difficulty (reading education report no. 66). urbana, illinois: center for the study of reading. retrieved from http://www.eric.ed.gov/ davison, a. (1988). linguistic complexity and text comprehension: readability issues reconsidered. lawrence erlbaum. dreyer, l. g. (1984). readability and responsibility. journal of reading, 27(4), 334-338. el-haddad, n., spooner, c., faruqi, n., denney-wilson, e., & harris, m. (2016). readability and content analysis of lifestyle education resources for weight management in australian general practice. bmc obesity. advance online publication. doi: 10.1186/540608-016-0097-1 flesch, r. (1948). a new readability yardstick. journal of applied psychology, 32(3), 221. flesch, r. (1951). how to test readability. new york: harper & brothers. flesch, r. f. (1979). how to write plain english: a book for lawyers and consumers. new york: harpercollins. fry, e. (1968). a readability formula that saves time. journal of reading, 11, 513–516. fry, e. (1975). the readability principle. language arts, 52(6), 847-851. greenspan, s. i. (1992). infancy and early childhood: the practice of clinical assessment and intervention with emotional and developmental challenges. madison, ct: international universities press. gunning, r. (1968). the technique of clear writing. new york: mcgraw-hill. instone, e. a. (2011). the variance amongst the results of readability formulas regarding us history textbooks (unpublished doctoral dissertation). bowling green state university. klare, g. r. (1976). a second look at the validity of readability formulas. journal of literacy research, 8(2), 129-152. krashen, s. (1982). theory versus practice in language training. in r.w. blair (ed.). innovative approaches to language teaching (pp. 15–24). rowley, ma: newburry house publishers. lenzner, t. (2014). are readability formulas valid tools for assessing survey question difficulty? sociological methods & research, 43 (4), 677-698. maftoon, p., & daghigh. m. (2001). the yardstick for measuring the readability level of persian translations of english texts. retrieved from http://www.ensani.ir/storage/files/20110126112624-35.pdf mcnamara, d. s., ozuru, y., & floyd, r. g. (2011). comprehension challenges in the fourth grade: the roles of text cohesion, text genre, and readers' prior knowledge. international electronic journal of elementary education, 4(1), 229-257. meade, c. d., & smith, c. f. (1991). readability formulas: cautions and criteria. patient education and counselling, 17(2), 153-158. oakland, t., & lane, h. b. (2004). language, reading, and readability formulas: implications for developing and adapting tests. international journal of testing, 4(3), 239-252. international electronic journal of elementaryeducationvol.9, issue1, 109-123, september2016 120 piaget, j. (1926). the language and thought of the child. london: routledge and kegan. pichert, j. w., & elam, p. (1985). readability formulas may mislead you. patient education and counseling, 7(2), 181-191. pishghadam, r. (2015, october). emotioncy in language education: from exvolvement to involvement. paper presented at the 2nd conference of interdisciplinary approaches to language teaching, literature, and translation studies. iran, mashhad. pishghadam, r. (2016a, may). emotioncy, extraversion, and anxiety in willingness to communicate in english. paper presented at the 5th international conference on language, education and innovation. england: london. pishghadam, r. (2016b, march). giving a boost to the working memory: emotioncy and cognitive load theory. a paper presented in the first national conference on english language teaching, literature, and translation. ghoochan, iran. pishghadam, r., adamson, b., & shayesteh, s. (2013). emotion-based language instruction (ebli) as a new perspective in bilingual education. multilingual education, 3(9), 1-16. pishghadam, r., baghaei, p., & seyednozadi, z. (in press). introducing emotioncy as a potential source of test bias: a mixed rasch modelling study. international journal of testing. pishghadam, r., jajarmi, h., & shayesteh, s. (2016). conceptualizing sensory relativism in light of emotioncy: a movement beyond linguistic relativism. international journal of society, culture & language, 4(2), 11–21. pishghadam, r., & shayesteh, s. (in press). emotioncy: a post-linguistic approach toward vocabulary learning and retention. sri lanka journal of social sciences. pishghadam, r, tabatabaeyan, m. s., & navari, s. (2013). a critical and practical analysis of first language acquisition theories: the origin and development. iran, mashhad: ferdowsi university of mashhad publications. richards, j. c., platt, j., & platt, h. (1992). longman dictionary of language teaching and applied linguistics. london: longman. raj, s., sharma, v., l., singh, a., j. & goel, s. (2016). evaluation of quality and readability of health information websites identified through india’s major search engines. advances in preventive medicine. advance online publication. doi: 10.1155/2016/4815285 rygiel, m. a. (1982). readability formulas: pluses and minuses. teaching english in the two-year college, 9(1), 45–49. sattler, j. m. (2002). assessment of children: behavioral and clinical applications. san diego: jerome m. sattler, publisher. schriver, k. a. (1989). evaluating text quality: the continuum from text-focused to reader-focused methods. ieee transactions on professional communication, 32(4), 238-255. schriver, k. a. (2000). readability formulas in the new millennium: what's the use? acm journal of computer documentation (jcd), 24(3), 138-140. shrum, j., & glisan, e. (1994). teacher's handbook: contextualized language instruction. usa: heinle & heinle. soltani, s., & koosha, m. (2015). text readability in translation of children’s literature. lap lambert academic publishing. spache, g. (1953). a new readability formula for primary-grade reading materials. the elementary school journal, 53(7), 410-413. emotioncy: a potential measure of readability / pishghadam & abbasnejad 121 stevick, e. w. (1996). memory, meaning and method: a view of language teaching. boston: heinle & heinle. tyson, l. (2006). critical theory today. london: routledge. van den broek, p., kendeou, p., lousberg, s., & visser, g. (2011). preparing for reading comprehension: fostering text comprehension skills in preschool and early elementary school children. international electronic journal of elementary education, 4(1), 259-268. wagner, a. d., schacter, d. l., rotte, m., koutstaal, w., maril, a., dale, et al. (1998). building memories: remembering and forgetting of verbal experiences as predicted by brain activity. science, 281, 1188–1191. wheeler, g., & sherman, t. f. (1983). readability formulas revisited. science and children, 20(7), 38– 40. yule, g. (2014). the study of language. cambridge university press. zakaluk, b. l., & samuels, s. j. (1988). readability: its past, present, and future. newark, del: international reading association. zamanian, m., & heydari, p. (2012). readability of texts: state of the art. theory and practice in language studies, 2(1), 43-53. international electronic journal of elementaryeducationvol.9, issue1, 109-123, september2016 122 appendices appendix 1: reading passage chosen from persian reading textbook of iranian students studying at the 5th grade (translated version) every day you listen to the radio, read newspapers, sift through magazines, watch tv, and hear or see the news which can be fantastic and incredible. one reports the discovery of a new star, others speak of depth of seas or oceans. what you see may even be the successful results of an experiment. appendix 2: a sample item for the emotioncy scale appendix 3: question for finding the difficulty level emotioncy: a potential measure of readability / pishghadam & abbasnejad 123 www.iejee.com this page is intentionally left blank http://www.iejee.com/ microsoft word iejee_5_3_bang international electronic journal of elementary education, 2013, 5(3), 219-232. issn:1307-9298 copyright © iejee www.iejee.com exploring impacts of the eed 420 science methods course on pre-service elementary teachers’ views regarding the nature of science∗∗∗∗ eunjin bang ∗∗∗∗∗∗∗∗ iowa state university, ames, iowa, usa received: 3 march 2013 / revised: 15 may 2013/ accepted: 12 june 2013 abstract this study explores the impact of a semester-long science methods course examining pre-service elementary teachers’ views on the nature of science (nos). also examined were nos characteristics that pre-service teachers incorporated into their science lesson plans and peer teachings, during the course. data used for this study were obtained from 21 pre-service teachers who participated in the pre/post card exchange game, pre/post vnos interviews, 5e lesson plans, and peer teaching performances. the results of the study showed that some changes were made as a result of eed 420— such as starting to view science as a data-gathering experimental endeavor, rather than just a theorydriven endeavor. none of the groups explicitly designed or taught their lesson’s nos aspects. the study posits that a mere one semester-long science method’s course is insufficient to adequately improve understanding of the nos, and to establish a sufficiently robust desire in pre-service teachers for them to implement nos into their lessons. keywords: pre-service elementary teachers, nature of science, science methods course, science as inquiry, 5e instructional model. introduction there has been an awareness among people in science education that a deeper understanding of the history and philosophy of science (hps) would greatly contribute to improving the quality of science teaching and learning (matthews, 1994; mccomas, clough, & almazroa, 2000). some go even further by positing that a deeper comprehension of the ∗ this study has been presented at a poster session at the 11th annual hawaii international conference on education, honolulu, hawaii, in 2013. ∗∗ n165d lagomarcino hall, school of education, iowa state university, ames, iowa, u.s.a., 500113192, phone: 011+1+515+294+7371, e-maill: ejbang@iastate.edu exploring impacts of the eed 420 science methods course / bang 220 true meaning of the concepts of fact, law, theory, observation, and experimental process are imperative to a full enhancement of science teaching (gardner, 1972; hainsworth, 1956; hodson, 1988; rhodes & schaible, 1989; trusted, 1979). along with this attention to hps, science teachers have received considerable criticism for failing to possess adequate conceptions about the nature of science (nos) itself, especially the tentativeness of scientific knowledge (lederman, 1992). specifically, pre-service elementary teachers tend to hold a positivist view of knowledge, a lens through which they perceive that science, as a body of empirical knowledge about the world around us, is absolute (southerland & gess-newsome, 1999). this is a critical problem within the viewpoint of constructivist philosophy, and inclusive science teaching. the most fundamental assumption of constructivism is that learners come to the classroom with prior concepts, and teachers must identify these concepts, and construct new knowledge from such existing points (hodson, 1988). some studies identify solutions to help teachers understand the nos better by addressing it explicitly (abd-el-khalick & lederman, 2000; bell, matkins, & gansneder, 2011), by educating novice science teachers who have not yet constructed solid conceptual frameworks about science teaching (brickhouse, 1990), and finally by exposing pre-service teachers exhaustively to the nos in science methods courses (abd-el-khalick & lederman, 2000; southerland & gess-newsome, 1999). therefore, the study has been designed from the outset to determine whether pre-service elementary teachers can adequately rectify their nos concepts by taking an inquiry-based science methods course. the study explored what changes the pre-service teachers made after engaging in a semester-long science methods course, in terms of their views of the nos. seven characteristics of the scientific views that the study used as a coding scheme were adopted from abd-el-khalick and lederman’s (2000) study: scientific knowledge is: (a) tentative (subject to change); (b) empirically-based (based on and/or derived from observations of the natural world); (c) subjective (theory-laden); (d) partially based on human inference, imagination, and creativity; and (e) socially and culturally embedded. two additional important aspects are the distinction between observation and inference, and the functions of, and relationship between scientific theories and laws (p. 1063). the following three questions are the main foci of the study; 1. what views of science does eed 420 impart to pre-service elementary teachers? 2. as a result of eed 420, what changes have they made in terms of their nos views? 3. what kinds of nos characteristics did teachers incorporate into their lesson plans and peer teachings? methods participants twenty-one pre-service teachers, fifteen female and six male, who enrolled in eed 420: elementary science methods, managements and assessments, at an accredited, state university, participated in this study. eed stands for elementary education major, and the three digits, 420, followed by the letter prefix, is course number given assigned to this program of study which is available to senior level undergraduate students. all the participants in the elementary teacher education program were taking eed 420 as a required course, and were interning at their own placement schools as student teachers during the data collection period of this study. their intern schools were varied from k-7. exploring impacts of the eed 420 science methods course / bang 221 context the science methods course. eed 420, in which the study was conducted, was an inquirybased science method’s course. the main emphasis of eed 420 rests on developing a community of active learners, and designing a student-centered and inquiry-based curriculum. throughout the fifteen weeks of the course, eleven explicit—and implicit—nos activities were implemented, in order for pre-service teachers to better understand the characteristics of the nos views in play (see table 1). among the activities, there were two long-term projects where participants observed seed germinations and the moon every day, while recording their observations as data. all participants were asked to abstain from looking at any references, so that they might obtain specific facts regarding the germination of seeds, or the phases of the moon during this project. therefore, when they consolidated their data with others, they had to use their personally obtained data in order to answer the various questions generated. most importantly, their answers could be either posited in either scientific or non-scientific terms, when they reported results to the class, and it was totally at the community’s discretion to be satisfied or insufficiently satisfied with their findings—or to direct them to pursue more definitive evidence. the instructor personally did not deliver any facts or “right answers.” table 1. overview of eed 420 and data gatherings week nos topics long-term projects general agenda data collection w1 inquiry cubes mystery tubes intro to concept maps 5e model of instruction concept map of history of physical science vnos the card exchange w2 buttons, leaves, & rocks mystery tracks (observations & inferences) direct, guided, and open inquiry classification advance organizers w3 best paper towel (controlled investigation) plant log seed germination (descriptive investigation) light vs. without light (controlled investigation) inductive and deductive reasoning w4 mystery bones fossils standards grouping techniques material management misconceptions safety rules form a group for 5e lesson plan & placement teaching w5 never cry wolf by farley mowat science in social & personal perspective concept maps 5e model of instruction aligning objectives & assessments w6 assessment techniques questioning skills talking science modeling (day and night) exploring impacts of the eed 420 science methods course / bang 222 table 1 (continue). overview of eed 420 and data gatherings week nos topics long-term projects general agenda data collection w7 multicultural aspect of science (native indian case study) vee map--how to interpret data (plant logs) moon chart starting moon observation 5e model of instruction experience (magnets) lost on the moon w8 exam 1 w9 spring break w10 batteries & bulbs hypothesis pictorial presentations poster assessment (circuits) w11 galileo’s story argumentation: ptolemy vs. copernicus peer teaching & placement teaching lesson plan, observation notes w12 how to interpret data (moon charts) vee map moon modeling lesson plan, observation notes w13 facts, laws, & theories (adhesive & cohesive) 10 myths about science vnos the card exchange w14 presentation (placement teaching experience) field trip (mars space flight facility center on campus) w15 exam 2 the contents, such as day & night, and series and parallel circuits, have been used for primarily utilitarian purposes. however, the participants were encouraged to investigate these more thoroughly after each concept had been introduced. two activities on the nos were explicitly presented to the participants. these included the multicultural aspects of science, including native american worldviews, and the tentativeness of science as illustrated in galileo’s story. the instructor purposefully tried to focus pre-service teachers’ attentions on specific nos characteristics through these activities. with the remaining activities, the nos was implicitly introducedwhere all participants were directed to exploring impacts of the eed 420 science methods course / bang 223 address certain characteristics of the nos, but the instructor did not explicitly introduce what those were. an example of the 5e instructional model: batteries and bulbs. 5e stands for engage, explore, explain, elaborate, and evaluate. this model provided the pre-service elementary teachers a structure for the inquiry process during the eed 420 class. the 5e instructional model is also considered as one of the inquiry-based methods of instruction, in that it is consistent with the way people spontaneously construct knowledge (bybee et al., 2006). for instance, when the topic of any given week was “batteries and bulbs,” pre-service elementary teachers began talking about the flow of electricity, ohm’s law, types of circuits, etc. while they were modeling the five stages. student interest and prior knowledge can be exposed by offering the brief theoretical dilemma below. three campers had strayed deep into the woods, far from their campsite. night had fallen, and they had no flashlight to find their way back in the darkness. however, one camper had a spare battery in his backpack, another had a flashlight bulb, and a third had a few pieces of copper wire. unfortunately, they did not know how to connect the battery, bulb, and wire to light the bulb (bass, contant, & carin, p. 99). pre-service elementary teachers will be given a core question, share some safety issues, and form collaborative learning roles (e.g. facilitator, recorder, reporter, materials manager, and time keeper). during the exploration stage, students are given time and materials to undergo a physical, hands-on experience to find ways in which to electrify/light a bulb, manipulate materials, make observations, and collect and analyze datawhile the teacher acts as an observer, a guide, and a question-poser. then, the students use the data they have gathered during the exploration to explain the idea, concept, or phenomenon they have been examining, using inductive reasoning, before the teacher helps the students learn about the core, and related scientific concepts about electricity. during the elaboration stage, the preservice elementary teachers were asked to design a device that used a closed circuit with a switch. they were required to describe its name, use, and arrangement, make a diagram, and prepare a story to share their experience with the experiment. this is the stage where the teacher provides an opportunity for students to apply science concepts learned during the previous stages. during the last stage, the teacher uses performance-based assessments to evaluate student learning. an example of the using the history of science. after pre-service elementary teachers completed their own moon observations over the semester, and learned about the relationship between the sun, the earth, and the moon, using the 5e instructional model, they read the true-life story of galileo galilei’s astonishing experiences from the book entitled galileo’s daughter (sobel, 2000). then, they shared what they learned from these resources: the class constructing a short play, which was 5 to 7 minutes long. the instructor provided a basic script, which pre-service elementary teachers could add to; or had them write their own complete script, depending on the grade level assigned to them as a group. each group member had one of the following roles: director, writer (two), plot developer, costume artist (three), make-up artist, actor, or actress. they then presented their short play to the class, demonstrating the most intriguing part of galileo’s story. exploring impacts of the eed 420 science methods course / bang 224 figure 1. two classroom artifacts illustrating the engagement stage (left), and the elaboration stage (right). figure 2. a scene of the trial of galileo played by pre-service elementary teachers. after the entire experience, pre-service elementary teachers were involved in classroom discussions related to the aspects of nos (e.g. how did he develop a spyglass based on knowledge established by other scientists? how did galileo collect data and how did he use these data?) help us!! exploring impacts of the eed 420 science methods course / bang 225 data/ analysis there are four types of qualitative data from the study, 1) pre and post card exchange game results, 2) pre and post answers from the vnos questionnaire-form b and c, 3) class room artifacts such as the 5e lesson plans, 4) and finally, observational notes of peer teachings. during week 1, participants were asked to complete the card exchange game which was adopted from cobern and loving (1998). three sets of cards (53 cards per set) were distributed to the class, and each person received about six or seven cards. they then started the card game as explained in cobern and loving’s paper (1998). this activity had a two-fold impact. first, the pre-service elementary teachers were naturally guided to the hps, and secondly they realized that each held different definitions about what science isthus, they had to negotiate. they were asked to perform this activity again during week 13. second, the “view of the nature of science” questionnaire (vnos), was administered both before and after the course. the questionnaire was adopted and modified from the vnos form b (seven questions), and c (ten questions), (lederman, abd-el-khalick, bell, & schwartz, 2002). for the purpose of the study, only eight open-ended questions from these two forms were used. finally, the aforementioned seven aspects of the nos were assessed. the researchers also collected the course artifactsnamely, the six groups’ lesson plans and the field notes of six groups’ peer teachingsin order to determine what aspects of the nos had been implemented, either implicitly or explicitly (abd-el-khalick, bell, & lederman, 1998). informal interviews were frequently conducted throughout the course by the researchers. results the card exchange game in order to answer the first question of the study, what views of science does eed 420 impart to pre-service elementary teachers? changes were made as a result of eed 420. most participants relinquished their previous views and theoretical emphasis, and moved mainly toward empirical emphasis. in short, the pre-service teachers initially perceived science as a rationalistic and theory-driven endeavor, then started to view science as a data gathering experimental endeavor in pursuit of physical evidence. seven out of eight pre-service teachers who initially held a balanced view of science changed their views after the eed 420 course. finally, only one pre-service teacher held a “cultural view” after the course. table 2. the results of the card exchange game categories pre (n = 21) post (n = 18) theoretical emphasis: science is primarily a rationalistic, theory-driven endeavor. 13 0 empirical emphasis: science is primarily a data gathering experimental endeavor in pursuit of physical evidence. 0 16 anti-science view: science is overrated. one should not give much credence to the aims, methods, or the results of science. 0 0 scientism: science is the way of knowing; it is the perfect discipline. 0 0 cultural view: science is embedded in a social, historical, and psychological context which affects all that goes on in science. 0 1 balanced view: science is a complicated affair that cannot easily be reduced to one or even a few simple descriptions. 8 1 the main categories and its definitions have been adopted from cobern & loving’s study (1998, p. 76). exploring impacts of the eed 420 science methods course / bang 226 the views of nature of science the second question was to find out what changes were made after eed 420 in terms of the nos. the study failed to find any conspicuous changes in the nos. the following are the aspects where the majority held normative views of the nos, the empirical nature of scientific knowledge, the nature of scientific theories, the subjectivity in science, and related social and cultural influences. although the pattern among these aspects was similar before and after eed 420, some aspects were normatively broadened, whereas some aspects were non-normatively skewed. none of them understood the relationship between “theories” and “laws” correctly. this was the aspect that showed prominent misconceptions, followed by a creative element in science. the following are the results of each nos aspect (see table 3). table 3. pre-service elementary teachers’ views of nature of science nos aspects pre (n = 21) post (n = 18) % % empirical nature of scientific knowledge observations used to make scientific claims 71.43 82.35 science does not rely solely on empirical evidence 4.76 0.00 supports rather than proves scientific claims 4.76 5.88 n/a 14.29 11.76 nature of scientific theories theories change due to new evidence 95.24 58.82 theories change due to new ways of looking at existing evidence 9.52 35.29 theories do not change (naive nos view) 0.00 5.88 explanatory power of scientific theories 0.00 0.00 scientific theories vs. laws nonhierarchical relationship 4.76 (wm)* 5.88 (wm)* hierarchical relationship (naive nos view) 9.52 23.53 laws may change 0.00 5.88 (wm)* laws are proven and cannot change (naive nos view) 76.19 64.71 n/a 4.76 0.00 creativity in science creativity permeates scientific processes 66.67(wm)* 52.94(wm)* inferential nature of science 0.00 5.88 no single scientific method 0.00 0.00 no creativity needed in science (naive nos view) 9.52 0.00 n/a 4.76 5.88 exploring impacts of the eed 420 science methods course / bang 227 table 3 (continue). pre-service elementary teachers’ views of nature of science nos aspects pre (n = 21) post (n = 18) % % subjectivity in science (theory-ladenness) differences in data interpretation 95.24 94.12 science is necessarily a mixture of objective and subjective components 4.76 0.00 different environments offer different data (naive nos view) 4.76 5.88 social & cultural influences science as a cultural within itself 42.86 58.82 peer review limits subjectivity 0.00 0.00 society as an influence on science 14.29 5.88 both (universal & social) and (naive nos view) 9.52 17.65 science is universal (naive nos view) 19.05 11.76 n/a 14.29 5.88 the categories have been adopted and modified from the lederman et al.’s study (2002, p.506). * wm: with misconceptions. empirical nature of scientific knowledge. most of the participants held normative views of the nos in this aspect; however, the only attention they gave included observations. pre and post vnos showed similar patterns. nature of scientific theories. before eed 420, all pre-service teachers thought that theories changed due to new evidence or due to the development of technology. there was only one participant who indicated that changes could result from observations being made differently. however, after eed 420, five participants thought that theory could change simply by thinking about it differently. there was one pre-service teacher who thought that theory did not change. however, that teacher failed to provide normative rationales for her assertion. linda: yes, even though theories have data and evidence to support them, they can change as we discover new ideas, things, and thoughts (pre-vnos). linda: yes, so others can either develop the theories further or think of a new theory to discredit or change the other theory (post-vnos). scientific theories vs. laws. there were conspicuous misconceptions surrounding the relationships between theories and laws. none of the teachers articulated this aspect of the nos correctly. although, some pre-service teachers thought that theories and laws were nonhierarchical, and that laws may change, their rationales were all non-normative. exploring impacts of the eed 420 science methods course / bang 228 anne: yes, there is a difference. scientific theory is like a “best guess.” for example, a theory can change with new discoveries. a scientific law is what we use to create theories. we may use many laws to create a theory (pre-vnos). anne: yes, a theory is something proven and always the same. scientific laws can vary in the outcome (post-vnos). creativity & inference in science. most of the pre-service science teachers indicated that creativity permeates scientific processes. however, some of them limited the use of creativity to during and after the data collection period during experiments, or to certain fields of science, such as astronomy. this misconception increased after the course. two pre-service teachers who thought that there was no creativity needed in science, changed their views of the nos to those classified as “normative” after eed 420. katie: yes, scientists use creativity and imagination during and after data collection...they must think about new ways to research and seek understanding (pre-vnos). sam: i feel that after the experiments/observations are complete, there is not much more room for creativity, because the investigation either proves or disproves the theory it is testing. so, there is little left to the imagination (pre-vnos). sam: yes, they use creativity and imagination, because if they didn’t, then we wouldn’t have theories and laws today (post-vnos). subjectivity in science (theory-ladenness). the majority of the pre-service elementary teachers held normative views of the nos, in terms of its theory-ladenness before and after the course. they thought that each scientist had a different opinion about the meaning of the data. social & cultural influences. approximately over half of the participants espoused the social and cultural aspects of the nos. some thought that there was a universal science, while some believed that both universal and socially and/or culturally influenced sciences could exist at the same time. dale: i believe both to be true. there are those universal theories that cannot be affected by society. then there are others that are clearly affected by society and culture (pre-vnos). ellen: i believe that science is universal. nature does not change according to social and cultural values, but the people might. for example, a flower grows in one place, just like it would in another place. the only difference is the way people interpret science (pre-vnos). sam: i believe science is universal because all cultures have some kind of scientific knowledge and belief. science is known throughout the world, and is not limited to one culture (post-vnos). finally, the study analyzed the teachers’ lesson plans and peer-teaching observation field notes, in order to answer the third question: “what kinds of nos characteristics did teachers incorporate into their lesson plans and peer teachings?” evidence from the lesson plans and field notes identified that none of the groups explicitly designed or taught their lessons. although their peer teachings mostly demonstrated either directed or guided inquiry, the aforementioned seven aspects were barely implemented into observed lesson plans and instructions. however, creativity in science and the empirical nature of science were implicitly found in two peer teachings. furthermore, all of the lessons and peer teachings placed a disproportionate emphasis on scientific process skills. exploring impacts of the eed 420 science methods course / bang 229 conclusions the study found three important messages by using extensive qualitative data. first, as a result of the two long-term projects in which the pre-service elementary teachers kept plant and moon-phase logs over a one-month period, their views were moved toward empirical emphasis. science was defined as a data gathering process, an experimental endeavor in pursuit of physical evidence. the two long-term projects were designed to implicitly impart the empirical nature of science. pre-service teachers were able to unpack related nos elements through these activities. this is an unusual finding, since explicitly taught nos was previously found to be effective (abd-el-khalick & lederman, 2000). the study asserts that if implicitly designed activities were exposed to learners for a sufficiently long period of time, they can be effective in teaching nos aspects. however, having two explicit activities and nine implicit activities, including the aforementioned long-term projects, did not result in any conspicuous differences between groups. although some progress was found, evidence showed that there was scarcely any impact of eed 420 on enhancing the understandings of nos for the pre-service elementary teachers. surprisingly, none of them adequately understood the normative aspects about the relationship between theories and laws and furthermore, demonstrated significant misconceptions as well. this finding is similar to those of abell, martini, and george (2001). they explored pre-service elementary teachers who were involved in the moon-phase observations for six weeks, while they were exposed to explicit nos teaching activities. they found that such long-term activities provided opportunities for teachers to understand that scientific knowledge is empirically based and socially embedded. yet, they failed to find evidence of pre-service teachers being aware of the fact that scientists are also involved in the conception and innovation of theories. the results of the study also urge teacher educators to constantly reflect upon the aspects of their curricula that address reform-based science. the study also suggests that teacher educators should establish frameworks for their science education curricula that include the following elements; 1) teaching science through inquiry practices, 2) a variety of authentic assessments, 3) the social context of science teaching and 4) the social context of the science itself (abell, martini, & george, 2001). it should be mentioned that riedinger, marbach-ad, mcginnis, hestness, and pease (2011) also suggested that the inclusion of innovative informal science activities within the science methods curriculum helped pre-service elementary teachers understand the normative views of science. upon exploring the areas where the pre-service elementary teachers were implementing the nos concepts, the study failed to identify any lesson plans or peer teachings that espoused nos elements explicitly. however, creativity in science and the empirical nature of science were found in two peer teachings. therefore, the results of this study reinforce the findings of other studies that conclude that a single semester-long science methods course will not adequately improve teachers’ understanding of the nos. furthermore, a single course failed to impart a sufficiently robust desire within teachers for them to adequately implement elements of the nos into their lessons. . . . acknowledgements the authors of the paper would like to extend a sincere thank you to all the pre-service teachers who participated in this study and all the pre-service teachers who shared their pictures. a special thank exploring impacts of the eed 420 science methods course / bang 230 you also goes to sara nelson and rhonda berry, who helped the author throughout the collection of articles for the literature review. eunjin bang is an assistant professor at iowa state university. she received her ph.d. in science education from arizona state university in 2008. she specializes in the development of mentoring programs for elementary science teachers, with a specific focus on online programs and reformedbased science practices and identity formation. she has been teaching pre-service elementary teachers for nine years. references abd-el-khalick, f., bell, r. l., & lederman, n. g. (1998). the nature of science and instructional practice: making the unnatural natural. science education, 82, 417-436. abd-el-khalick, f. & lederman, n. (2000). the influence of history of science course on students' views of the nature of science. journal of research in science teaching, 37, 1057-1095. abell, s., martini, m., & george, m. (2001). 'that's what scientists have to do': pre-service elementary teachers' conceptions of the nature of science during a moon investigation. international journal of science education, 23(11), 1095-1109. bass, j., contant, t., & carin, a. (2009). teaching science as inquiry. (11 ed.). boston, ma: pearson education, inc. bell, r. l., matkins, j. j., & gansneder, b. m. (2011). impacts of contextual and explicit instruction on preservice elementary teachers’ understandings of the nature of science. journal of research in science teaching, 48(4), 414-436. brickhouse, n. (1990). teachers' beliefs about the nature of science and their relationship to classroom practice. journal of teacher education, 41, 53-62. bybee, r., taylor, j. a., gardner, a., van scotter, p., carlson, j., westbrook, a., landes, n. (2006). the bscs 5e instructional model: origins and effectiveness. colorado springs, co: bscs. cobern, w. & loving, c. (1998). the card exchange: introducing the philosophy of science. in w. f. mccomas (ed.), the nature of science in science education (pp. 73-82). cordrecht, the netherlands: kluwer academic publishers. gardner, p. l. (1972). structure-of-knowledge theory and science education. educational philosophy & theory, 4, 25-46. hainsworth, m. d. (1956). the effect of previous knowledge on observation. school science review, 37, 234-242. hodson, d. (1988). experiments in science and science teaching. educational philosophy and theory, 20, 52-65. lederman, n. g. (1992). students' and teachers' conceptions of the nature of science: a review of the research. journal of research in science teaching, 29, 333-359. lederman, n. g., abd-el-khalick, f., bell, r. l., & schwartz, r. s. (2002). views of nature of science questionnaire: toward valid and meaningful assessment of learners' conceptions of nature of science. journal of research in science teaching, 39, 497-521. matthews, m. r. (1994). the rapprochement between history, philosophy, and science education. in science teaching: the role of history and philosophy of science (pp. 1-9). new york, ny: routledge. exploring impacts of the eed 420 science methods course / bang 231 mccomas, w. f., clough, m. p., & almazroa, h. (2000). the nature of science in science education: rationales and strategies. in w. f. mccomas (ed.), the role and character and the nature of science in science education (pp. 3-39). cordrecht, the netherlands: kluwer publishers. rhodes, g. & schaible, r. (1989). fact, law, and theory: ways of thinking in science and literature. journal of college science teaching, 18, 228-232. riedinger, k., marbach-ad, g., mcginnis, j., hestness, e., & pease, r. (2011). transforming elementary science teacher education by bridging formal and informal science education in an innovative science methods course. journal of science education and technology, 20(1), 51-64. sobel, d. (2000). galileo's daughter. new york, ny: penguin books. southerland, s. & gess-newsome, j. (1999). pre-service teachers' views of inclusive science teachings as shaped by images of teaching, learning and knowing. science education, 83, 131-150. trusted, j. (1979). the logic of scientific inference: an introduction. london: macmillan. exploring impacts of the eed 420 science methods course / bang 232 www.iejee.com this page is intentionally left blank microsoft word iejee_5_3_bebetsos international electronic journal of elementary education, 2013, 5(3), 233-248. issn:1307-9298 copyright © iejee www.iejee.com relationship among students’ attitudes, intentions and behaviors towards the inclusion of peers with disabilities, in mainstream physical education classes evangelos bebetsos ∗∗∗∗ democritus university of thrace, greece vassiliki derri democritus university of thrace, greece stratos zafeiriadis democritus university of thrace, greece pavlos kyrgiridis democritus university of thrace, greece received: 9 june 2013 / revised: 23 july 2013 / accepted: 26 july 2013 abstract students’ attitudes, intentions and behaviours towards their peers with disabilities are important to their mutual co-existence and development. the aim of this study was to investigate a) whether students’ attitudes and intentions towards their schoolmates with disabilities are related to their general and modified behavior in mainstream physical education classes, and b) if the former variables could serve as predictors of the latter. the participants, 172 children without disabilities (mage=11.15, sd=.70), completed the revised version of the planned behavior theory questionnaire (pbt) and the children’s’ attitudes towards inclusion in physical education – revised questionnaire (caipe-r). although results revealed several correlations among the variables under study, only general attitudes accounted for both general and modified behavior, and attitudes for modified behavior. these findings could assist in educating students to develop and perform appropriate behaviours towards their peers in order to facilitate their co-existence, and their mutual development and learning. keywords: attitudes, general behavior, modified behavior, elementary school. ∗ evangelos bebetsos, democritus university of thrace, komotini, 69100, greece, phone: +3025310-39712fax: +3025310-39723 422-0930. e-mail: empempet@phyed.duth.gr students’ attitudes, intentions and behaviors / bebetsos, derri, zafeiriadis & kyrgirdis 234 introduction over the past years inclusion has become increasingly the focus of many national and international policies of education (armstrong, 1998). inclusion has been identified as placing students with disabilities in mainstream classes, including physical education, and educating them with their non-disabled peers (block, 2000, 2007; murata, hodge, & little, 2000). in this frame, school societies try to support full participation of students with disabilities in all areas of their lives on equal terms and conditions (campbell & gilmore, 2003). in line to this policy, the greek government voted the public law 2817/2000. based on this law, a child with disabilities can study in an ordinary school class with parallel support by the special education teacher or in specifically organized and appropriately staffed classes of inclusion, which function in the schools of mainstream and technical professional education. according to buswell and schaffner (1990), primary schools are the best way to fight discrimination since they provide students with the appropriate space and experiences to achieve the learning objectives but also to enhance their social skills. specifically, the embodiment offers many benefits to both children with and without disabilities, some of which reflect on their social development and specifically on their ability to ask, discuss, and interact with each other (chesley & calaluce, 1997; lipsky & gartner, 1997). likewise, literature indicates that all students with disabilities should experience positive interactions with their peers in the physical education class as part of their growth and development (baalpe, 1996). the endeavor towards understanding whether positive interactions and social behaviors are related to attitudes and intentions of people, in general, who have daily contact with children with disabilities resulted in the development of many models, i.e. the model of planned behavior (ajzen & madden, 1986). according to planned behavior theory (ajzen, 1987; 1988), the performance of a behavior is related but not limited to the person’s intention. although a behavior can be totally under a person’s control, in most cases various obstacles are present which impinge on the person’s decision to perform it. such obstacles can be internal factors such as agility, knowledge, and planning, or external factors such as time, opportunity, and cooperation with others (ajzen & madden, 1986). in particular, the probability of performing a specific behavior is referred to as “behavioral intention”. intention is determined by a combination of two factors: (i) attitude towards the behavior (that is, a positive or negative predisposition towards a specific behavior); and (ii) subjective norms (ajzen & fishbein, 1972). these subjective norms are of two kinds: (i) behavioral beliefs (which affect attitude towards the behavior); and (ii) normative beliefs (which reflect social factors). each behavioral belief reflects whether important others would approve or disapprove a specific behavior (tesser & shaffer, 1990). according to ajzen and fishbein (1980), the stronger the intention of a subject the greater the likelihood is that the subject will behave according to his or her intention. also, information has often been mentioned as an important factor in understanding how behavior is consistent with attitude (ajzen & madden, 1986; krosnick, boninger, chuang, berent, & carnot, 1993; theodorakis, 1994). limited information and knowledge about the behavior in question can represent a serious obstacle in carrying it out (theodorakis, 1994) and, consequently, in assessing it with accuracy. despite that, information is a construct that has not received much attention in recent research based on planned behavior theory. it is frequently reported as an important factor in the literature on attitude, but few studies provide a clear definition of it—although krosnick and his colleagues did define information (or, rather, interest in relevant information, to use their terminology) as being ‘the extent to students’ attitudes, intentions and behaviors / bebetsos, derri, zafeiriadis & kyrgirdis 235 which an individual is motivated to gather information about an attitude object’ (krosnick, et al., 1993, p. 1133). another factor that influences a person’s attitude towards a specific behavior is the perceived subjective control of this behavior that a person has. literature identifies as “perceived behavioral control” how easy or difficult it is for an individual to adopt a certain behavior (conner & norman, 1995). perceived behavioral control is influenced by internal and external factors. the internal factors include variables such as skills, abilities and individual differences. external factors include time, opportunity and dependence on others. although research has been conducted in the area of including students with disabilities in primary schools, research directly associated with physical education is fairly new (o’brien, kudláček, & howe, 2009). however, there is evidence that the inclusion in physical education can work effectively for children with disabilities (goodwin & watkinson, 2000), and this can be achieved without negative peer experience on behalf of children without disabilities (faison-hodge & porretta, 2004; obrusníková, válková, & block, 2003). this is really important since hutzler and levi (2008) suggested that most students would not consider as an outcome a behavior in which they are expected to lower their own performance in favor of the student with disabilities. besides, the claim to concede individual performance in favor of a student with disability has never been considered as an inclusion objective; on the contrary, most researchers insist that inclusion can be conducted without negative effects to the non-disabled students (block, 2007; sherrill, 2004). block and malloy (1998) investigated the attitudes of students without disabilities on the inclusion of peers with mental disabilities, in a softball team, and their opinion as to the adjustments of the exercise that needs to be done to enable disabled students to follow up. the findings were encouraging and showed that students without disabilities had positive attitudes towards the participation of children with disabilities in their group(s) with appropriate exercise adjustments. also, research evidence (hutzler, 2003) dealt with depicting barriers associated with professional, personal, and peer attitudes toward the participation of children with disabilities in physical education classes, showed that female students (block, 1995; loovis & loovis, 1997; woodward, 1995; tripp, french & sherrill, 1995; slininger et al., 2000) and individuals with a family member or a close friend with a disability (block, 1995), had more positive attitudes. at the same time, in greece limited research used the model of planned behavior in order to identify the attitudes and intentions of students and physical education teachers towards their collaboration with students with disabilities in class (batsiou, et al., 2006) and recreation environments (magouritsa, kokaridas, & theodorakis, 2005). as theodorakis et al. (1995) mentioned the main determinant of a behavior is the intention of the individual to accept to yield this behavior, which is for example to accept the presence of students with disabilities in the regular physical education class. in another study by magouritsa et al. (2005), the examination of planned behavior theory in the context of inclusion in physical activity and/or physical education showed that children’s intentions of including peers with disabilities are an outcome of their attitudes, normative beliefs, and perceptions of control or competence, during and after an activity. results also indicated that the application of an intervention program improved students’ attitudes, verifying sable’s (1995) point of view that a person may learn attitudes through social and environmental experiences. a more recent study which was based on the same theory (kalyvas, koutsouki, & skordilis, 2011) pointed out that university students exhibited positive attitudes and intentions towards integrating disabled individuals in school while female students scored higher than their male peers. on the subject of teachers’ opinion and intention (batsiou et al., 2006), students’ attitudes, intentions and behaviors / bebetsos, derri, zafeiriadis & kyrgirdis 236 results revealed that past experience, specific subject education (physical education for disabled students), knowledge and information are the main elements for more positive attitudes. although attitudes and intention may affect people’s behavior, research identified the need for developing and using specific instruments to measure behavioral outcomes. one of the most frequently used instruments to measure peers’ behavior towards inclusion in physical education is the “children’s’ attitudes towards inclusion in physical education – revised” questionnaire (caipe-r: block, 1995; block & malloy, 1998; sherill, 1998). this instrument was developed to measure how children without disabilities feel about having children with disabilities in their physical education class (block, 1995). more specifically, the particular tool investigates two factors; general and modified behaviour, through children’s beliefs. additionally, panagiotou, evaggelinou, doulkeridou, koidou, and mouratidou (2009) pointed out that students’ general behavior might change and become more positive towards integration of children with disabilities, unlike their modified behavior, where the element of winning in sports, makes them be selective towards the teammates that they want to have (in order to win). nevertheless, education and especially school curriculum subject matters like physical education play a major and very important role in changing students’ behaviors toward disability in general and disability in sports in particular, contributing this way to the successful inclusion of students with disabilities into primary schools and social communities (evaggelinou, 2006; sherrill, 1998). although the aforementioned studies provide valuable information on the subject of inclusion of disabled students in the mainstream school, they examined either students’ attitudes or behaviors. also, although students’ attitudes are considered to lead towards the performance of a behavior, there is no research evidence, to our knowledge, which proves a direct relation between attitudes and behaviors. the identification of such a relation is considered significant because attitudes could serve as a diagnostic tool for the performance of students’ behaviors towards their disabled peers in mainstream classes. such a diagnosis could assist professionals and physical education teachers in designing and implementing the curriculum, according to all students’ behavioral needs, within the dynamic, social environment of physical education. therefore, this study was conducted in an attempt to investigate (a) whether there are any relations between the variables of pbt (attitudes, intention, perceived behavioral control, moral satisfaction, information, and general attitudes), and caipe-r (general behavior, and modified behavior), and (b) the extent to which the former variables account for the latter. the study addressed the following questions: � is there any relation between the variables of pbt (attitudes, intention, perceived behavioral control, moral satisfaction, information, and general attitudes) and caipe-r (general behavior and modified behavior)? � which one(s) of the aforementioned variables of pbt account for the variables of caipe-r? students’ attitudes, intentions and behaviors / bebetsos, derri, zafeiriadis & kyrgirdis 237 method participants the sample consisted of 172 primary grade school children; 78 boys and 94 girls, between 10 to 12 years of age (m=11.15, sd=.70) (table 1). table 1. participants’ descriptive characteristics sex age grade boys 78 (45.3%) 10 31 (18%) 5th 87 (50.6%) girls 94 (54.7%) 11 85 (49.4%) 6th 85 (49.4%) 12 56 (32.6%) instruments students completed the revised version of the planned behavior theory (magouritsa et al., 2005), and the greek version of the children’s’ attitudes towards inclusion in physical education – revised questionnaire (caipe-r) (panagiotou, 2006). before completing the second questionnaire, students were introduced to a hypothetical scenario: “before we begin i would like to talk to you about a child whose name is john. john has the same age as you. but has moderate mental retardation and therefore he cannot be taught things and learn them as fast as you can. because he has moderate mental retardation, he cannot speak very clearly and that is why it is sometimes difficult to understand what he says. john likes to play the same games you play when you exercise, but he is not very good at these games. although he can run, he is slower than you and gets tired easily. he likes football but he cannot kick the ball very well. he also likes basketball but he is not very good at shooting and dribbling the ball and he cannot understand the rules of the game very well”. 1. the planned behavior theory questionnaire consists of questions of the following seven factors: a) attitudes were estimated by the mean score of responses to the question “for me to accept a student with disabilities in my class, is…”. responses were rated on a 7-point likert type scale, on six bipolar adjectives (7=good to 1=bad, 1=unethical to 7=ethical, 7=smart to 1=foolish, 7=useful to 1=unuseful, 7=nice to 1=ugly, 7=pleasant to 1=unpleasant). b) intention was estimated by the mean score of the responses to three different questions: “i intend/i will try/i am determined to accept a student with disabilities in my class” were rated on a 7-point scale from 1=very unlikely to 7=very likely. a 7-point likert type scale with endpoints labelled 1=definitely no to 7=definitely yes, was used for the other two questions. c) perceived behavioral control for the specific behavior was estimated by the mean score of four questions. examples of questions are: “for me to accept a student with disabilities in my class is”, “if i wanted i could accept a student with special needs in my class”, “is totally up to me, if i will accept or not a student with disabilities in my class”, and “how much is under your control, to accept or not a student with special needs in your class?”. a 7-point likert type scale was used, ranging from 1=difficult to 7=easy for the first question, from 1=incorrect to 7=correct for the second, from 1=disagree to 7=agree for the third and 1=not at all to 7=complete control for the forth. students’ attitudes, intentions and behaviors / bebetsos, derri, zafeiriadis & kyrgirdis 238 d) moral satisfaction was estimated by the mean score of three questions: “i wouldn’t feel guilty if i didn’t accept a student with special needs in my class”, “to not accept in class a student with special needs, is against my principles”, and “it would be unethical to me, if i wouldn’t accept a student with special needs in my class”. a 7-point likert type scale was used, ranging from 1=incorrect to 7=correct for the first question, from 1=impossible to 7=possible for the second, and from 1=disagree to 7=agree for the third question. e) subjective norms were estimated by the mean score of responses to four questions: “some individuals, who are important in my life, believe that i must accept a student with disabilities in my class”, “some very important people to me, would accept a student with disabilities in the class”, “some people to whom i value their opinion, would agree the idea of having a student with disabilities in my class”, and “some very important people would approve the idea of having a student with special needs in my class”. a 7-point likert type scale was used, ranging from 1=i must not to 7=i must for the first question, from 1=incorrect to 7=correct for the second and forth, and from 1=disapprove to 7=approve for the third question. f) information was measured by four questions: “some individuals told me that they pay attention to different information about inclusion of students with disabilities to regular classrooms. how much attention do you pay to different information about inclusion of students with disabilities to regular classrooms?”; “how often do you pay attention to different information about inclusion of students with disabilities to regular classrooms?”; “i am very interested in any information regarding the inclusion of students with disabilities to regular classrooms”; “how often do you pay attention to information regarding inclusion of students with disabilities to regular classrooms?”. responses were given on 7-point likert type scales, ranging from 1=i never pay attention to 7=i very much pay attention for the first, from 1=never to 7=very often for the second, from 1=i strongly disagree to 7=i strongly agree for the third, and from 1=i never pay attention to 7=i pay a lot of attention for the fourth question. g) additionally, a factor named general attitudes was added to the questionnaire (nikolaraizi & reybekiel, 2001) which was translated into greek by magouritsa, kokaridas and theodorakis (2005). it was measured by nine questions in an effort to explore not only the attitude of the students’ planned behavior in school activity, but the overall students’ attitude without disabilities and their willingness to approach, build a relationship, and accept students with disabilities in their classroom. before the questions, the students were presented with a scenario: “lets hypothesize that a student with disabilities comes in your class for this school year”, and then they were asked: “will you have him/her become your best friend?”. responses were given in a 5-point likert type scale, ranging from 1=definitely not to 5=definitely yes. 2. the (caipe-r) questionnaire consists of eleven questions that concern general and modified behavior. specifically, general behavior is measured with six questions (e.g., “it would be ok having john come to my physical education class”), whereas the remaining five questions concern modified behavior (e.g., “if you were playing basketball would you be willing to make a pass to john?”). a 4-point likert type scale, from 1=no to 4=yes, was used. at the end of both questionnaires the students were asked to indicate their sex, age and grade. responses were given in a numerical format. their participation was voluntary. data analyses person correlation analyses were conducted to identify any possible correlation between the seven variables of the pbt and the two variables of caipe-r. also, hierarchical regression students’ attitudes, intentions and behaviors / bebetsos, derri, zafeiriadis & kyrgirdis 239 analyses were used to identify whether pbt’s variables account for general behavior and modified behavior of caipe-r. results descriptive statistics descriptive statistics were computed for all assessed variables and are presented in table 2. the results indicated that all scales showed acceptable internal consistency since cronbach’s alpha was higher than .70. table 2. internal reliability and descriptive statistics of all variables. variable m sd cronbach’s alpha attitudes 5.6 .96 .80 intention 5.9 1.2 .74 perceived behavioral control 5.3 1 .87 moral satisfaction 4.2 1.8 .71 subjective norms 6 1 .76 information 4.6 1.2 .86 general attitudes 3.9 1.1 .78 general behavior 3.8 1.3 .71 modified behavior 4.1 1.5 .77 pearson correlations table 3 shows the pearson correlations between the variables of pbt questionnaire and the variables of caipe-r questionnaire. specifically, general behavior was significantly correlated with general attitudes and moral satisfaction whereas modified behavior was significantly correlated with general attitudes, attitudes, intention, perceived behavioral control, subjective norms, and information. table 3. pearson correlation matrix among questionnaire for planned behavior model and caipe-r. variables general behavior modified behavior 1. general attitudes .47* .24* 2. attitudes n.s. .27* 3. intention n.s. .22* students’ attitudes, intentions and behaviors / bebetsos, derri, zafeiriadis & kyrgirdis 240 table 3 (continue). pearson correlation matrix among questionnaire for planned behavior model and caipe-r. variables general behavior modified behavior 4. perceived behavioral control n.s. .27* 5. moral satisfaction .27* n.s. 6. subjective norms n.s. .24* 7. information n.s. .20* *p<.001 hierarchical regression analyses 1. results from hierarchical regression analysis that concern general behavior are presented in table 4. in the analysis, general attitudes was entered at step 1; attitudes was entered at step 2; intention was entered at step 3; perceived behavioral control were entered at step 4; moral satisfaction was entered at step 5; subjective norms was entered at step 6; and information was entered at step 7. only the variable general attitudes, in step 1, significantly accounted for the 22% of the total variance of general behavior, r2 change=.22, f(1,17)=46.35, p<.001. overall, the variables accounted for the 24% of the total variance of general behavior. table 4. hierarchical regression analysis for general behavior. variables entered b β r2 change se b step prediction of “general behavior” 1 general attitudes 0.32 .48* .22 .05 2 general attitudes attitudes 0.33 0.02 .48 .03 n.s. .05 .03 3 general attitudes attitudes intention 0.32 0.01 0.03 .47 .08 .10 n.s. .05 .04 .03 4 general attitudes attitudes intention perceived behavioral control 0.32 0.04 0.04 0.01 .47 .08 .10 .01 n.s. .05 .04 .03 .03 students’ attitudes, intentions and behaviors / bebetsos, derri, zafeiriadis & kyrgirdis 241 table 4 (continue). hierarchical regression analysis for general behavior. variables entered b β r2 change se b step prediction of “general behavior” 5 general attitudes attitudes intention perceived behavioral control moral satisfaction 0.29 0.03 0.03 0.001 0.03 .42 .07 .10 .001 .13 n.s. .05 .04 .03 .03 .02 6 general attitudes attitudes intention perceived behavioral control moral satisfaction subjective norms 0.29 0.04 0.03 0.004 0.03 0.02 .42 .08 .08 .001 .13 .05 n.s. .05 .04 .03 .03 .02 .04 7 general attitudes attitudes intention perceived behavioral control moral satisfaction subjective norms information 0.29 0.04 0.03 0.007 0.03 0.02 0.01 .41 .08 .08 .02 .14 .05 .03 n.s. .05 .04 .03 .03 .02 .04 .03 *p<.001 2. results from hierarchical regression analysis that concern modified behavior are presented in table 5. in the analysis, general attitudes was entered at step 1; attitudes was entered at step 2; intention was entered at step 3; perceived behavioral control was entered at step 4; moral satisfaction was entered at step 5; subjective norms was entered at step 6; and information was entered at step 7. in step 1, general attitudes significantly accounted for the 6% of the variance of modified behavior, r2 change=.06, f(1,17)=10.57, p<.001, and in step 2 attitudes increased significantly its variance to 12%, r2 change=.06, f(2,17)=10.72, p<.001. overall, the variables accounted for the 15% of the total variance of modified behavior. students’ attitudes, intentions and behaviors / bebetsos, derri, zafeiriadis & kyrgirdis 242 table 5. hierarchical regression analysis for modified behavior. variables entered b β r 2 change se b step prediction of “modified behavior” 1 general attitudes 0.20 .24* .06 .06 2 general attitudes attitudes 0.17 0.13 .21 .24* .06 .06 .04 3 general attitudes attitudes intention 0.17 0.11 0.3 .20 .20 .07 n.s. .6 .5 .4 4 general attitudes attitudes intention perceived behavioral control 0.15 0.9 0.02 0.6 .18 .17 .04 .13 n.s. .06 .05 .04 .04 5 general attitudes attitudes intention perceived behavioral control moral satisfaction 0.15 0.09 0.02 0.06 0.003 .18 .16 .04 .13 .01 n.s. .07 .05 .04 .04 .02 6 general attitudes attitudes intention perceived behavioral control moral satisfaction subjective norms 0.15 0.07 0.001 0.05 0.001 0.06 .18 .14 .003 .11 .003 .12 n.s. .07 .05 .04 .04 .02 .05 7 general attitudes attitudes intention perceived behavioral control moral satisfaction subjective norms information 0.14 0.07 0.000 0.04 0.008 0.07 0.05 .17 .13 .000 .08 .03 .13 .10 n.s. .07 .05 .04 .04 .02 .05 .04 *p<.001 students’ attitudes, intentions and behaviors / bebetsos, derri, zafeiriadis & kyrgirdis 243 discussion the aim of the study was twofold: a) to investigate any possible relations between the variables of pbt (attitudes, intention, perceived behavioral control, moral satisfaction, information, and general attitudes) and caipe-r (general behavior, and modified behavior), and b) to investigate to which extent the former variables account for the latter that is to children’s general and modified behaviour (accepting and interacting with disabled students) in their mainstream physical education class. to our knowledge, no similar studies have been conducted on the specific topic either in physical education or other courses. therefore, discussion and conclusions from the present study reflect a first attempt to interpret the relation of attitudes, intentions and behaviours of primary students towards the integration of disabled students in a mainstream physical education class. with regard to the first question of the study, results proved that general behaviour was significantly related to two of the seven pbt variables; general attitudes and moral satisfaction. in contrast, moral satisfaction was the only unrelated variable to modified behaviour. this really interesting but contradictory finding may be attributed to the fact that social admission to children of a certain age group mainly is based on their athletic ability (ellery & rauschenbach, 2000; jesina, kudlacek, janecka, machova, & wittmannona, 2006; panagiotou et al., 2009; van biesen, busciglio, & vanlandewijck, 2006). as a result, children might believe that it is moral to interact with their disabled peers (general behaviour), but when it comes to the point of performing well or even winning in a movement activity or game, they want a teammate that can execute as good as possible (modified behaviour). nevertheless, the physical education goals (movement, cognitive, health-related fitness, and emotional/social) are strongly interrelated, and should be achieved by all students, regardless of their differences. to this end, individual improvement and social interaction are considered important elements (derri, 2007; ministry of education, lifelong learning and religious affairs, 2011a, b). although modified behavior was unrelated to moral satisfaction, it was related to the rest of the pbt variables; attitudes, intention, perceived behavior control, subjective norms, information, and general attitudes. this result verifies the perspective that even though attitudes and intention are related to the performance of a behavior (ajzen, 1987; 1988), there are other aspects such as subjective norms and perceived behavioral control that also influence that behavior (ajzen & madden, 1986). moreover, the relation between modified behavior and information is consistent with previous research (davidson, et al., 1985; wilson, et al., 1989; krosnick et al., 1993; theodorakis, 1994; bebetsos & antoniou, 2004) which indicated that the amount of information available about an attitude could be a determinant of the attitude-action consistency. finally, results showed that general attitudes are related to both general and modified behavior. in line to the planned behavior theory and previous research (magouritsa et al., 2005), the final decision of the students to accept disabled peers in the physical education class seems to be a) influenced by their general attitudes on the subject of inclusion, and b) largely determined by their modified behavior (which is directly linked on how easy or difficult they think it is to perform this task). nevertheless, when hierarchical chains of the variables were performed in order to answer the second study question, general attitudes played the most important role for both analyses, with the addition of attitudes on modified behaviour. researchers speculate that the nature of the questions initiated these results; the variable of general attitudes consists of questions that are generalized on students’ interaction with a disabled peer in a physical education class, while the variable of attitudes concerns the specific involvement and interaction of students with and without disabilities, within a physical education class. students’ attitudes, intentions and behaviors / bebetsos, derri, zafeiriadis & kyrgirdis 244 it is evident that general attitudes of non-disabled students play the most important role in predicting both their general and modified behaviour towards their disabled peers, in the physical education class. therefore, general attitudes could be used as a diagnostic tool in terms of unveiling non-disabled students’ corresponding behaviors. on the other hand, taking into account the percentage of explained behavior by general attitudes, it seems that other factors play also an important role on student behaviour. such factors could concern teacher attitudes and past experience, student education, appropriate school facilities, opinion of other important others, information etc.). former bibliographical research supports the importance of these factors (ajzen & madden, 1986; anderson, klassen, & georgiou 2007; batsiou et al., 2006; rose, 2001; smith & smith, 2000). although the study provides useful information for the schedule and the implementation of a physical education lesson that is sensitive to the inclusion of students with disabilities, it focused on children with moderate mental retardation. therefore, its results concern only this type of disability. also, students’ responses were based on a hypothetical scenario. a real setting scenario with john in their class might produce different answers with regard to their attitudes and behaviors towards him. conclusion in this study non-disabled students’ attitudes were proved powerful in predicting their behaviour towards their disabled peers in the physical education class. therefore, physical education teachers should evaluate students’ attitudes, and take them into account during planning and teaching in order to provide all students with the appropriate learning experiences, and assist them in achieving the lesson goals. students’ specific training could improve their general attitudes and consequently generate more positive general and modified behaviors towards their peers. also, the enhancement of teachers’ knowledge and skills on the subject of inclusion in physical education, in general, and on attitudes and behaviors, in specific, could further assist the above endeavor and produce the desirable outcomes for all students. further research is required to evaluate the relation between students’ attitudes and behaviors, by observing them during the conduction of the lesson. also, this relation should be further studied for different types of disability in order to provide a clear understanding of students’ attitudes and behaviors towards their disabled peers. finally, future studies could investigate the effect of gender and students’ previous experience with individuals with a disability on their attitudes and behaviors. . . . evangelos bebetsos ph.d., is an assistant professor in the department of physical education and sport science at democritus university of thrace in greece. his main area of interest and research is sport and exercise psychology. he teaches the undergraduate courses of sport psychology, applied sport psychology on performance maximization, rehabilitation counseling, badminton, as well as the graduate courses of developmental psychology and rehabilitation counseling. vassiliki derri̇ ph.d., is an associate professor in the department of physical education and sport science at democritus university of thrace in greece. she teaches the undergraduate courses of elementary physical education, pedagogy, preschool physical education, and assessment in physical education as well as the graduate courses elementary physical education, and abilities and skills in the developmental ages. she collaborates with the institute of educational policy of the ministry of students’ attitudes, intentions and behaviors / bebetsos, derri, zafeiriadis & kyrgirdis 245 education, lifelong learning and religious affairs on physical education teachers’ and counselors’ professional development/training. stratos zafei̇ri̇adi̇s, is a physical education teacher with a master’s degree from department of physical education and sport science at democritus university of thrace in greece. pavlos kyrgi̇ri̇di̇s, is a physical education teacher with a master’s degree from department of physical education and sport science at democritus university of thrace in greece. he is also a ph.d. candidate at the same department. references ajzen, i. (1987). attitudes, traits and actions: dispositional prediction of behavior in personality and social psychology. in: l. berkowitz (ed.), advances in experimental social psychology (pp. 1-63). new york: academic press. ajzen, i. (1988). attitudes, personality, and behavior. bristol: open university press. ajzen, i. & fishbein, m. (1972). attitudes and normative beliefs as factors influencing behavioral intentions. journal of personality and social psychology, 21, 1-9. ajzen, i. & fishbein, m. (1980). understanding attitudes and predicting social behavior. englewood cliffs, nj: prentice-hall. ajzen, i., & madden, t. (1986). prediction of goal directed behavior: attitudes, intentions, and perceived behavioral control. journal of experimental social psychology, 22, 453-474. allport, g. w. (1935). attitudes. in c. munchinson (ed.), a handbook of social psychology (pp.798-844). worchester, ma: clark university press. anderson, c. j. k., klassen, r. m., & georgiou, g. k. (2007). inclusion in australia: what teachers say they need and what school psychologists can offer. school psychology international, 28, 131-147. armstrong, d. (1998). changing faces, changing places: policy routes to inclusion. in p. clough (ed.), managing inclusive education: from policy to experience (pp. 31-47). london: paul chapman. avramidis, e., & kalyva, e. (2007). the influence of teaching experience and professional development on greek teachers’ attitudes towards inclusion. european journal of special needs education, 22, 367-89. baalpe. (1996). physical education for pupils with special educational needs in mainstream education. west midlands: the british association of advisors and lecturers in physical education. batsiou, s., bebetsos, e., panteli, p., & antoniou, p. (2006). attitudes and intention of greek and cypriot primary education teachers towards teaching pupils with special educational needs in mainstream schools. international journal of inclusive education, 12 (2), 201-219. bebetsos, e., & antoniou, p. (2004). knowledge and information in prediction of intention to play badminton. perceptual and motor skills, 98, 1210-1218. block, e. m. (1995). development and validation of the children’s attitudes toward integrated physical education – revised (caipe-r) inventory. adapted physical activity quarterly, 12, 6077. block, e. m. (2000). a teacher’s guide to including students with disabilities in general physical education (2nd ed.). baltimore: paul h. brookes. block, e. m. (2007). a teacher’s guide to including students with disabilities in general physical education (3rd ed.). baltimore: paul h. brookes. students’ attitudes, intentions and behaviors / bebetsos, derri, zafeiriadis & kyrgirdis 246 block, m. e., & malloy, m. (1998). attitudes of girls towards including a child with severe disabilities in a regular fast-pitch softball league. mental retardation, 36, 137-144. busswell, b., & schaffner, c. b. (1990). families supporting inclusive schooling. in stainback s. & stainback w (eds), support networks for inclusive schooling. baltimore, md: paul brookes. campbell, j., & gilmore, l. (2003). changing student teachers’ attitudes towards disability and inclusion. journal of intellectual & developmental disability, 28 (4), 369-379. chesley, g. m., & calaluce, p. d. (1997). the deception of inclusion. mental retardation, 35, 488-490. conner, m., & norman, p. (1996). predicting health behavior. research and practice with social cognition models (pp. 121-162), buckingham, open university press. davidson, a. r., yantis, s., norwood, m., & montano, d. e. (1985). amount of information about the attitude object and attitude-behavior consistency. journal of personality and social psychology, 49, 1184-1198. derri, v. (2007). physical education in the beginning of the 21st century. standards, goals and objectives in primary education. thessaloniki: christodoulidi publications. de boer, a., sip jan pijl, s. j., & minnaert, a. (2010). regular primary schoolteachers’ attitudes towards inclusive education: a review of the literature. international journal of inclusive education, 1-23, ifirst article. ellery, j. p., & rauschenbach, j. (2000). impact of disability awareness activities on nondisabled student attitudes toward integrated physical education with students who use wheelchairs. research quarterly for exercise and sport, supplement, 71(1), a-106. evaggelinou, c. (2006). creating a school for all in greece: the model of paralympic education. proceedings of the 8th european conference of adapted physical activity. faculty of physical culture, palacky university, olomouc. retrived from http://www.eufapa.upol.cz/www/eucapa2006/full/evag gelinou1. faison-hodge, j., & porretta, d. l. (2004). physical activity levels of students with mental retardation and students without disabilities. adapted physical activity quarterly, 21, 139-152. goodwin, d. l., & watkinson, e. j. (2000). inclusive physical education from the perspective of students with physical disabilities. adapted physical activity quarterly, 17, 144-160. hutzler, y. (2003). attitudes toward the participation of individuals with disabilities in physical activity: a review. quest, 55, 347-373. hutzler, y., & levi, i. (2008). including children with disability in physical education: general and specific attitudes of high-school students. european journal of adapted physical activity, 1(2), 21-30. jesina, o., kudlacek, m., janecka, z., machova, i., & wittmannona, j. (2006). εffect of an intervention program on attitude of elementary school children toward inclusion of children with a disability. in eucapa 2006. proceedings of the 8th european conference of adapted physical activity faculty of physical culture of palacky university (p. 48). olomouc, czech republic. kalyvas, v. a., koutsouki, d., & skordilis, e. k. (2011). attitudes of greek physical education students towards participation in a disability-infusion curriculum. education research journal, 1(2), 2430. krosnick, j. a., boninger, d. s., chuang, y. c., berent, m. k. & carnot, c. g. (1993). attitude strength: one construct or many related constructs? journal of personality and social psychology, 65, 1132-1151. lipsky, d. k., & gartner, a. (1997). inclusion and school reform: transforming america’s classrooms. baltimore, md: paul h. brooks. loovis, e. m., & loovis, c. l. (1997). a disability awareness unit in physical education and attitudes of elementary school students. perceptual and motor skills, 84, 768-770. students’ attitudes, intentions and behaviors / bebetsos, derri, zafeiriadis & kyrgirdis 247 lockhart, r. c., french, r., & gench, b. (1998). influence of empathy training to modify attitudes in physical education of normal children toward peers with physical disabilities. clinical kinesiology, 52, 34-41. magouritsa, g., kokaridas, d., & theodorakis, y. (2005). attitudes of secondary school students toward the inclusion of peers with borderline intelligence prior and after the application of a recreation program. inquiries in sport & physical education, 3(3), 212-224. ministry of education, lifelong learning and religious affairs (2011a). program studies for elementary school physical education. available on line at: http://digitalschool.minedu.gov.gr/info/newps.php ministry of education, lifelong learning and religious affairs (2011b). the elementary physical educator's manual. available on line at: http://digitalschool.minedu.gov.gr/info/newps.php moberg, s. (2003). education for all in the north and the south: teachers’ attitudes towards inclusive education in finland and zambia. education and training in developmental disabilities 38, 417-428. murata, n. m., hodge, s. r., & little, j. r. (2000). students’ attitudes, experiences and perspectives on their peers with disabilities. clinical kinesiology, 54 (3), 59-66. nikolaraizi, m., & reybekiel, n. (2001). a comparative study of children’s attitudes towards deaf children, children in wheelchairs and blind children in greece and in the uk. european journal needs education, 16, 167-182. o’brien, d., kudláček, m., & howe, p. d. (2009). a contemporary review of english language literature on inclusion of students with disabilities in physical education: a european perspective. european journal of adapted physical activity, 2 (1), 46-61. obrusníková, i., válková, h., & block, m. e. (2003). impact of inclusion in general physical education on students without disabilities. adapted physical activity quarterly, 20, 230-245. panagiotou, a. (2006). impact of “paralympics day in schools” program in students’ attitude towards integration of disable children in a physical education class, within a typical school. unpublished master degree dissertation. aristotle university of thessaloniki, greece. panagiotou, a. k., evaggelinou c., doulkeridou, a., mouratidou, k., & koidou, e. (2008). attitudes of 5th and 6th grade greek students toward the inclusion of children with disabilities in physical education classes after a paralympic education program. european journal of adapted physical activity, 1(2), 31-43. panagiotou, a., evaggelinou, c., doulkeridou, a., koidou, e., & mouratidou, k. (2009). evaluation of student’s attitudes from conventional primary schools toward inclusion of children with disabilities in physical education after the implementation of a program. inquiries in sport & physical education, 7(2), 103-113. rose, r. (2001). primary school teacher perceptions of the conditions required to include pupils with special educational needs. educational review 53, 147-156. sable, j. r. (1995). efficacy of physical integration, disability awareness and adventure programming on adolescents’ acceptance of individuals with disabilities. therapeutic recreation journal, 29, 206217. sherrill, c. (1998). adapted physical activity, recreation and sport: cross disciplinary and lifespan (5th ed.). mcgraw-hill, companies, inc. usa. sherrill, c. (2004). adapted physical activity, recreation and sport (6th ed.). boston: mcgraw-hill, companies, inc. usa. slininger, d., sherrill, c., & jankowski, c. m. (2000). children’s attitudes toward peers with severe disabilities: revisiting contact theory. adapted physical activity quarterly, 17, 176-196. smith, m. k., & smith, k. e. (2000). ‘i believe in inclusion, but …’: regular education early childhood teachers’ perceptions of successful inclusion. journal of research in childhood education 14, 161184. students’ attitudes, intentions and behaviors / bebetsos, derri, zafeiriadis & kyrgirdis 248 tesser, a., & shaffer, d. r. (1990). attitudes and attitude change. annual review psychology, 41, 479-523. tripp, a., french, r., & sherrill, c. (1995). contact theory and attitudes of children in physical education programs toward peers with disability. adapted physical activity quarterly, 12, 323-332. slininger, d., sherrill, c., & jankowski, c. (2000). children’s attitudes toward peers with severe disabilities-revisiting contact theory. adapted physical activity quarterly, 17, 176-196. soodak, l. c., podell, d. m., & lehman, l. r. (1998). teacher, student and school attributes as predictors of teachers’ responses to inclusion. journal of special education, 31, 480-497. theodorakis, y. (1994). planned behavior, attitude strength, self-identity, and the prediction of exercise behavior. the sport psychologist, 8, 149-165. theodorakis, y., bagiatis, k., & goudas, m. (1995). attitudes toward teaching individuals with disabilities: application of planned behavior theory. αdapted physical activity quarterly, 12, 151-160. tripp, a., french, a., & sherrill, c. (1995). contact theory and attitudes of children in physical education programs toward peers with disabilities. adapted physical activity quarterly, 12(4), 323-332. van biesen, d., busciglio, a., & vanlandewijck, y. (2006). attitudes towards inclusion of children with disabilities: the effect of the implementation of "a paralympic school day" on flemish elementary children. in eucapa 2006. proceedings of the 8th european conference of adapted physical activity faculty of physical culture of palacky university (p. 66). olomouc, czech republic. wilson, t. d., kraft, d., & dunn, d. s. (1989). the disruptive effects of explaining attitudes: the moderating effect of knowledge about the attitude object. journal of experimental social psychology, 25, 379-400. woodward, r. (1995). the effects of gender and type of disability on the attitudes of children toward their peers with physical disabilities. therapeutic recreation journal, 29, 218-227. microsoft word 4coskun international electronic journal of elementary education vol. 2, issue 3, july, 2010.     evaluation of learning and teaching process in turkish courses eyyup coşkun* mustafa kemal university, turkey murat alkan mustafa kemal university, turkey abstract a radical educational reform occurred in turkey in 2005; and curriculum of primary education courses was renewed. new curriculum was prepared based on constructivist approach. in this scope, curriculum of turkish course was also renewed. this study aims at evaluating applications and opinions of teachers and students about learning and teaching process prescribed in turkish course (1st-5th grades) curriculum. within the scope of the study, semi-structured interview was made with 10 teachers and 12 students. in addition, process teaching a text was evaluated via structured observation method in 5 different classes. according to the results of the study, primary school teachers find some stages in learning – teaching process prescribed in the curriculum unnecessary and therefore do not apply them. teachers mentioned that some texts are above the student level; and they sometimes experience time and material problems. it was seen in the present study that teachers do not have enough information about learning and teaching process in the new curriculum; they do not have high success levels in the applications; and they usually do not apply the forms for evaluating the process in the curriculum. it was found out that, in spite of these problems, courses are student-centred as prescribed in the curriculum; and students have positive opinions about stages of learning and teaching process. keywords: turkish course, curriculum, learning and teaching process introduction a radical educational reform occurred in turkey in 2005; and curriculum of primary education courses was renewed. new curriculum was prepared * correspondence: ecoskun2002@yahoo.com, mustafa kemal university, education faculty, 31000 hatay / turkey, tel.: +90 326 2456000, fax: +90 326 2456005   international electronic journal of elementary education vol.2, issue 3, july, 2010 388   based on constructivist approach. in this scope, curriculum of turkish course was also renewed. learning and teaching process prescribed in the curriculum aims at turning students into active individuals who think, criticize, express themselves and construct the knowledge (meb, 2005). change of turkish course (1st-5th grades) curriculum led to an alteration also in learning and teaching process. learning and teaching process was prepared in a more detailed way in this curriculum when compared to the previous ones. in this curriculum, learning and teaching process was dealt in five stages: “preparation, understanding, and constructing in mind, selfexpression, measurement and evaluation” table 1. learning and teaching process according to turkish (1st-5th grades) curriculum stage sub-stage i. preparation 1. preliminary preparation 2. mental preparation a. activating the foreknowledge b. working with key words c. recognizing and predicting the text d. goal setting e. determining types, methods and techniques ii. understanding 1. visual reading, listening and reading a. visual reading b. listening c. reading d. working with unknown words 2. examining the text 3. developing the vocabulary iii. constructing in mind 1. associating with daily life 2. associating with kemalism, other courses and subdisciplines 3. research iv. self-expression 1. preliminary preparation 2. mental preparation a. activating the foreknowledge – determining topic b. goal setting c. determining methods and techniques ç. determining type and presentation format 3. applying the rules 4. speaking, writing and visual presentation 5. using the vocabulary v. measurement and evaluation stages of learning and teaching process are explained below: i. preparation: main target of preparation stage is to enable students to be prepared for the course physically and mentally. this stage comprises of preliminary preparation and mental preparation sub-stages. preliminary preparation includes the actions such as student and teacher’s preparing equipments, choosing materials, bringing objects and models necessary for the presentation and determining the place to sit (güneş, 2007; meb, 2005). mental preparation stage covers practices such as bringing out the learning and teaching process in turkish courses / coskun & alkan 389   foreknowledge of students about the text to be covered in turkish course and supporting this knowledge with various activities (meb, 2005). according to temur (2007), teacher should allocate enough time to preparation stage and take into consideration environmental conditions and social opportunities present in the preparation stage as well as personal characteristics, knowledge and experiences of students. ii. understanding: understanding refers to the way of thinking on information obtained via listening, reading and visual reading; searching the reasons of it; and making deductions and assessments about this information. understanding stage in the curriculum comprises of activities such as visual reading, listening, and reading, working with unknown words, examining the text and developing the vocabulary (meb, 2005: 163). the students use the ideas, which they form in their minds in mental preparation stage for understanding the text. iii. constructing in mind: this stage was called “learning via text” in original format of the curriculum (2005), but it was named as “constructing in mind” with the amendment in 2009. in the curriculum, it was required “to enable student to associate what is learnt with daily life, other courses and sub disciplines and to investigate a new topic based on the text” (meb, 2005, p. 153) in order for students to construct the acquired knowledge in their minds. practices relating to “thinking, questioning, conceptualizing, making decisions and solving problems” will be made in order to ensure constructing in mind. what is learnt will be associated with daily life, topics of kemalism and other courses in order to ensure transfer and continuity of knowledge. iv. self-expression: most important target of mother tongue education is to develop understanding and explaining skills of the students (yıldız, 2003; güzel, 2010). student’s explanation of information, which he/she learns from the text in learning and teaching process, in different situations in the class environment increases the continuity of what is learnt. stage of selfexpression stage was constituted in turkish course (1st-5th grades) curriculum in order for student to transfer the things, which he/she learns, in written or orally in the class environment. self-expression stage includes explanation-based speaking, writing and visual presentation skills among language skills. v. measurement and evaluation: measurement and evaluation is an inseparable component of learning and teaching process (balcı & tekkaya, 2000; kutlu, 2005). new turkish curriculum introduced important changes also in the topic of measurement and evaluation. measurement and evaluation approach in the curriculum was prepared in order to guide the students and to determine what students know rather than what they do not know, based on directing the process so as to create most appropriate learning-teaching environment instead of giving marks to the students (yangın, 2005; göçer, 2007; birgin & gürbüz, 2009). international electronic journal of elementary education vol.2, issue 3, july, 2010 390   measurement tools such as portfolio, rubric, project assignment, performance assignment, group evaluation, peer evaluation and selfevaluation forms, observation forms, concept map, attitude scale and control list intended for evaluating the process are used in new curriculum in addition to traditional measurement and evaluation methods such as written examination, oral examination, multiple choice test, true-false questions, short answered questions and matching questions. these measurement tools do not intend to evaluate only results, but also learning process as a whole (coşkun, 2005) purpose of the study the purpose of this study is to evaluate learning and teaching process in turkish course (1st-5th grades) curriculum via semi-structured interview and in-class observation conducted with teachers and students. sub problems of the study are as follows: 1. what are the opinions of primary school teachers about learning and teaching process, prescribed in turkish course (1st-5th grades) curriculum? 2. according to structured observation results; to what degree can stages of learning and teaching process, prescribed in turkish course (1st5th grades) curriculum, be applied? 3. what are the opinions of primary school students about learning and teaching process, prescribed in turkish course (1st-5th grades) curriculum? method this research is a descriptive study in survey model implemented by benefiting qualitative data collection techniques. semi-structured interview was conducted with teachers and students in the study. in addition, structured observation was conducted. participants in the present study, semi-structured interview was conducted with 10 teachers (2 teachers from each grade) performing duty in 1st-5th grades in primary education and 12 students (4 students from 4th grade, 2 students from each of other grades). interview is a mutual and interactive communication process conducted for a predetermined and serious purpose based on asking and answering questions (yıldırım & şimşek, 2006). interview provides in-depth knowledge about a particular research topic or question (büyüköztürk, 2008). seven of the teachers participating in the study are female while 3 of them are male. 4 teachers are graduated from institute, 4 teachers are graduated education faculty graduates, and 2 teachers are graduated from other faculties. almost all of the teachers (f=9) have professional experiences of more than 10 years. 8 of the teachers took in-service training about curriculum, but 2 of them did not take such training. structured learning and teaching process in turkish courses / coskun & alkan 391   observation was made in the classes of 5 teachers (1 teacher from each grade level). semi-structured interview was made with 12 students of 5 teachers in whose class observation was made. 7 students are female, and 5 students are male. data collection instruments in the study, “teacher interview form” and “student interview form” were constituted by utilizing the related literature (collins, 2005; coşkun, 2005; bulut, 2006; güven, 2008; karadağ, 2008; taşkaya, muşta, 2008; korkmaz, 2009) in order to determine opinions of teachers and students about learning and teaching process in the curriculum. 5 field experts were asked to evaluate content validity for interview questions based of indicator chart. some amendments were made in measurement tools in accordance with the suggestions made by the experts. final version of teacher interview form comprised of 8 open ended questions. one question was asked for each stage of learning ad teaching process, and 3 questions were asked for evaluating the whole process. in student interview form, one question was asked for each stage, and one question was asked for the whole process. that is, 6 open-ended questions were asked in student interview form. interview forms were applied on 2 teachers and 2 students not included in the sample as a pilot study. it was seen in the pilot study that there was no unclear point in interview questions. another measurement tool used in the study is “observation form relating to turkish course learning and teaching process.” the purpose of this form is to evaluate how learning and teaching process is applied in turkish courses through observation. “positive” and “negative” aspects relating to application of each stage of learning and teaching process were noted by the researchers in the observation form. in addition, it was intended to give a mark to teachers in relation to each stage of learning and teaching process. a rubric relating to observation form was prepared in order to ensure objectivity and reliability of marking in the observation form (see appendix). the rubric was prepared by taking into consideration “relation with the text, application of activities, student participation” aspects of learning and teaching process. according to these aspects, scores between 0 and 5 were given to the teachers. the score 0 was given for the stages never applied in the class, and the score 5 was given for the stages applied best in the class. following the formation of draft of the observation form, expert opinions were taken; and some corrections were made in the measurement tool according to suggestions of the experts. the number of students in the classes where the observation was made is between 35 and 40. in each class, manner of teaching a text in the course book was observed from beginning to end. observations lasted for 4-7 hours for each class. data analysis sound records obtained from teacher and student interviews were decoded. then, these decoded texts were evaluated via content analysis method. international electronic journal of elementary education vol.2, issue 3, july, 2010 392   content analysis refers to gathering together similar data within the frame of particular concepts and themes, and interpreting them by arranging them in an understandable way (yıldırım & şimşek, 2006). “sentence” was taken as evaluation unit in the analysis. opinions of teachers and students were written one under the other in the form of sentences. when sentences with the same meaning are repeated, frequency mark was put in the written sentence. opinions, whose frequencies were determined, were presented via categorical analysis technique. stages of learning and teaching process in the curriculum were taken into consideration in categorization of the opinions. in the observation form, applications of teachers in learning and teaching process and problems emerging during these applications were determined via rubric prepared beforehand and notes taken during the observation. data obtained in this way were classified according to the stages of learning and teaching process; and they were presented with frequency and average values. findings findings obtained in the study via teacher and student interview forms and observation form were classified and presented according to the stages of learning and teaching process. findings relating to the preparation stage for the question “what kind of problems do you have while applying preparation stage?” in the interview form, 4 teachers mentioned that they do not have any problem relating to the preparation stage, but 6 teachers mentioned the following problems: • i have problems with bringing equipments for the activities. (f=4) • text prediction activities may come to be functionless because students read the text beforehand. (f=2) • if the teacher is prepared, he/she experiences no problem. if not, the subject is broached without attracting the attention of the students. (f=1) according to the observation results, teachers do preliminary preparation practices (telling students to open their books, making students ready for listening etc.) which are necessary to be done in the class in “preliminary preparation” which is the first stage of preparation practices. however, it was observed that 2 teachers do not make the activities such as preparing visuals relating to the text and bringing different texts to the class which must be done before the class. accordingly, observation results support the finding that a problem is experienced in bringing materials relating to the text, mentioned also in teacher interviews. according to observation results, the following problems are experiences in the mental preparation stage: • teachers could not use time efficiently in the activities in mental preparation stage. while more than enough time was allocated to some learning and teaching process in turkish courses / coskun & alkan 393   activities, some activities were taken so short that they could not achieve their targets. • no relationship could be established between some activities and the text in this stage. • some activities relating to this stage given in the guidebook were not applied. • some teachers (f=2) wrote the meanings of key words on the blackboard without allowing students to think of and discuss key words. observation scores of teachers (out of 5) pertaining to preparation stage in the evaluation based on rubric are showed in table 2. table 2. observation scores relating to preparation stage no sub-stage n* x 1. preliminary preparation 5 2.80 2. mental preparation 5 2.88 2.1 activating the foreknowledge 5 3.40 2.2 working with key words 5 3.60 2.3 recognizing and predicting the text 1 3.00 2.4 goal setting 3 0.67 2.5 determining type, method and technique 2 0.50 total 5 2.67 * the number of classes in which stage exists in the guide book. table 2 indicates that highest success (3.60) is achieved in the stage of “working with key words”, and lowest success (0.50) is achieved in the stage of “determining type, method and technique” in preparation practices. average success in the preparation stage is 2.67. when students were asked the question “what do you do as preparation for the texts you cover in turkish courses?” 9 students stated that preparation practices are definitely conducted in the class, 1 student mentioned that they are not conducted every time. students mentioned the following activities as preliminary preparation practices: • i take out my pencil and notebook for the purpose of preparation. (f=3) • our teacher checks whether or not we bring course books. ( f=1) • students mentioned that the following activities are made as mental preparation practice: • our teacher asks us questions about the text before starting to read the text. (f=7) • in every text we cover, our teacher tells us something about the text before reading the text. (f=2) • we examine the visuals for the purpose of examination for the text; and we try to find out or predict what is intended to be explained in the text. (f=2) • our teacher reads something from the beginning, something from the middle and something from the end of the text, and makes us predict the content of the text. (f=2) international electronic journal of elementary education vol.2, issue 3, july, 2010 394   findings relating to the understanding stage in the curriculum, understanding stage was phased as visual reading, listening, reading and working with unknown words, examining the text and developing the vocabulary. for the question “what kind of problems do you have in application of the understanding stage?” 3 teachers mentioned that they do not experience any problem. opinions of teachers stating that they have problems in this stage are as follows: • lack of questions to help understand the text in the stage of examining the text negatively impacts understanding. (f=4) • students have difficulty in understanding some texts. (f=2) • students cannot achieve adequate understanding in some texts. (f=2) • i experience problems because students do not have enough reading habits. (f=1) according to the observation results, 2 teachers partly apply the activities mentioned in the guidebook in the stages of visual reading and working with unknown words; 3 teachers partly apply the activities mentioned in the guidebook in the stages of listening and reading and examining the text; and teachers have difficulty in focusing the attention of students on text and ensuring student participation. observation scores of teachers relating to the understanding stage are showed in table 3. table 3. observation scores relating to understanding stage table 3 indicates that highest success (4.4) in understanding practices is achieved in the stage of “working with unknown words” and lowest success (1.6) is achieved in the stage of “visual reading”. average success in preparation stage is 3.27. this indicates that prospective teachers cannot succeed in applying a stage (visual reading) to which they are not accustomed. for the question “can you understand the texts given in the course books sufficiently?” 6 students mentioned that they understand texts sufficiently, but 6 students stated that they have difficulty in understanding some texts. students mentioned the following opinions: no sub-stage n* x 1. visual reading 5 1.60 2. listening 5 3.20 3. reading 5 3.40 4. working with unknown words 5 4.40 5. examining the text 5 3.40 6. developing the vocabulary 5 3.60 total 5 3.27 * the number of classes in which stage exists in the guide book. learning and teaching process in turkish courses / coskun & alkan 395   • i encounter with some unknown words in the texts, but it does not prevent me from understanding the text. (f=3) • i encounter with some unknown words in the texts, so i cannot understand some texts completely. (f=2) • i understand poems more easily. (f=2) • when there are many visuals relating to the text, i understand the text better. (f=1) findings relating to stage of constructing in mind in the curriculum, stage of “constructing in mind” was phased as associating with daily life, associating with kemalism, other courses and subdisciplines, and research. when teachers are asked the question “what kind of problems do you have while applying the stage of constructing in mind?” 3 teachers gave the answer “i experience no problem while applying this stage”. on the other hand, 7 teachers mentioned that they have difficulty in applying the activities given in the guidebook in the stages of “research” and “associating with other courses and sub-disciplines”. according to the teachers, problems experienced in this stage are as follows: • i have problems with associating with other courses and disciplines. (f=3) • research practices are difficult to apply as they are too hard for the students. (f=3) • in the research section, students print papers and bring them. (f=2) • research topics in the guidebooks are not accessible topics. (f=1) according to the observation findings relating to sub stages of stage of constructing in mind, some teachers (1 teacher in the stage of associating with daily life; 3 teachers in the stage of associating with other courses and sub disciplines; 2 teachers in the stage of research) never applied the related stage in the class though it was present in the covered text. it was observed that, in the classes where stage of associating with daily life is applied, students are very eager for participating in the lesson and they want to share their experiences with the class. it was seen that teachers applying the stage of associating with other courses and sub disciplines have difficulty in guiding the students while conducting the activities stated in the guidebook. it was observed that 3 teachers applying the research stage do not establish any relationship between research homework they give and the covered text. table 4. observation scores relating to the stage of constructing in mind no sub-stage n* x 1. associating with daily life 5 3.2 2. associating with kemalism, other courses and sub-disciplines 5 1.2 3. research 5 1.6 total 5 2.0 * the number of classes in which stage exists in the guide book. international electronic journal of elementary education vol.2, issue 3, july, 2010 396   table 4 shows that even though teachers are relatively successful in the stage of associating with daily life, they are not successful in other stages. for the question “can you apply what you learn in turkish courses in your life? can you give example?” 10 students gave the answer “yes, i can use” while 2 students said “i use them from time to time”. it was seen that students can give concrete examples in this topic. for instance, 2 students mentioned that, thanks to a text they covered, they have learnt how to do conscious consumption and necessary points to be paid attention during shopping. accordingly, they pay attention to whether or not the product they buy is closed, and whether or not it is an expired product. another student mentioned that they covered a text relating to traffic lights, so they know what to do when the light is red, yellow or green as explained in the text, and they pay attention to these lights while crossing. findings relating to the self-expression stage for the question “what kind of problems do you experience while applying self-expression stage?”, while 2 teachers mentioned that they do not have any problem in this stage, opinions of teachers stating that they have problems in this topic are as follows: • this stage was unnecessarily divided into sub-stages. we cannot apply all of the stages every time. we apply the stages collectively. (f=3) • since this stage is new for us, we do not know how to apply it. (f=2) • in this stage, students cannot sometimes display the behavior expected from them.(f=2) • we cannot sometimes enable student to speak; we just force him/her to speak. (f=2) • i have so many problems. i think teachers should be trained with regard to this stage. (f=1) • we have sometimes problems with conducting the topics relating to self-expression in the guidebook.(f=1) according to observation results relating to self-expression stage, some teachers did not do the activities in the related stage (1 teacher in the stages of activating the foreknowledge-determining topic and speaking, writing and visual presentation; 2 teachers in the stages of goal setting, determining method and techniques, type and presentation format and applying the rules; 3 teachers in the stage of using the vocabulary). purpose in the activities in this stage is for student to express himself/herself in written, orally or visually. however, it was seen that teachers do not allocate enough time for students to express their ideas. learning and teaching process in turkish courses / coskun & alkan 397   table 5. observation scores relating to self-expression stage no sub-stage n* x 1. mental preparation 5 2.05 1.1 activating the foreknowledge-determining topic 5 2.60 1.2 goal setting 3 0.67 1.3 determining methods and techniques 2 0 1.4 determining type and presentation format 5 1.20 2. applying the rules 5 2.20 3. speaking, writing and visual presentation 5 2.80 4. using the vocabulary 5 1.80 total 5 1.86 * the number of classes in which stage exists in the guide book. according to table 5, application success is low in all the sub-stages of selfexpression practices. it is seen that some sub-stages are never applied or they are applied by very few teachers. for the question “can you express your ideas with writing, words or pictures in turkish courses?” students mentioned the activities they do in speaking, writing and visual presentation practices within learning and teaching process as follows: a) speaking • our teacher makes us speak about the text. (f=10) b) writing • our teacher makes us write about the text. (f=9) • the teacher gives some words, and we write (paragraphs or poems) about these words. (f=2) • our teacher tells us to write something about the text at home. then, he/she makes us tell what we wrote (f=1) • our teacher makes us interpret and write about the visuals. (f=1) c) visual presentation • our teacher makes us draw pictures about the text. (f=7) • our teacher asks us what we understand from the visuals. (f=3) • our teacher tells us to visualize what we understand from the text. (f=1) findings relating to measurement and evaluation stage for the question, “what kind of problems do you have while applying measurement and evaluation stage??” 6 teachers stated that they have time problems while 6 teachers mentioned that abundance of forms leads to paper waste and application difficulty. 3 teachers consider measurement and evaluation as the stage with lowest applicability of turkish course (1st-5th grades) curriculum. teachers mentioned the following opinions about the problems they experience while applying measurement and evaluation stage: • i cannot apply the forms. (f=6) • i want spend time with students rather than spending time with the forms. (f=4) • since we did not receive education about this stage, i have some problems with applying it. (f=2) • i do not approve the application of observation forms as i cannot observe the students properly. (f=2) international electronic journal of elementary education vol.2, issue 3, july, 2010 398   • i apply just post-theme evaluations as i cannot find time. (f=1) according to observation results, 3 teachers conducted the activities mentioned in measurement and evaluation stage in the guidebook, but 2 teachers did not apply any of these practices. it was observed that teachers applying the stage just conduct the activities, but they do not use the forms regarding the evaluation of the process. point averages of teachers in this stage were found 1.2 according to the result of scoring based on rubric. for the question “what does your teacher take into consideration while giving a mark to you in turkish courses? do you think what your teacher does is true?” students gave the following answers: • our teacher makes us fill up self-evaluation and peer evaluation forms in the book. (f=7) • he/she gives marks according to the beauty of italic handwritings. (f=5) • our teacher sometimes distributes forms to us and asks us to evaluate our friends or ourselves. (f=4) • my teacher takes notes while we are speaking. (f=3) • our teacher gives us mark according to whether or not we do our homework. (f=2) • our teacher gives us mark according to participation in class activities. (f=1) • our teacher makes us fill up a form at the beginning of each theme. (f=1) almost of all of the students (f=9) mentioned that they approve teachers’ way of giving marks. analyzing the findings relating to measurement and evaluation all together, it is seen that measurement and evaluation is the stage in which teachers have most difficulty in learning and teaching process. difficulties in applying the measurement tools in crowded classes top the list of problems in this topic. in addition, it is seen that students are not knowledgeable enough in the topic of techniques for evaluating the process in new curriculum. according to the interviews conducted with the students, awareness was created among students in the topic of new measurement approaches. findings relating to the whole learning and teaching process for the question “are students active in turkish courses? can you explain it?”, while 7 teachers mentioned that students come to be active in this learning and teaching process, 3 teachers stated that students sometimes come to be active, but they are not sometimes active in this learning and teaching process. teachers mentioning that students come to be active stated that students, never wanting to talk in the class in the past, start to participate in the class in new learning and teaching process and start to make comments on the topics. for the question “do you think distribution of activities is balanced according to learning areas (reading, speaking, writing, listening, visual reading, and visual presentation)? if not, which ones are more intense? which ones should be allocated more space? can you explain it?” 5 teachers mentioned that there is a balanced distribution of activities according to learning areas. other teachers mentioned the following opinions with regard to distribution of activities according to learning areas. learning and teaching process in turkish courses / coskun & alkan 399   • i think the activities given in the topic of grammar in learning and teaching process are inadequate. (f=4) • activities are unnecessarily intense in visual reading and presentation. (f=3) • i think there are very few activities relating to writing skill. (f=1) for the question “do you encounter with physical insufficiencies in the school and class while applying learning and teaching process prescribed in the curriculum and course book? can you explain it?, 6 teachers mentioned that they encounter with physical insufficiencies while 4 teachers stated that they do not encounter with this kind of insufficiencies. opinions of teachers about this topic are as follows: • we do not have projection or computer. (f=4) • classes are too crowded; it is difficult to walk even between the desks. (f=2) • there is no empty corner in the class in order to apply a drama practice (visual presentation, speaking) relating to the text. (f=1) • we have a television, but it is functionless. (f=1) for the question “do you think turkish courses are enjoyable or boring? why?” almost all of the students (f=11) mentioned that turkish courses are enjoyable and they like expressing themselves by writing or speaking. however, 1 student thinks that turkish courses are not enjoyable because he/she cannot sometimes give answers to the questions asked by the teacher in the class. table 6. scores relating to the stages of learning and teaching process in the classes where the observation was made no learning and teaching process stages n* x 1. preparation 5 2.67 2. understanding 5 3.27 3. constructing in mind 5 2.00 4. self-expression 5 1.86 5. measurement and evaluation 5 1.20 total 5 2.20 *.the number of classes in which stage exists in the guide book. according to table 6, highest success (3.27) relating to the application of stages of learning and teaching process in the class was achieved in “understanding”, but lowest success (1.20) was achieved in “measurement and evaluation”. observation score average is 2.20 for all of the stages in learning and teaching process. discussion and conclusion according to the results of this study, teachers have difficulty particularly in providing the material to be brought to the class for the purpose of preparation for the course. in also the study conducted by anılan et al. (2008), teachers mentioned that they have problems like incapability of accessing the materials relating to the preparation and spending much time while preparing materials. teachers stated that prediction activities come to be functionless as texts are read by students beforehand in the preparation stage. observation results indicate that many of the teachers do not apply the stages of “goal setting” and “determining type, method and technique” in preparation international electronic journal of elementary education vol.2, issue 3, july, 2010 400   practices. the study conducted by akyol et al. (2008) reported that teachers do not adequately know and apply the practices relating to determining type, method and technique before the reading. on the other hand, it was determined that students are very conscious in the topic of application of preparation practices. evaluating findings relating to understanding stage all together, it is seen that teachers and students find some texts in the book above the student level; stage of examining the text should be supported with questions; teachers cannot apply the activities in understanding stage completely; and they are unsuccessful particularly in visual reading practices. it is reported in many studies (coşkun, 2005; kuru, 2008; balun, 2008; odabaşı, 2007) that students are active in visual reading practices, and they like these activities. according to the observation results in the present study, teachers displayed the lowest success in understanding stage in the sub-stage of “visual reading”. even though this is a stage conducted tastefully by students, it is seen that teachers cannot sufficiently recognize this stage included in the curriculum for the first time. evaluating the findings relating to the stage of constructing in mind all together, it is seen that there is no important problem in application of the stage of associating with daily life and this stage is considered very interesting and beneficial for the students. on the other hand, it is seen that teachers are not successful enough in the stage of associating with other courses and subdisciplines, and research. in the present study, it is seen that some of the activities in self-expression stage are not applied by the teachers; and some problems are experienced in the application, time problem being in the first place. in addition, some teachers stated that they do not have enough information about this stage. on the other hand, based on student opinions, it is understood that, in spite of the problems in application, this stage makes contribution to students’ expressing themselves orally, in written and visually. evaluating findings relating to measurement and evaluation in the study all together, it is seen that the stage in which teachers have most difficulty in learning and teaching process is measurement and evaluation. difficulties of applying measurement tools in crowded classes are mentioned to top the list of difficulties in this topic. findings of the studies conducted by elvan (2007), karadağ (2008) and anılan et al. (2008) correspond to these results. in addition, it is seen that teachers are not knowledgeable enough in the topic of techniques for evaluating the process in new curriculum. many previous studies (özpolat et al, 2007; yapıcı, 2007; yiğitoğlu, 2007; gözütok, et al., 2005; yaşar et al., 2005; collins, 2005; bulut, 2006; damlapınar, 2008; rençber, 2008; tüfekçioğlu & turgut, 2008; korkmaz, 2009) report that teachers do not adequately know and use process-based evaluation methods. according to the interviews conducted with the students, awareness was created among students in the topic of new measurement approaches. one of the basic principles of constructivist approach making up the core of new curriculum is student-centred education (özer, 2007). in many studies aiming at evaluation of new curriculum (collins, 2005; coşkun, 2005; bulut, 2006; güven, 2008; karadağ, 2008; taşkaya, muşta, 2008; korkmaz, 2009) it was concluded that learning and teaching process in turkish courses / coskun & alkan 401   students are more active compared to the previous system; and there is a more student-centred approach in teaching of new courses. teachers in this study also mentioned that learning and teaching process in the curriculum makes students more active when compared to the past and this contributes to their developments. in their evaluations relating to the turkish courses, students frequently used the sentences expressing that they are active in the classes. this indicates that, in spite of many problems in the application, new curriculum constitutes a studentcentred education environment. the most important problem revealed in this study is that teachers do not perceive stages of learning and teaching process as parts of a whole, and they apply them as independent activities. according to the instructions given in guidebooks, teachers do not adequately know or think of what do to and how and why they will do them in the classes. observation results in this study indicate that teachers never do or superficially do many of the activities in the guidebooks in stages of learning and teaching process. in this case, application success of stages comes to be very low. in addition, since the relationship of applied stages with text cannot be established adequately, turkish course which is supposed to be a text-centred course (coşkun, 2005) may come to be a heap of independent activities. in the present study, teachers mentioned that some texts are above the level of student level, time problem is experienced in some texts, and material problem is experienced in application of some activities. these results support the results of some previously conducted studies. it was concluded in the study conducted by özoğul (2007) that teachers think that many texts are not suitable for the student level; and it was found out in the study conducted by coşkun (2005: 421-476) that material problem is experienced from time to time. one of the important amendments made in turkish course (1st-5th grades) curriculum is that grammar education is not handled as a separate learning area, but it is considered enough to teach it in other learning areas just via adumbration. in the present study, some teachers stated that non-existence of grammar education in learning and teaching process is a deficiency. in other studies (coşkun, 2005; bulut, 2006; elvan, 2007; yiğitoğlu, 2007) it was reported that teachers think grammar should be allocated space in the curriculum. accordingly, it can be said that many of the teachers do not adopt the approach of curriculum in the topic of grammar education. this difference in approaches can be explained as some teachers have difficulty in accepting the situations which are “contrary” to the system they are “accustomed” to. in addition, it can be said that students’ dealing with test questions in central examinations as from early ages in the current education system leads to pressure on teachers in the topic of grammar education. in many studies (yaman, 2009; uşun, 2008; kumral et al., 2008) it is revealed that physical insufficiencies decrease student success and efficiency of the teachers. in the present study, teachers mentioned that class environment is insufficient for some activities in learning and teaching process, and there is a lack of technological instruments. in the study conducted by güven (2008), primary school teachers stated that insufficiency of physical facilities in the school environment negatively impacts education. in the present study, students mentioned that turkish courses are enjoyable, and they can apply what they learn in their daily lives. studies conducted by international electronic journal of elementary education vol.2, issue 3, july, 2010 402   coşkun (2005), bulut (2006), elvan (2007) and güven (2008) report that education given in turkish courses via new curriculum has positive impacts on students, and students like turkish courses. in the present study, some problems emerging in turkish course learning and teaching process were determined as a result of teacher and student interviews and observations. none of these problems is unimportant. on the other hand, it should be admitted that a program cannot be perfect by itself. each curriculum can be applied efficiently only if stakeholders such as teacher, student, parents, school administration, ministry are in harmony and support one another. accordingly, problems revealed in the present study cannot be considered as problems just stemming from the curriculum. necessary updates should be made in the curriculum according to the problems emerging in the application considering the fact that curriculum development is a dynamical and continuous process. suggestions 1. stages of learning and teaching process in the curriculum should be decreased. it will be proper to give education relating to the stages of “goal setting” and “determining type, method and technique”, “associating with other courses and sub-disciplines” mentioned as functionless by teachers and relating to the stages of “goal setting”, “determining methods and techniques”, “determining type and presentation format” in self-expression practices just once in each theme. 2. texts in the course books should be reviewed; long texts should be shortened; incomprehensible texts should be simplified. 3. the fact that some activities particularly in the preparation stage in learning and teaching process are based on materials difficult to be provided by teachers and students makes these activities inapplicable. this kind of activities should be rearranged by taking into consideration also the crowded classes. 4. one of the weakest stages of learning and teaching process in the curriculum and teacher guidebooks is the stage of “examining the text”. in this stage, questions which will enable text to be understood and interpreted better should be allocated space. 5. text prediction practices should be limited to listening practices not present in student course books. 6. purpose and usage of measurement and evaluation methods in learning and teaching process in the curriculum and course books should be explained better. in addition, it will be good to decrease the number of these tools considering the problems to emerge in application of these tools in crowded classes. 7. teachers should learn new teaching approaches and course books introduced by the curriculum with all the details, increase their knowledge levels relating to learning and teaching process prescribed by the curriculum, and develop themselves in the topic of how learning and teaching process stages should be applied in the class environment. 8. it is not a right behaviour for teachers to skip the stage in which they have difficulty rather than solving the problems emerging in application of learning and teaching process and producing alternatives in this topic. teachers should make use of group studies as an opportunity to discuss these problems and produce solutions to the emerging problems. 9. teachers should read the text to be covered in the class before coming to the class together with the instructions in the guidebook, and make preliminary preparations rather than reading the text together with the class for the first time. learning and teaching process in turkish courses / coskun & alkan 403   10. teachers should adopt new approach brought in by curriculum in the topic of grammar education. • • • acknowledgement this study is supported by mustafa kemal university unit of scientific research projects dr. eyyup coşkun, is an assistant professor faculty of education of mustafa kemal university, turkey. he specializes in reading and writing education, text linguistics in language education. murat alkan, is a turkish language teacher. he has master degree in program of turkish language education. international electronic journal of elementary education vol.2, issue 3, july, 2010 404   references akyol, h., ateş, s. & yıldırım, k. (2008). sınıf öğretmenlerinin kullandıkları okuma stratejileri ve bu stratejileri tercih nedenleri. vii. ulusal sınıf öğretmenliği eğitimi sempozyumu (2-4 mayıs 2008), ankara: nobel yayınları. 723-728. anılan, h., çalışır, f. , genç, b. & okkirman, u. (2008). sınıf öğretmenlerinin türkçe dersinin uygulama aşamasında karşılaştıkları sorunlar ve çözüm önerileri. vii. ulusal sınıf öğretmenliği eğitimi sempozyumu (2-4 mayıs 2008). ankara: nobel yayın dağıtım, 783-788. balcı, e. & tekkaya, c. (2000). ölçme ve değerlendirme tekniklerine yönelik bir ölçeğin geliştirilmesi. hacettepe üniversitesi eğitim fakültesi dergisi, 18, 42-50. balun, h. (2008). i̇lköğretim i. kademede uygulanan görsel okuma ve görsel sunu öğrenme alanının türkçe öğretiminde kazanımlara ulaşmadaki etkililiği (bingölelazığ-diyarbakır örneği). yayımlanmamış yüksek lisans tezi. elazığ: fırat üniversitesi. birgin, o. & gürbüz, r. (2008). sınıf öğretmeni adaylarının ölçme ve değerlendirme konusundaki bilgi düzeylerinin i̇ncelenmesi. selçuk üniversitesi sosyal bilimler enstitüsü dergisi, 20, 163-179. bulut, i̇. (2006). yeni i̇lköğretim birinci kademe programlarının uygulamadaki etkililiğinin değerlendirilmesi. yayımlanmamış doktora tezi. elazığ: fırat üniversitesi. büyüköztürk, ş., çakmak, e. k., akgün, ö. e., karadeniz, ş. & demirel, f. (2008). bilimsel araştırma yöntemleri, ankara: pegem akademi yayınları. collins, a. (2005). i̇lköğretim türkçe programları pilot uygulama değerlendirmesi. eğitimde yansımalar viii: yeni i̇lköğretim programlarını değerlendirme sempozyumu. ankara: tekışık eğitim araştırma geliştirme vakfı yayınları, 220-229. coşkun, e. (2005). i̇lköğretim 4. ve 5. sınıf öğretmen ve öğrencilerinin yeni türkçe dersi öğretim programı’yla i̇lgili görüşleri üzerine nitel bir araştırma. kuram ve uygulamada eğitim bilimleri, v (2), 421-476. damlapınar, g. (2008). i̇lköğretim 1. kademe öğretmenlerinin yapılandırmacı öğrenme yaklaşımına i̇lişkin görüşlerinin i̇ncelenmesi. yayımlanmamış yüksek lisans tezi, konya: selçuk üniversitesi, sosyal bilimler enstitüsü. elvan, z. (2007). türkçe (1-5. sınıflar) ders programının öğretmen görüşlerine göre değerlendirilmesi. yayımlanmamış yüksek lisans tezi. ankara: gazi üniversitesi, eğitim bilimleri enstitüsü. göçer, a. (2007). türkçe öğretiminde ölçme ve değerlendirme. (ed.: kırkkılıç, a. & akyol, h.) i̇lköğretimde türkçe öğretim yöntemleri. ankara: pegem akademi yayıncılık, 431-467. gözütok, f. d., özcan e. a. & karacaoğlu, ö. c. (2005). i̇lköğretim programlarının öğretmen yeterlikleri açısından değerlendirilmesi. eğitimde yansımalar viii: yeni i̇lköğretim programlarını değerlendirme sempozyumu. ankara: tekışık eğitim araştırma geliştirme vakfı yayınları, 17-40. güneş, f. (2007). türkçe öğretimi ve zihinsel yapılandırma. ankara: nobel yayınları. güven, a. z. (2008). i̇lköğretim 4. ve 5. sınıf türkçe derslerinde teknolojik araç-gereçlerin kullanımı üzerine nitel bir araştırma. vii ulusal sınıf öğretmenliği eğitimi sempozyumu (2-4 mayıs 2008), ankara: nobel yayınları, 658-661. learning and teaching process in turkish courses / coskun & alkan 405   karadağ, r. (2008). türkçe öğretim programının uygulanmasında yaşanan sorunlar ve çözüm önerileri, vii. ulusal sınıf öğretmenliği eğitimi sempozyumu (2-4 mayıs 2008). ankara: nobel yayın dağıtım, 718-722. korkmaz, i̇. (2006). yeni i̇lköğretim birinci sınıf programının öğretmenler tarafından değerlendirilmesi, selçuk üniversitesi sosyal bilimler enstitüsü dergisi, 16, 393-402. kumral, o., kanmaz, a. & kıran, h. (2008). yeni i̇lköğretim programının uygulanmasında i̇lköğretim okullarındaki teknik donanım ve fiziki ortamın yeterliği. vii. ulusal sınıf öğretmenliği eğitimi sempozyumu (2-4 mayıs 2008). ankara: nobel yayınları, 315-318. kuru, a. (2008). i̇lköğretim beşinci sınıf türkçe dersi öğretim programında yer alan görsel okuma ve görsel sunu becerilerinin öğretmen görüşleri doğrultusunda i̇ncelenmesi. yayımlanmamış yüksek lisans tezi. adana: çukurova üniversitesi. kutlu, ö. (2005). yeni i̇lköğretim programlarının öğrenci başarısındaki gelişimi değerlendirme boyutu açısından i̇ncelenmesi. eğitimde yansımalar viii: yeni i̇lköğretim programlarını değerlendirme sempozyumu. tekışık eğitim araştırma geliştirme vakfı yayınları, 64-76. meb. (2005). i̇lköğretim türkçe dersi (1-5. sınıflar) öğretim programı ve kılavuzu. ankara: devlet kitapları müdürlüğü basımevi. odabaşı, n. (2007). öğretmenlerin görüşlerine göre 4. sınıf türkçe öğrenci çalışma kitabı etkinliklerinin kazanımları gerçekleştirme düzeyi (nevşehir i̇li örneği). yayımlanmamış yüksek lisans tezi, niğde: niğde üniversitesi. özer, b. (2007). öğretim yönteminin öğretimde planlama değerlendirme dersinde akademik başarı ve eleştirel düşünmeye etkisi. yayımlanmamış doktora tezi, elazığ: fırat üniversitesi, sosyal bilimler enstitüsü. özoğul, ç. (2007). i̇lköğretim 4. sınıf türkçe programındaki değişiklikler üzerine bir çalışma. yayımlanmamış yüksek lisans tezi. konya: selçuk üniversitesi. özpolat, a., ragıp, f. s., i̇şgör, i̇. y. & sezer, m. (2007). sınıf öğretmenlerinin yeni i̇lköğretim programına i̇lişkin görüşlerinin i̇ncelenmesi. milli eğitim dergisi, 174, 206-213. rençber, i̇. (2008). yeni i̇lköğretim programının uygulanmasında karşılaşılan sorunlara i̇lişkin müfettiş, öğretmen ve yönetici görüşleri (konya i̇li örneği). yayımlanmamış yüksek lisans tezi. konya: selçuk üniversitesi. taşkaya, s. m. & muşta, m. c. (2008). sınıf öğretmenlerinin türkçe öğretim yöntemlerine i̇lişkin görüşleri. elektronik sosyal bilimler dergisi, 7 (25), 240-251. temur, t. (2007). yeni türkçe öğretim programları. (ed. kırkkılıç, a. & akyol, h.) i̇lköğretimde türkçe öğretim yöntemleri. ankara: pegem akademi yayıncılık, 351394. tüfekçioğlu, n. & turgut, z. (2008). yenilenen i̇lköğretim programı çerçevesinde değişen ölçme değerlendirme uygulamalarına sınıf öğretmenlerinin bakış açıları ve karşılaştıkları zorluklar. vii. ulusal sınıf öğretmenliği eğitimi sempozyumu (2-4 mayıs 2008). ankara: nobel yayınları, 131-133. uşun, s. (2008). türkiye’de yeni uygulamaya koyulan (1-5. sınıflar) i̇lköğretim programlarının program geliştirme süreçleri açısından değerlendirilmesi. vii. ulusal sınıf öğretmenliği eğitimi sempozyumu (2-4 mayıs 2008). ankara: nobel yayınları, 191-197. yaman, h. (2009). i̇lköğretim türkçe dersi programı’nın kalabalık sınıflarda uygulanabilirliğine i̇lişkin öğretmen görüşleri. kuram ve uygulamada eğitim bilimleri, 9 (1), 329-359. international electronic journal of elementary education vol.2, issue 3, july, 2010 406   yangın, b. (2005). i̇lköğretim türkçe dersi öğretim programı ve kılavuzunun değerlendirilmesi. kuram ve uygulamada eğitim bilimleri. 5 (2), 477-516. yapıcı, m. & leblebicier, n. h. (2007). öğretmenlerin yeni i̇lköğretim programına i̇lişkin görüşleri. i̇lköğretim online, 6 (3), 480-490. yaşar, ş., gültekin, b. t., yıldız, n. & girmen, p. (2005). yeni i̇lköğretim programlarının uygulanmasına i̇lişkin sınıf öğretmenlerinin hazırbulunuşluluk düzeylerinin ve eğitim gereksinimlerinin belirlenmesi (eskişehir i̇li örneği). eğitimde yansımalar viii: yeni i̇lköğretim programlarını değerlendirme sempozyumu. tekışık eğitim araştırma geliştirme vakfı yayınları, 51-63. yıldırım, a. & şimşek, h. (2006). sosyal bilimlerde nitel araştırma yöntemleri. ankara: seçkin yayıncılık yıldız, c. (2003). ana dili öğretiminde çağdaş yaklaşımlar ve türkçe öğretimi. ankara: pegem akademi yayıncılık. yiğitoğlu, r. (2007). i̇lköğretim dördüncü ve beşinci sınıf türkçe programı hakkında öğretmen görüşleri. yayımlanmamış yüksek lisans tezi. konya: selçuk üniversitesi. learning and teaching process in turkish courses / coskun & alkan 407   appendix rubric relating to turkish course learning and teaching process score qualities 0 (not done) nothing relating to the stage was done in the process. 1 (very bad) no relationship was established with the covered text in operation of the stage. activities relating to this stage given in the course book were applied very unsuccessfully. in this stage, participation of students in the applications could not be ensured. 2 (bad) relationship with the covered text is weak in operation of the stage. activities relating to this stage given in the course book could not be applied successfully. in this stage, participation of students in the applications was ensured just in a very limited level. 3 (medium) the relationship with the covered text could not be established adequately in operation of the stage. activities relating to this stage given in the course book were applied, but it cannot be said that these activities are adequately successful. in this stage, participation of students in the applications could not be ensured adequately. 4 (good) the relationship with the covered text was established in operation of the stage. activities relating to this stage given in the course book were applied successfully. in this stage, participation of students in the applications was ensured. 5 (very good) the relationship with the covered text was established very well in operation of the stage. activities relating to this stage given in the course book were enriched and applied successfully. in this stage, participation of students in the applications was ensured very efficiently. microsoft word 11iejee_6_2_uygun_kunt international electronic journal of elementary education, 2014, 6(2), 357-370. an analysis of the relationship between prospective teachers’ thinking styles and their attitudes to teaching profession according to various variables mehmet uygun ∗∗∗∗ dumlupınar university, turkey halil kunt dumlupınar university, turkey received: 3 october 2013 / revised: 14 december 2013 / accepted: 27 march 2014 abstract this study aimed to analyze the relationship between primary school prospective teachers’ thinking styles and their attitudes to teaching profession. the study group for the research consisted of fourth-grade primary school of elementary education, social sciences education and science education students studying at dumlupınar university education faculty in the fall term of 2013-2014 academic year. the scale was applied to 222 students. survey model was used. in order to define the prospective teachers thinking styles, “thinking styles inventory”’ that was adapted into turkish by fer (2005) and “teaching profession attitude scale” that was developed by çetin (2006) were used in this research. according to the results of this study, prospective teachers’ attitude to teaching profession has been found to be positive and women’s attitude to teaching profession is higher than men in all the dimensions. while the most preferred thinking styles are legislative, hierarchic and executive thinking styles, the least preferred thinking styles are oligarchic and conservative thinking styles. in addition, there is a significant relationship between prospective teachers and total attitude point of prospective teachers to teaching profession. keywords: thinking styles, attitude to teaching profession, prospective teacher. introduction thinking is the disciplined process of the conceptualizing, applying, analyzing and evaluating knowledge gathered through observation, experience, intuition, reasoning and other ways (özden, 2003). thinking is a very complex and abstract skill. according to saban (2005), it would be defined as “going beyond the available knowledge’’ and ∗ mehmet uygun, department of elementary education, faculty of education, dumlupınar university, kutahya, turkey, phone:+90 274 265 20 31 e-mail: muyguntr@gmail.com international electronic journal of elementary education vol.6, issue 2, 357-370,2014 358 “reaching new information by using available knowledge”. style is the way that individuals prefer while using their skills, doing something or thinking over something (sternberg, 1997; zhang and sternberg, 2005). thinking processes and preferences would vary according to individuals. sternberg (1997) advises a thinking style, based on mental self-government, which is related to how an individual prefers to think about a subject during the learning and after the learning. thinking styles are approaches and leanings that are the results of mental processes related to various problems, events, phenomena and variables (sünbül, 2004).thinking styles are related to social environment and they change according to time and culture (zabukovec and kobal-grum, 2004). this theory claims that thinking styles direct people’s daily activities like governing a society. according to this, thinking cannot be defined as skill or intelligence but as the way of using skill or intelligence (fer, 2005; emir, 2013). mental self-government speaks of 13 thinking styles under five categories. these categories are: functions, forms, levels, scopes, and leanings. these categories and general features of thinking styles related to these categories can be defined as (fer, 2005; sternberg, 1997; zhang and sternberg, 2005; sternberg, grigorenko and zhang, 2008): i. functions: 1. legislative: innovative, ideogenetic. 2. executive: coherent, regular, following the commands. 3. judicial: judging, evaluating, idea expressing. ii. forms: 4. monarchic: deals best with one goal and they focus on it. 5. hierarchic: focuses on multiple goals at once and recognizes that all goals cannot be fulfilled equally. s/he can prioritize goals easily. 6. oligarchic: deals well with goals that are of equal weight, but s/he has difficulty in prioritizing goals of different weights. 7. anarchic: chooses randomly and abstains from the existing mindsets. iii. levels: 8. global: interested in abstract ideas and general portrait. 9. local: interested in concrete ideas and details. iv. scopes: 10. internal: independent, self-sufficient, abstains from communication; likes to be by himself. 11. external: likes to collaborate and work in groups; s/he is addicted. v. leanings: 12. liberal: innovator, anti-traditionalist, dreamy. 13. conservative: traditionalist, prefers the one that has been experienced, realistic. according to mental self-government, individuals have many thinking styles rather than a unique thinking style (sternberg, 1997). styles may change in order to conform with different duties and events (sümbül, 2004; buluş, 2005). because of the environmental factors, individuals’ dominant thinking style may change and differentiate an analysis of the relationship between prospective teachers’ / uygun & kunt 359 in time. thus, various styles may be classified as different; may not be classified as good or bad (duru, 2004). in thinking styles, one can show different attitudes towards different events and stimuli. people can learn many attitudes towards the things that can be attitude objects (near-distant) through their experiences, parents, friend environment, mass media, effect of other individuals and conditioning (üstüner, 2006). attitude is “a learned leaning of an individual towards apparent people, objects and situations that direct him to defined behaviors” (demirel, 2005). according to i̇nceoğlu (2004), attitude is a mental, sensational and behavioral reaction inclination of an individual that he organizes by his knowledge, sense, experience and motive towards himself, an event, an object or a subject. knowing the ideas of prospective teachers about teaching is important for the organization of teaching activities. prospective teachers’ gaining attitude to teaching and their evaluating it as valuable are as much important as knowledge (çetin, 2006). knowledge, feeling and skills that can be gained in teacher education program is to make their behaviors more influential. determining the attitudes of students that are educated in teacher education program will also give information about the necessary attitudes that these students should gain during education (üstüner, 2006). this study aims to show thinking styles of prospective teachers educated in dumlupınar university, primary school education program (elementary education, social sciences education and science education) and the relationship between these thinking styles and prospective teachers’ attitudes to teaching profession. method model of the research because an existing situation is described, survey model (of quantitative research methods) has been used in this research (gay, mills and airasian, 2006; karasar, 2011). the survey model is a research approach that aims to define an existing or a past situation as itself (kaptan, 1998).general survey models try to define an individual or an object in a sample or in the whole population within its own conditions in order to come up with a general judgment on the population (karasar, 2011). the study group the study group for the research consisted of fourthgrade primary school of elementary education, social sciences education and science education students studying at dumlupınar university education faculty in the fall term of 2013-2014 academic year. the scale was applied to 222 students. all the students answered the whole scale sincerely. table 1 includes their demographical information. table 1. demographical information of participants variables demographical information n (%) gender female male total 153 69 222 68.9 31.1 100 department science education elementary education social sciences education total 74 78 70 222 33.3 35.1 31.5 100 according to descriptive statistics, 68.9% of 222 participants are women and 31.1% are men 33% of the participants are the students of science education department, international electronic journal of elementary education vol.6, issue 2, 357-370,2014 360 35% are students of elementary education department and 31% of them are from students of social sciences education department. data gathering tool in order to understand the thinking styles of prospective teachers, “thinking styles inventory” adapted into turkish by fer (2005) and “teaching profession attitude scale” developed by çetin (2006) were used in this research. thinking styles inventory (tsi). thinking styles inventory, developed by sternberg and wagner (1992) and adapted into turkish by fer (2005), was used as the data gathering tool. turkish form that was adapted by fer (2005) was prepared in order to test 13 thinking styles in 5 basic categories. it tests each of them by 8 articles. seven-point likert scale item was used: it is totally unsuitable for me (1), it is not very suitable for me (2), it is a little suitable for me (3), it is almost suitable for me (4), it is suitable for me (5), it is mostly suitable for me (6), it is totally suitable for me (7). there is not a total point in the item because a thinking style that is dominant in an individual, available in a sub-scale under a basic aspect is measured independent from other aspects. possible point of a sub-scale is between 8 and 56. constant points is used to group individuals according to thinking style under basic categories. each individual is appointed to the thinking style that is a part of the sub-scale which is under basic thinking category that s/he got the highest point in it (fer, 2005). if the point increases, (shown) thinking style under the sub-scale is accepted as high. according to pearson product-moment correlation coefficient that was carried out check the consistency between tsi’s turkish and english form, in all the articles (except 4. and 73) the significance level was found to be between .40 and .99 (fer, 2005). according to pearson product-moment correlation coefficient, the relationship between english and turkish application of the scale to the same individual was: legislative .78, executive .95, judicial .83, monarchic .83, hierarchic .94, oligarchic .93, anarchic .93, global .95, local .88, internal.88, external .80, liberal .92 and conservative .54. there are significant and positive (p < .01) values in all sub-scales. average correlation coefficient of sub-scales is .79. according to factor analysis questioning (structural) validity of turkish scale, there is a scale of 70 articles including %45 variance. while internal reliability alpha of (the whole) turkish form of tsi that contains 104 articles is .90, alpha coefficient of the form of 70 articles is .89. findings showed that the13 sub-scales had internal consistency reliabilities ranging from .37 to .88 (fer, 2005). while reliability co-efficient of the scale is calculated as .94 for n: 222 according to total point in this study, they are calculated according to sub-scales as: legislative .89, executive .88, judicial .89, monarchic .70, hierarchic .91, oligarchic .76, anarchical .75, global .81, local .82, internal .87, external .91, liberal .91 and conservative .90. according to arithmetic mean; the point is: 1. totally unsuitable (11,85), 2. not very suitable (1,86-2,71), 3. a little suitable (2,72-3,57), 4. almost suitable (3,58-4,43), 5. suitable (4,43-5,28), 6. mostly suitable (5,29-6,14), 7. totally suitable (6,15-7,00). teaching profession attitude scale (tpas). teaching profession attitude scale developed by çetin (2006) was used in this study. in the five-point likert scale, there are 35 articles; 15 of them are positive and 20 of them are negative. the scale that consists of three aspects (love, value and coherence) gives different points for the total point and 3 sub-scales. variance for each factor of the scale is for love (29.6%), for an analysis of the relationship between prospective teachers’ / uygun & kunt 361 value (12.2%) and coherence (9.4%). total variance of the scale is (51.2%). alpha reliability coefficient of the scale in total is .95. it is .95 for love, .81 for value and .76 for the coherence (çetin, 2006). in this research, reliability coefficient of the scale in total is .94; for love .92, for value .88 and for the coherence .74. while positive sentences of the scale change from “i strongly agree” to “i strongly disagree” and from 5 to 1, there is a change from “i strongly agree” to “i strongly disagree” and from 1 to 5.if the point is close to 5, it shows that students agree premise and if it is close to 1,00 it shows their disagreement. if the point is 2,59 or lower, it shows negative attitude and if it is over than that, it shows positive attitude. points for each options are so: i strongly agree (4,20-5.00), i agree (3.40-4.19), i agree moderately (2.60-3.39), i don’t agree (1.802.59) and i strongly disagree (1.00-1.79). analysis of the data while evaluating the thinking styles of the prospective teachers and the result of the relationship between these styles and their attitude to teaching profession, t-test; anova and lsd test (lsd test has been carried out in order to find the reason of statistically significant difference after anova) were used in addition to descriptive statistical methods (mean, standard deviation, frequency, percentage). furthermore, the relationship between thinking styles of prospective teachers and their attitude to teaching profession were analyzed after the calculation of pearson correlation coefficient. p < 0.05 has been accepted as significant difference. results this part of the study firstly defined the prospective teachers’ thinking styles and their attitude to teaching profession. secondly, prospective teachers’ thinking styles and their attitude to teaching profession were compared according to variables such as gender and the program they study. table 2. includes data of prospective teachers’ attitude to teaching profession. table 2. average of prospective teachers’ attitude to teaching profession points and standard deviation value factors n ( x ) sd love 222 4.00 0.56 value 222 4.33 0.52 coherence 222 4.24 0.62 teaching profession attitude (total) 222 4.11 0.52 prospective teachers’ total attitude to teaching profession degree is ( x = 4.11) is positive, their total attitude to teaching profession is the highest in the sub-dimension “value” ( x = 4.33) and the lowest in the sub-dimension “love” ( x = 4.00).table 3 includes the data of the comparison of prospective teachers’ attitude to teaching profession according the “gender” variable table 3. t-test results of the prospective teachers’ attitude to teaching profession according the gender variable factors gender n ( x ) sd p total male 69 3.96 0.57 0.004 female 153 4.17 0.49 love male 68 3.83 0.61 0.005 female 153 4.06 0.53 value male 69 4.18 0.59 0.004 international electronic journal of elementary education vol.6, issue 2, 357-370,2014 362 table 3 (cont). t-test results of the prospective teachers’ attitude to teaching profession according the gender variable factors gender n ( x ) sd p female 153 4.40 0.47 coherence male 69 4.14 0.66 0.101 female 153 4.29 0.60 according to independentsample t-test, male and female prospective teachers’ attitudes to teaching profession points are very high while points of women are significantly higher than men. this finding would show that gender is an effective factor on prospective teachers’ attitude to teaching profession. table 4 includes the data of the comparison of prospective teachers’ attitude to teaching profession according to department variable. this is shown by one-way analysis of variance (anova). table 4. anova results of the prospective teachers’ attitude to teaching profession according the department variable department factors n ( x ) sd p difference science education (1) love 74 4.12 0.51 0.009* 1-3 value 74 4.43 0.51 0.022* 1-3 coherence 74 4.38 0.53 0.022* 1-3 total 74 4.23 0.46 0.007* elementary education (2) love 78 3.98 0.51 value 78 4.33 0.46 coherence 78 4.21 0.66 total 78 4.10 0.46 social sciences education (3) love 70 3.87 0.58 value 70 4.23 0.58 coherence 70 4.14 0.66 total 70 3.99 0.56 (*) p < 0.01 (according to social sciences education). according to the results of analysis, prospective teachers’ attitude to teaching profession points are the highest in science and the lowest in social sciences (department). points of prospective teachers studying social sciences education are significantly lower than the ones’ studying science education. this finding would show that department is an effective factor on prospective teachers’ attitude to teaching profession. table 5 includes the data of prospective teachers’ thinking styles. table 5. prospective teachers’ thinking styles, arithmetic mean and standard deviation value factors sub-dimensions n ( x ) sd functions: legislative 222 5.70 0.79 executive 222 5.22 0.94 judicial 222 5.17 0.93 forms monarchic 222 4.58 0.88 hierarchic 222 5.44 0.94 oligarchic 222 3.99 1.00 anarchic 222 4.41 0.94 levels global 222 4.49 0.99 local 222 4.32 1.04 scopes internal 222 4.72 1.09 an analysis of the relationship between prospective teachers’ / uygun & kunt 363 table 5 (cont). prospective teachers’ thinking styles, arithmetic mean and standard deviation value factors sub-dimensions n ( x ) sd external 222 4.55 1.14 leanings liberal 222 5.10 1.05 conservative 222 4.00 1.20 when we analyze table 5, we see that the most preferred thinking style is legislative ( x = 5,70).hierarchic ( x = 5,44), executive ( x = 5,22), judicial ( x = 5,17), liberal ( x = 5,10) are the following ones consecutively. the lowest thinking style is monarchic ( x = 3.99) and conservative thinking style ( x = 4.00) is higher than it. table 6 contains the data of the comparison of thinking styles according to the gender variable. table 6. t-test results of the prospective teachers’ thinking styles according to gender variable sub-dimensions gender n ( x ) sd p legislative male 69 5.78 0.77 ,262 female 153 5.65 082 executive male 69 5.08 0.91 ,122 female 153 5.29 0.95 judicial male 69 5.24 0.91 ,415 female 153 5.13 0.95 monarchic male 69 4.57 0.97 ,889 female 153 4.59 0.85 hierarchic male 69 5.25 0.96 ,045* female 153 5.52 0.92 oligarchic male 69 4.17 1.05 ,066 female 153 3.90 0.95 anarchic male 69 4.58 0.94 ,062 female 153 4.33 0.93 global male 69 4.55 1.02 ,361 female 153 4.45 0.99 local male 69 4.42 1.08 ,284 female 153 4.27 1.02 internal male 69 4.96 1.01 ,024* female 153 4.61 1.06 external male 69 4.55 1.12 ,989 female 153 4.56 1.15 liberal male 69 5.2 0.82 ,305 female 153 5.08 1.13 conservative male 69 5.20 1.30 ,047* female 153 3.81 1.14 when we analyze table 6, we see that hierarchic thinking style is significantly higher among the women and internal and conservative thinking style points are significantly higher among the men. there is not a significant difference in other thinking styles according to gender variable. table 7 contains one-way analysis of variance (anova) results of the comparison of prospective teachers’ thinking styles according to department variable. international electronic journal of elementary education vol.6, issue 2, 357-370,2014 364 table 7. t-test results of the prospective teachers’ thinking styles according to gender variable subdimensions department n ( x ) sd p difference legislative science education (1) 74 5.66 0.92 elementary education (2) 78 5.61 0.76 ,680 social sciences education (3) 70 5.82 0.90 ,253 total 222 5.70 0.74 executive science education (1) 74 5.45 0.91 elementary education (2) 78 5.10 0.78 ,026 2-1 social sciences education (3) 70 5.11 1.08 ,040 3-1 total 222 5.23 0.93 judicial science education (1) 74 5.15 0.88 ,088 elementary education (2) 78 4.96 0.84 ,003 2-3 social sciences education (3) 70 5.41 1.02 total 222 5.16 0.93 monarchic science education (1) 74 4.73 0.78 elementary education (2) 78 4.53 0.82 ,158 social sciences education (3) 70 4.48 0.84 ,083 total 222 4.57 0.88 hierarchic science education (1) 74 5.65 0.95 elementary education (2) 78 5.30 0.83 ,024 2-1 social sciences education (3) 70 5.37 1.08 ,074 total 222 5.44 0.92 oligarchic science education (1) 74 3.81 0.95 ,003 1-3 elementary education (2) 78 3.86 0.89 ,006 2-3 social sciences education (3) 70 4.31 1.04 total 222 3.99 1.01 anarchic science education (1) 74 4.30 0.96 ,015 1-3 elementary education (2) 78 4.26 0.82 ,006 2-3 social sciences education (3) 70 4.68 0.94 total 222 4.40 0.93 global science education (1) 74 4.50 1.03 elementary education (2) 78 4.39 0.85 ,477 social sciences education (3) 70 4.58 1.09 ,630 total 222 4.49 0.99 local science education (1) 74 4.23 1.07 ,055 elementary education (2) 78 4.15 0.96 ,013 3-2 social sciences education (3) 70 4.58 1.05 total 222 4.31 1.04 internal science education (1) 74 4.61 1.01 ,013 1-3 elementary education (2) 78 4.52 1.02 ,002 2-3 social sciences education (3) 70 5.06 1.11 total 222 4.85 1.09 external science education (1) 74 4.56 1.20 elementary education (2) 78 4.50 0.98 ,738 social sciences education (3) 70 4.62 1.03 ,845 an analysis of the relationship between prospective teachers’ / uygun & kunt 365 table 7 (cont). t-test results of the prospective teachers’ thinking styles according to gender variable subdimensions department n ( x ) sd p difference total 222 4.55 1.11 liberal science education (1) 74 5.20 0.98 ,030 2-1 elementary education (2) 78 4.83 1.09 social sciences education (3) 70 5.28 1.01 009 2-3 total 222 5.10 1.05 conservative science education (1) 74 3.73 1.11 elementary education (2) 78 3.96 0.98 ,213 social sciences education (3) 70 4.22 094 ,017* 3-1 total 222 3.96 0.89 when we analyze table 7, we see that there is a significant difference in “executive”, “judicial”, “hierarchic”, “monarchic”, “anarchic”, “global”, “local”, ”internal”, “liberal” and “conservative” thinking styles according to the program they study. there is a significant difference in “hierarchic” and “liberal” thinking styles between social sciences in favor of social sciences. there is a significant difference in ‘’executive’’ thinking style between science and others in favor of science. there is a significant difference in “monarchic”, “anarchic” and “internal” thinking styles between social sciences and others in favor of social sciences. “executive”, “monarchic” and “hierarchic” thinking styles’ points are highest among prospective science teachers. in addition, “legislative”, “judicial”, “monarchic”, “anarchic”, “global”, “local”, “internal”, “external”, “liberal” and “conservative” thinking styles’ points are highest among prospective social studies teachers. table 8 includes correlation calculations of relationship between prospective teachers’ thinking styles and their attitude to teaching profession. table 8. the relationship between prospective teachers’ thinking styles and their attitude to teaching profession factors love value coherence total legislative ,025 ,105 ,054 ,050 executive ,159* ,176** ,156* ,175** judicial ,037 ,107 ,092 ,066 monarchic ,162* ,074 ,045 ,135* hierarchic ,186** ,155* ,176** ,192** oligarchic -,010 ,004 -,016 -,009 anarchic ,011 ,016 ,009 ,012 global ,240** ,104 ,117 ,205** local -,068 ,004 ,029 -,039 internal -,059 -,048 -,067 -,062 external ,069 ,035 ,093 ,071 liberal ,068 ,115 ,130 ,095 conservative -,011 -,081 -,086 -,039 there is a statistically significant relationship between prospective teachers’ attitude to teaching profession points and “executive” (r = .175, p < .01), ”monarchic” (r = .135, p < .05), ”hierarchic” (r = .192, p < .01) and “global” (r = .205, p < .05) thinking styles. when we analyze the relationship between sub-dimensions, we see that relationships are positive except “monarchic”, “internal” and “conservative” thinking styles. according to these findings, we may say that there is a statistically significant relationship between thinking styles and prospective teachers’ attitude to teaching profession. international electronic journal of elementary education vol.6, issue 2, 357-370,2014 366 conclusions and discussions this study aims to show thinking styles of prospective teachers and the relationship between these thinking styles and prospective teachers’ attitudes to teaching profession. according to this, it has been tried to express whether students of the research show significant difference due to their gender and department. according to research, prospective teachers (fourth grade, last year of education) educated in dumlupınar university primary school education show positive attitudes to teaching profession. other researches support this finding (doğan and çoban, 2009; terzi and tezci, 2007; oral, 2004). this finding can be explained by their intentional choice, their love for teaching, their paying attention to teaching and their belief that they are going to be successful. female students’ attitude to teaching is more positive than male students’ in all aspects and results of other researchers are similar to this (aydın ve sağlam, 2012; çapri and çelikkaleli, 2008; özbek, kahyaoğlu and özgen, 2007; bozdoğan, aydın and yıldırım, 2007; terzi and tezci, 2007; öztürk, doğan and koç, 2005; saracaloğlu, serin, bozkurt and serin, 2004). this would be an outcome of their perceptions. there is a general idea among the people that women are more suitable for teaching. it is also suitable for women because of work and life conditions. the reason of higher average point of women related to teaching would be this ideal image accepted for women; this belief of society would support women’s positive attitude to teaching and their positive behaviors. attitude of fourth grade science teaching prospective teachers is more positive than class and social studies teaching prospective teachers. attitudes of science teaching prospective teachers are not significant with primary school teaching prospective teachers while it is significant with social studies teaching prospective teachers. attitude points in the studies of özbek, kahyaoğlu and özden (2007); terzi and tezci (2007) show that points of social studies teaching prospective teachers are higher than others. there is not a significant difference between social studies teaching prospective teachers and science teaching prospective teachers in the study of bozdoğan, aydın and yıldırım (2007). this would be the result of differences of universities, departments and instructors. most preferred thinking styles are legislative, hierarchic and executive. legislative thinking style is innovative and ideogenetic. students that adopt this style like to find solutions by themselves. hierarchic thinking style means paying attention to many goals at the same moment related to different duties and areas. those who adopt this style would do many things at the same moment by paying attention to their importance and by grading them. owing to their being aware of more necessary things, they are more inclined to organize events and problems (duru, 2004). executive thinking style is related to carrying out a duty and a practice. instead of obeying the existing mindsets, preferring structured problems and building a structure by themselves; they can be defined as individuals that like to practise available structures and methods. individuals that have this style may be said to have practicing structure (akbulut, 2006; innerst, 1998). these individuals try to do their best in a subject and abstain from the works that need independent working (buluş, 2005; grigorenko ve sternberg, 1997; sternberg, 1997). prospective teachers prefer oligarchic and conservative thinking styles less than others. oligarchic thinking style means to do many things at the same moment without grading them. they like to do something quickly but they have problems in ordering. conservative thinking style is traditionalist, realist and prefers the experienced. people of this style like to work according to experienced methods and they like to follow traditions. according to these findings, prospective teachers prefer an analysis of the relationship between prospective teachers’ / uygun & kunt 367 legislative; hierarchic and executive thinking styles more than others and oligarchic and conservative thinking styles less than others. according to oflar (2010)’s research on teachers, primary school teachers prefer legislative more and conservative thinking style less. this finding supports the present study. according to genders, hierarchic thinking style is significantly high among the women and internal and conservative thinking style is significantly high among the men. according to yıldızlar (2010)’s study, men are internal and more conservative. according to buluş (2005)’s study, men are more internal and conservative. according to başol and türkoğlu (2009), men are more conservative. however, according to grigorenko and sternberg (1997); and zhang (1999) there is not a significant difference according to gender. these differences would be the outcome of a subcultural features and attitudes of students. executive, monarchic and hierarchic thinking style points of science teachers are higher than others. legislative, judiciary, monarchic, anarchic, global, local, internal, external, liberal and conservative thinking style points of social studies teachers are higher. according to sünbül (2004)’s study, there is one significant difference based on some of prospective teachers’ departments. saracaloğlu et al (2008) could find a difference in a sub-dimension not in anything else. doğanay, akbulut-taş and erden (2007) show us that there is a significant difference according to their department. as a result, it can be said that prospective teachers’ thinking styles change according to their departments. there is a significant relationship between the total attitude point of prospective teachers to teaching and executive, monarchic, hierarchic and global thinking styles. if we analyze the relations between sub-dimensions, we see that relations are positive among thinking styles except monarchic, internal and external thinking styles. these findings show that there is a significant relationship between thinking styles and total attitude point of prospective teachers to teaching profession. prospective teachers’ having these thinking styles and their high attitude to teaching profession show the value, love and attention they pay to teaching. in addition, we may say that this would positively affect their relationships with their students (in classroom), their methods and their technics. suggestions prospective teachers in the faculties of education should be educated about thinking style leanings, and about the things that influence thinking styles. in addition, they should be educated about weak and strong points of these thinking styles. curriculum of education faculties and their programs should include activities, projects and programs that would help students to have different thinking styles. in order to make prospective teachers’ attitude to teaching better and to increase their professional consciousness, social and economic conditions of teaching should be enhanced. this research was carried out in science teaching, primary school teaching and social studies teaching departments. prospective teachers’ thinking styles and their attitude to teaching profession could be compared in different sample groups. • • • international electronic journal of elementary education vol.6, issue 2, 357-370,2014 368 mehmet uygun is an assistant professor in the department of elementary education at dumlupınar university, kutahya. his research interest areas are literacy, reading and writing skills. halil kunt is an assistant professor in the department of science education at dumlupınar university, kutahya. his research interest areas are science teaching and environmental education references aydın, r. & sağlam, g. (2012). öğretmen adaylarının öğretmenlik mesleğine yönelik tutumlarının belirlenmesi (mehmet akif ersoy üniversitesi örneği). türk eğitim bilimleri dergisi, 10(2), 257-294. bozdoğan, a. e., aydın, d. & yıldırım, k. (2007). öğretmen adaylarının öğretmenlik mesleğine ilişkin tutumları. ahi evran üniversitesi kırşehir eğitim fakültesi dergisi (kefad) 8(2), 8397. buluş, m. (2000). öğretmen adaylarında yükleme karmaşıklığı, düşünme stilleri ve bilişsel tutarlılık tercihinin bazı psikososyal özellikler ve akademik başarı çerçevesinde incelenmesi. i̇zmir: dokuz eylül üniversitesi eğitim bilimleri enstitüsü, yayımlanmamış doktora tezi. buluş, m. (2005). i̇lköğretim bölümü öğrencilerinin düşünme stilleri profili açısından incelenmesi. ege eğitim dergisi, 1(6), 1-24. çapri, b. & çelikkaleli, ö. (2008). öğretmen adaylarının öğretmenliğe ilişkin tutum ve mesleki yeterlik inançlarının cinsiyet, program ve fakültelerine göre incelenmesi. i̇nönü üniversitesi eğitim fakültesi dergisi, 9(15), 33-53. çetin, ş. (2006). öğretmenlik mesleği tutum ölçeğinin geliştirilmesi (geçerlik ve güvenirlik çalışması). gazi üniversitesi endüstriyel sanatlar eğitim fakültesi dergisi, 18, 28-37. çubukçu, z. (2004). öğretmen adaylarının düşünme stillerinin belirlenmesi. trakya üniversitesi sosyal bilimler dergisi, 5(2), 87-106. demirel, ö. (2005). eğitim sözlüğü. ankara: pegem akedemi. doğan, t. & çoban, a. e. (2009). eğitim fakültesi öğrencilerinin öğretmenlik mesleğine yönelik tutumları ile kaygı düzeyleri arasındaki ilişkinin incelenmesi. eğitim ve bilim dergisi, 34(153), 157-168. doğanay, a., akbulut-taş, m. & erden, ş. (2007). üniversite öğrencilerinin bir güncel tartışmalı konu bağlamında eleştirel düşünme becerilerinin değerlendirilmesi. kuram ve uygulamada eğitim yönetimi, 52, 511-546. duru, e. (2002). öğretmen adaylarında kişi-durum yaklaşımı bağlamında yardım etme davranışı eğilimi empati ve düşünme stilleri ilişkisi ve bu değişkenlerin bazı psikososyal değişkenler açısından incelenmesi. i̇zmir: dokuz eylül üniversitesi eğitim bilimleri enstitüsü,yayımlanmamış doktora tezi. duru, e. (2004). düşünme stilleri: kavramsal ve kuramsal çerçeve. eğitim araştırmaları, 14, 171-186. emir, s. (2013). öğretmenlerin düşünme stillerinin eleştirel düşünme eğilimlerini yordama gücü (i̇stanbul-fatih örneği). kuram ve uygulamada eğitim bilimleri. 13(1), 325-347. fer, s. (2005). düşünme stilleri envanterinin geçerlik ve güvenirlik çalışması (validity and reliability study of the thinking styles inventory). kuram ve uygulamada eğitim bilimleri (educational sciences theory & practice), 5(2), 433-461. an analysis of the relationship between prospective teachers’ / uygun & kunt 369 gay, l. r., mills, g. e. & airasian, p. (2006). educational research: competencies for analysis and application (8th. ed.). upper saddle river, nj: prentice hall. i̇nceoğlu, m. (2004). tutum, algı, iletişim. ankara: elips kitap kesit tanıtım. ltd. şti. kaptan, s. (1998). bilimsel araştırma ve istatistik teknikleri. ankara: tekışık web ofset tesisleri. karasar, n. (2011). bilimsel araştırma yöntemi (22. baskı). ankara: nobel yayın dağıtım oral, b. (2004). eğitim fakültesi öğrencilerinin öğretmenlik mesleğine ilişkin tutumları. eğitim araştırmaları, 15, 88-98. özbek, r, kahyaoğlu, m. & özgen, n. (2007). öğretmen adaylarının öğretmenlik mesleğine yönelik görüşlerinin değerlendirilmesi. sosyal bilimler dergisi, ix(2), 221-232. özden, y. (2003). öğrenme ve öğretme. ankara: pegema yayıncılık. öztürk b., doğan, o. & koç, g. (2005). eğitim fakültesi öğrencileri ile fen-edebiyat fakültesi mezunlarının öğretmenlik mesleğine yönelik algılarının karşılaştırılması (gazi üniversitesi örneği). türk eğitim bilimleri dergisi,3(1),1-19. saban, a. (2005), öğrenme, öğretme süreci. ankara: nobel yayın dağıtım. saracaloğlu, a. s., serin, o., bozkurt, n. & serin, u. (2004). öğretmen adaylarının mesleğe yönelik tutumlarını etkileyen faktörler. çağdaş eğitim, 29(311), 16-27. saracaloğlu, s., yenice, n. & karasakaloğlu, n. (2008), eğitim fakültesi öğrencilerinin düşünme stillerinin çeşitli değişkenler açısından karşılaştırılması. uluslararası sosyal araştırmalar dergisi, 1/5, 732-751. sternberg, robert j. (1997). thinking styles. new york: cambridge university press. sternberg, r. j., grigorenko, e. l., & zhang, li-f. (2008). styles of learning and thinking matter in instruction and assessment. perspectives on psychological science, 36, 486-506. sünbül, a. m. (2004). düşünme stilleri ölçeğinin geçerlik ve güvenirliği. eğitim ve bilim dergisi, 132, 25-42. terzi, a. r. & tezci, e. (2007). necatibey eğitim fakültesi öğrencilerinin öğretmenlik mesleğine ilişkin tutumları. kuram ve uygulamada eğitim yönetimi, 52, 593-614. üstüner, m. (2006). öğretmenlik mesleğine yönelik tutum ölçeğinin geçerlik ve güvenirlik çalışması. kuram ve uygulamada eğitim yönetimi, 45,109-127. zabukovec, z., & kobal-grum, d. (2004). relationship between student thinking styles and social skills. psychology science, 46, 156-166. zhang, l. f. (1999). further cross-cultural validation of the mental self-goverment. the journal of psychology, 133, 165-181. zhang, l. f. & sternberg, r. j. (2005). a threefold model of intellectual styles. educational psychology review, 17(1), 1-53. international electronic journal of elementary education vol.6, issue 2, 357-370,2014 370 www.iejee.com this page is intentionally left blank the effectiveness of the creative writing instruction program based on speaking activities (cwipsa) international electronic journal of elementary education, june 2016, 8(4), 617-628. issn:1307-9298 copyright © iejee www.iejee.com the effectiveness of the creative writing instruction program based on speaking activities (cwipsa) seher bayata a university of ordu, turkey received: 21 october 2015 / revised: 2 june 2016 / accepted: 8 june 2016 abstract this study aims to develop a creative writing instruction program based on speaking activities and to investigate its effect on fourth-grade primary school students' creative writing achievements and writing attitudes. the experimental method based on the pre-test/post-test model was used in this research. the research was conducted with 42 students, who were picked from fourth-grade primary school students studying in ordu, turkey in the academic year of 2014/2015. the students were divided into two groups: experimental group (21 students) and control group (21 students). cwipsa was applied to the experimental group and the standard fourth-grade grade turkish curriculum was applied to the control group. it was found that, as a result of the research, there was an increase in the control group students' scores for writing attitudes and creative writing achievements, as well as a significant difference between the control and experimental groups' scores for writing attitude and creative writing achievements, in favour of the experimental group. keywords: creative writing, speaking activities, instruction program, creative writing achievements, writing attitudes. introduction language learning is based on four basic linguistic skills, which it aims to improve: listening, speaking, reading and writing. these are the basic skills that affect an individual’s academic achievement throughout their academic lives and need to be used effectively both in day-to-day life and during their careers. in addition to these four basic skills, creativity is another skill which language education aims to provide to individuals in the turkish curriculum. as highlighted in the literature (fishkin & johnson, 1998; honig, 2001; runco, 2003), creativity can be defined in the broadest sense as “originality; going beyond the ordinary; the ability to think independently and versatilely”. there are two different views on creativity: the first one is that it is innate; the second one is that it is a skill that exists in everyone and can be improved and revealed. according to runco (2003), everyone unequivocally accepts the creativity of people such as michelangelo, mozart and mark twain. however, as far as the creativity of children in the educational environment is concerned, children who are original and beyond the norm can be defined as being creative even though this is  corresponding author: seher bayat, faculty of education, department of primary education, university of ordu, turkey. phone: +(90) 505 598 27 20 e-mail: seherbayat@hotmail.com http://www.iejee.com/ international electronic journal of elementary education vol.8, issue 4, 617-628, june 2016 618 unbelievable to adults. common behaviours that include the personality traits of people who have the ability to think creatively can be listed as follows: 1) thinking differently than others: the most important behavioural trait of people with improved creativity is confidence. 2) delaying judgment: individuals usually judge their thoughts without evaluating them and tend to classify and place their thoughts in a systematic manner, even doing it unwittingly most of the time. 3) thinking flexibly: they think without rules, boundaries or principles. they do not see the world in black and white. they use the gray in every area of life and they are flexible. 4) spontaneity: thinking fast, they make it possible for many ideas to reveal themselves without passing through the filter of logic. 5) synthesizing: this is the ability to associate certain concepts, ideas and symbols with each other, using imagination and achieving new, creative and different results. the more such associations are performed, the more creative ideas come out (temizkan, 2010: 624-625). the way that the educational environment is organized is also important for the development of the creativity the educational programs aim to provide students with. producing creative and original ideas is only possible in a free educational environment. it is important that students can communicate with their friends and teachers comfortably, that they know that they will not be judged for their ideas and that they can feel safe and at ease. evaluation should not leave students just feeling good or bad about what that have done, it should inform learners about the valuable parts of their works and provide guidance for future actions (brookfield, 2015). in addition, the second important factor is that methods and techniques which improve creative thinking are used in the learningteaching process. as stated by marzano et al. (1988), “creative thinking can also be taught with the aid of methods and materials especially designed for that purpose”. students can mention their ideas through their speaking and writing skills, which are the narrative skills used in turkish classes. therefore, it should be an aim that, through speaking and writing activities, the skill of creativity is improved and creative ideas are revealed. one of the types of writing addressed in the turkish curriculum is creative writing. creative writing is the activity of reconstructing current knowledge, concepts, sounds, images and dreams in the memory and associating them with each other, therefore creating a new piece of writing; it is the transfer of an individual's feelings and thoughts about something onto paper, done freely, through imagination (arthur & zell, 1996). based on the process approach commonly used in the literature (akyol, 2012; bayat, 2014; çoşkun, 2009; seban & tavsanlı, 2015; tekşan, 2013), creative writing studies should be performed in accordance with the steps of preparation prewriting, drafting and writing by organizing, editing and publishing/sharing. sharples (1996) also suggested that creative writing activities should be performed within the cycle seen in figure 1. the effectiveness of the creative writing instruction program based on speaking activities / bayat 619 figure 1. the cycle of engagement and reflection in writing (sharples, 1996:7). the process approach rather than the product approach should be adopted in creative writing because creative writing requires editing, planning and the exploration of thoughts without constraint. the importance of this approach in terms of children's writing development has been stressed in a recent global review of writing pedagogy (dombey, 2013). utilizing speaking activities in the preparation step of the creative writing process not only enables speaking and writing skills to be managed in a holistic way but also for the improvement of creative ideas. in the preparation stage, the development of creative thoughts can be contributed to by generating ideas, discussing ideas and sharing through techniques such as brainstorming, creative drama, the station method, marketplace, crime scene investigation, debate and the six thinking hats technique. according to marzano, debra & ronald (1990), if schools are to raise more skilful thinkers, far more thoughtful interaction must occur in classrooms, ranging from large group discussion of controversial issues to small group and paired problem solving. based on this it was aimed, in this study, to investigate the effect a creative writing curriculum based on speaking activities would have on students' writing attitudes and achievement levels for their creative writing skills. in order to be active, self-confident, independent, creative, social and show personal adaptation, individuals need to analyze ideas in their minds and to express them properly (demir, 2010). it is thought that this research will reveal the effect of performing speaking and writing activities at the same time, based on the techniques that improve creative thinking in the educational environment and using the “preparation” stage effectively in the process approach to creative writing. it is possible to come across studies that investigate the effects of certain instruction methods and techniques (creative drama, cooperative teaching method, creative writing approach, etc.) in the literature (colantone, cunningham-wetmore & dreznes, 1998; erdoğan, 2013; maden & durukan, 2010; mayo, 1992; vass, 2007). however, what was designed in this study is a holistic instruction program that includes multiple instruction methods for improving creative thinking. accordingly, it is thought that investigating the effectiveness of the product will contribute to the branch. 1.1. problem/question: what effect does the creative writing instruction program based on speaking activities (cwtpsa) in primary school fourth-grade turkish classes have on students' writing attitudes and creative writing achievements? 1) what is the effect of cwipsa on fourth-grade students' writing attitudes? international electronic journal of elementary education vol.8, issue 4, 617-628, june 2016 620 2) what is the effect of cwipsa on fourth-grade students' creative writing achievements? 3) is there a significant difference between the experimental and control groups' creative writing achievements? 4) is there a significant difference between the experimental and control groups' writing attitudes? method the research design is a pre-test/post-test control group quasi-experimental design. pretests were applied to 88 students in total, who were picked randomly from two different schools and four classrooms in ordu central district. according to the pre-test results, one class, randomly selected among two peer classes, was assigned as the control group and the other as the experimental group. the data concerning the control and experimental groups according to the pre-test results are given in the tables below. table 1. experimental and control groups' writing attitude scores groups n m sd df t p experimental group 21 123.14 17.94 40 .344 .733* control group 21 121.24 17.95 * p ≤ 0.05 p value was found to be “.733” as a result of the t-test. since the p value was found to be higher than 0.05, there is no significant difference between experimental and control groups' writing attitudes. it can be said that the two groups were equal in terms of writing attitudes. * p ≤ 0.05 p value was found to be “.744” as a result of the t-test. since the p value was found to be 0.05 higher, there is no significant difference between the experimental and control groups' creative writing achievements. it can be said that the two groups were equal in terms of creative writing achievements. data collection tools “attitude to writing scale (aws)”, developed by susar-kırmızı (2009), was used in the study. the scale aimed to determine the attitudes to writing among students at fourth and fifth grades of primary school (susar-kırmızı, 2009). in the formation of the scale, primarily literature reviews and students’ views were consulted. the first form of the itemized scale was presented to experts. to determine students’ agreement level to these items, judgment of agreement to statements was presented in five scales in the likert-type items: “strongly agree, highly agree, agree, agree somewhat, do not agree”. the pilot study of the 52-item scale was conducted on students at fourth and fifth grades of state and private primary schools (n= 258). to determine the construct validity of aws, the data obtained in the pilot study were subjected to factor analysis. before factor analysis was administered on the data from the pilot study, the suitability for factor analysis was table 2. experimental and control groups' creative writing pre-test scores groups n m sd df t p experimental group 21 33.57 16.59 40 .329 .744* control group 21 31.90 16.23 the effectiveness of the creative writing instruction program based on speaking activities / bayat 621 examined. for this, the kmo (kaiser-meyer-olkin) value was taken as the reference value. the first results of the analysis of aws showed that the kmo value of the scale was 0.90. based on the results obtained from this analysis, the scale was deemed suitable (pallant, 2001; in akkaya & susar-kırmızı, 2010). whether the data came from multiple variables in normal distribution was tested using the bartlett test of sphericity value. the result obtained from the bartlett test of sphericity administered on the data was approx. chisquare = 5864.589. the result of the bartlett test of sphericity was found to be significant at 0.05 (p= 0.000). following these analyses, the data were found to be fit for factor analysis. then, rotated factor analysis was administered. when 18 items were eliminated, the final form of the scale was given with the remaining 34 items. factor eigenvalues of the items ranged between 0.50 and 0.86. the cronbach’s alpha reliability coefficient was found as 0.90. the weighted raw score that each student might get was a minimum of 34 and a maximum of 170. high total scores indicate that students have positive attitudes, while low total scores indicate that students have negative attitudes. cronbach's alpha reliability coefficient for the scale was found to be 0.87 in the research. two other data collecting tools developed by susar-kırmızı (2011) were used during the research: “open-ended question/subject of composition for creative writing” and “evaluation criteria for creative writing (eccw)”. the students were asked to write compositions on the following subject at the beginning and the end of the program: “assume that you are in a time tunnel. you can travel back or forward in time. to which period would you like to go? what kind of a world would you like to see in that period?” the compositions were evaluated according to the criteria given in table 3. table 3. evaluation criteria for creative writing criteria score 1 the content includes certain innovations out of the ordinary 20 2 a well-known element is expressed in a new way in the composition 20 3 there are original similes in the composition 15 4 the new ideas in the composition are clearly explained 10 5 emotions and ideas in the composition are put forth effectively and fluently 20 6 the title is appropriate for the composition 15 total 100 the data were collected in one hour classes, after necessary explanations had been made by the researcher, once before and once after the experiment. the compositions were analyzed by two different experts for the data analysis. the correlation value between the data obtained from the compositions individually graded by the researchers is “r = .81”. it can be inferred that this value is the indicator of the reliability of the data obtained. the instruction program applied to the experimental group the instruction program was applied by the researcher for 22 class hours (1 class hours= 40 minutes) over a period of 10 weeks. the researcher had 12 years’ experience of teaching classes, as well as an academic specialty. international electronic journal of elementary education vol.8, issue 4, 617-628, june 2016 622 table 4. the creative writing instruction program based on speaking activities applying the pre-test and the attitude scale week /class hour subject method – technique type of text evaluation week 1 1 class hour examination of discussions, criticism and conversations direct instruction, question-answer discussion, letter multiplechoice test week 1 1 class hour examination of letters, stories, tales, etc. direct instruction, question-answer story, tale multiplechoice test week 2 1 class hour examination of fable, poem, writing direct instruction, question-answer fable, poem multiplechoice test week 2 1 class hour writing rules in respect to the process approach question-answer, practice …….. week 3 2 class hours if i were to talk to my favourite artist gossip letter peer/selfevaluation week 4 2 class hours i am building a new school six hats thinking technique story peer/selfevaluation week 5 2 class hours i am the protagonist of a tale marketplace tale peer/selfevaluation week 6 2 class hours in the realm of animals case study fable peer/selfevaluation week 7 2 class hours love of atatürk/mother/father/n ature etc. brainstorming poem peer/selfevaluation week 8 2 class hours travel to a new planet creative drama tale peer/selfevaluation week 9 2 class hours contribution/harms of social networks to our lives debate discussion peer/selfevaluation week 10 2 class hours the place of technology in our lives station method student selection peer/selfevaluation week 11 2 class hours publishing-sharing publishing/sharing 22 class hours applying the pre-test and the attitude scale as seen in table 4, the researcher gave lectures on types of writing in the first two weeks of the instruction program to the experimental group. for eight weeks, speaking activities that improve creative thinking and activities for producing ideas and designing what to write (preparing a draft) were performed in the first class hour; the second class hour was allocated to writing and evaluation of writing by students. peer and self-evaluation methods were used as the evaluation method. on the other hand, the turkish curriculum was applied to fourth-graders with no change. the effectiveness of the creative writing instruction program based on speaking activities / bayat 623 analysis of data the arithmetic mean, standard deviation, t-test and effect size were calculated in the quantitative data analysis. the spss 13 software package was utilized in the calculations. findings data concerning the second sub-problem are shown in table 5. table 5. t-test results for experimental group's writing attitudes * p ≤ 0.05 the p value was found to be “.000” as a result of the t-test. since it is p ≤ 0.05, a significant difference was found between pre-test and post-test scores for writing attitude of the experimental group. as for the means, it is seen that the pre-test attitude scores were “m= 121.23” and the post-test attitude scores were “m= 142.95”. it can be said in respect to this finding that the instruction program based on the speaking activities that improve creative thinking contributed to writing attitude positively. in addition, it was determined that the effect size calculated (cohen’s d= 1.23) was on a high level. data concerning the first sub-problem are shown in table 6. table 6. t-test results for experimental group's creative writing achievements groups n m sd df t p pre-test 21 33.57 16.59 40 5.49 .000* post-test 21 63.09 18.17 * p ≤ 0.05 as a result of the t-test, p value was found to be “.000”. since it is p ≤ 0.05, a significant difference was found between pre-test and post-test scores for creative writing of the experimental group. as for the means, it is seen that the pre-test achievement was “m= 33.57” and the post-test achievement was “m= 63.09”. it can be said in respect to this finding that the instruction program based on the speaking activities that improve creative thinking contributed to the creative writing achievement positively. in addition, it was determined that the effect size calculated (cohen’s d= 1.73) was on a high level. data concerning the third sub-problem are shown in table 7. table 7. t-test results for experimental and control groups' creative writing achievements * p ≤ 0.05 p value was found to be “.000” as a result of the t-test. since it is p ≤ 0.05, a significant difference was found between experimental and control groups' post-test scores for creative writing. as for the means, it can be seen that the experimental group was “m= 63.09” and the control group was “m= 36.19”. it can be said in respect to this finding that groups n m sd df t p pre-test 21 121.23 17.95 40 .3.88 .000* post-test 21 142.95 18.30 groups n m sd df t p experimental group 21 63.09 18.17 40 5.716 .000* control group 21 36.19 11.60 international electronic journal of elementary education vol.8, issue 4, 617-628, june 2016 624 the instruction program based on the speaking activities that improve creative thinking made a significant difference in creative writing achievement. in addition, it was determined that the effect size calculated (cohen’s d= 1.80) was on a high level. data concerning the fourth sub-problem are shown in table 8. table 8. data concerning experimental and control groups' writing attitude scores * p ≤ 0.05 p value was found to be “.002” as a result of the t-test. since it is p ≤ 0.05, a significant difference was found between experimental and control groups' post-test scores for writing attitude. as for the means, it can be seen that the experimental group was “m= 142.95” and the control group was “m= 123.23”. it can be said in respect to this finding that the instruction program based on the speaking activities that improve creative thinking is a factor that makes a significant difference in writing attitude. in addition, it was determined that the effect size calculated (cohen’s d= 1.03) was on a high level. discussion, results and recommendations according to the findings of the research, it was found that there was an increase in the writing attitude scores of fourth-grade students who undertook cwtpsa, as well as a significant difference between control and experimental groups' scores for writing attitude in favour of the experimental group. studies on the effect of creative writing activities on writing attitude (ak, 2011; beydemir, 2010; conroy, 2009; erdoğan, 2013; maden & durukan, 2010) support the findings of this research. however, examining the characteristics of this program and the informal researcher observations may give clearer ideas about the increase in the attitude scores since different programs/methods are applied for creative writing activities in every study. it can be said that techniques such as the six hats thinking method, snowball, marketplace, drama, etc. utilized in the opinion generation stage of cwipsa contributed to students' willingness to participate in the class. it is also worth noting that selecting the composition subjects from those that attract students' attention (travel to a new planet, if i were to talk to my favourite artist, i am building a new school, etc.) is an important factor for them to enjoy writing activities. if students write about something that draws their attention, their motivation will increase and they will want to write (anneralla, 2000; bruing & horn, 2000; daly & sharko, 2010). moreover, it was observed that they felt excited/happy about having a small-scale magazine at the end of the program (see annex 2). it can be concluded that these aspects of cwipsa were the reasons why the experimental group's writing attitude scores increased and were different than the control group’s. the increase in the experimental group’s achievements in creative writing might have affected their attitudes in a positive way, just as demir (2013) determined that students with high writing competence are generally good at creative writing. according to the findings of the research, it was found that there was an increase in the creative writing achievement scores of fourth-grade students who had undertaken cwtpsa, as well as a significant difference between control and experimental groups' scores in creative writing achievements in favour of the experimental group. similar studies in the literature (beydemir, 2010; colantone, cunningham-wetmore and dreznes, groups n m sd df t p experimental group 21 142.95 18.30 40 3,263 .002* control group 21 123.23 20.79 the effectiveness of the creative writing instruction program based on speaking activities / bayat 625 1998; conroy, marchand & webster, 2009; erdoğan, 2013; maden & durukan, 2010; wendell, 1992; susar, 2008; vass, 2007) support the results of this research. studies showed that students' creative writing achievements increased over a certain period when creative writing activities were performed. each individual has the power of creative thinking; however, the creativity of students can improve with the support of teachers and the exercises and techniques within the process (runco, 2003). cunningham and allington (1994) argue that practice of writing stages by teachers is a critical factor for revealing successful authors. the process approach was adopted in cwipsa and the speaking activities that improve creative thinking were conducted along with a holistic program, including multiple methods and techniques in which students could express their ideas actively and in cooperation. one class hour during the writing activities was allocated to speaking activities and the idea generation stage. it was observed that students had fun with the methods and techniques selected for the speaking activities. in the research by akkaya (2014), the idea that “creative writing can be perceived as a game or enjoyment by the students” was suggested. creating a class environment in which students could feel free and comfortable in the speaking activities was of great importance. original ideas were supported during the activities and students were encouraged to believe that none of their thoughts would be viewed as extreme, ridiculous or wrong and that there were no boundaries for their imagination. they were provided with the chance to share their ideas and sentences with their friends during the writing stage from time to time. in the examination of compositions, original ideas were emphasized and criticized in a constructive way by the researcher. according to the research results, the following suggestions can be made for practitioners and researchers in respect to the creative writing instruction. 1) methods and techniques that ensure students' active participation in the class and which students will enjoy should be used in creative writing studies. 2) original subjects that will attract students' attention should be chosen for creative writing studies. 3) creating a class environment in which students can feel free and comfortable during writing instruction is of great importance. 4) the process approach should be adopted in creative writing activities and applied along with the speaking activities. 5) enough time should be spared for the preparation, planning, evaluation and sharing stages of the process approach. 6) creative writing activities should be given enough time in language education. 7) cwipsa's effectiveness in wider student groups can be separately investigated. • • • international electronic journal of elementary education vol.8, issue 4, 617-628, june 2016 626 references annarella, l. a. (2000). using creative drama in the writing and reading process. retrieved from http://www.eric.ed.gov/pdfs/ed445358.pdf ak, e. (2011). yaratıcı yazma tekniklerinin ilköğretim 5. sınıf öğrencilerinin türkçe dersindeki yazılı anlatım becerileri üzerindeki etkisi. unpublished doctoral dissertation. dokuz eylül üniversitesi, i̇zmir. akkaya, n. (2014). elementary teachers’ views on the creative writing process: an evaluation. educational sciences: theory & practice, 14(4), 1499-1504. akkaya, n., & susar kırmızı, f. (2010). relationship between attitudes to reading and time allotted to writing in primary education. procedia-social and behavioral sciences, 2(2), 4742-4746. akyol. h. (2012). türkçe öğretim yöntemleri. ankara: pegem akdemi yayınları. arthur, b. & zell, n. a. (1996). strategy for teaching creative writing skills to emotionally disturbed students. preventing school failure, 34(4), 26-31. bayat, n. (2014). the effect of the process writing approach on writing success and anxiety. educational sciences: theory & practice, 14(3), 1133-1141. beydemir (2010). i̇lköğretim 5. sınıf türkçe dersinde yaratıcı yazma yaklaşımının yazmaya yönelik tutumlara, yaratıcı yazma ve yazma erişisine etkisi. unpublished master dissertation. pamukkale üniversitesi sosyal bilimler enstitüsü, denizli. brookfield, s. d. (2015). the skillful teacher: on technique, trust, and responsiveness in the classroom. san francisco: john wiley & sons. bruning, r., & horn, c. (2000). developing motivation to write. educational psychologist, 35(1), 2537. https://books.google.com.tr colantone, l., cunnigham-wetmore, m. & dreznes, j. (1998). improwing creative writing. research project reports, saint xavier university, chicago. conroy, m, marchand, t. & webster, m. (2009). motivating primary students to write using writer's workshop. online submission, master of arts action research project, saint xavier university. http://www.eric.ed.gov/ ed504817 çoşkun, e. (2009) “yazma eğitimi”, i̇lköğretimde türkçe öğretimi (ed. h. akyol) ankara: pegem akademi yayıncılık. cunningham, p. & allington, r. (1994). classrooms that work: they can all read and write. new york, ny: harper collins college publishers. daly, l. & sharko, s. (2010). motivating students to write through the use of children's literature. online submission, master of arts action research project, saint xavier university. http://www.eric.ed.gov/ed509396 demir, tazegül. (2010). konuşma eğitiminde benmerkezci konuşmaya yönelik bir deneme. kastamonu eğitim dergisi, 18(2), 415-430. demir, t. (2013). i̇lköğretim öğrencilerinin yaratıcı yazma becerileri ile yazma özyeterlik algısı ilişkisi üzerine bir çalışma. uluslararası türkçe edebiyat kültür eğitim (teke) dergisi, 2(1).84114. dombey, h. (2013). what we know about teaching writing. preschool and primary education, 1, 2240. erdogan, t. (2013). the effect of creative drama method on pre-service classroom teachers’ writing skills and attitudes towards writing. australian journal of teacher education, 38(1), 4. fishkin, a. s., & johnson, a. s. (1998). who is creative? identifying children's creative abilities. roeper review, 21(1), 40-46. the effectiveness of the creative writing instruction program based on speaking activities / bayat 627 honig, a. s. (2001) how to promote creative thinking? early childhood today, 15(5), 7-34. maden, s. & durukan, e. (2010). i̇stasyon tekniğinin yaratıcı yazma becerisi kazandırmaya ve derse karşı tutuma etkisi. türklük bilimi araştırmaları (tübar), 28, 299–312. marzano, r.j., debra j. p., & ronald s. b.(1990). integrating instructional programs through dimensions of learning. educational leadership. 47(5), 17-24. mayo, w. (1992). creative writing and collaborative learning. retrieved from http://www.eric.ed.gov/fulltext/ed343147.pdf runco, m. a. (2003). education for creative potential. scandinavian journal of educational research, 47(3), 317-324. seban, d. & tavsanlı, ö. f. (2015). children’s sense of being a writer: identity construction in second grade writers workshop. international electronic journal of elementary education, 7(2), 217234. susar-kırmızı f. (2009). türkçe dersinde yaratıcı drama yöntemine dayalı yaratıcı yazma çalışmalarının yazmaya yönelik tutuma etkisi [effects of creative writing practices based on creative drama method in turkish lessons on attitudes to writing]. yaratıcı drama dergisi, 2(5). susar-kırmızı, f. (2011). yaratıcı yazma ürünlerinin bazı ölçütler açısından değerlendirilmesine ilişkin nitel bir çalışma. dil dergisi, 151, 22-35. sharples, m. (1996), an account of writing as creative design, in the science of writing, theories, methods, individual differences and applications (eds. c.m. levy & s. ransdell) pp. 127-148. lawrence erlbaum, mahwah, nj. temizkan, m. (2010). türkçe öğretiminde yaratıcı yazma becerilerinin geliştirilmesi. türklük bilimi araştırmaları, 27(27), 621-643. tekşan, k. (2013) yazma eğitimi. i̇stanbul: kriter yayınları. vass, e. (2007). exploring processes of collaborative creativity. the role of emotions in children's joint creative writing. thinking skills and creativity, 2(2), 107-117 international electronic journal of elementary education vol.8, issue 4, 617-628, june 2016 628 www.iejee.com this page is intentionally left blank http://www.iejee.com/ microsoft word iejee_4_3_habok international electronic journal of elementary education, 2012, 4(3), 459-472. issn:1307-9298 copyright © iejee www.iejee.com evaluating a concept mapping training programme by 10 and 13 year-old students anita habók ∗∗∗∗ university of szeged, hungary received: march, 2011 / revised: may, 2012 / accepted: june, 2012 abstract the pisa 2000 study found large differences between countries in terms of quality of learning. in some countries, students prefer rote learning to meaningful learning. however, such memorisation of the material does not lead to recallable and usable knowledge in the long run. ausubel's (1968) wellknown theory serves as a basis for several techniques of meaningful learning, including concept mapping, which emphasises the visual organization of comprehended information. the present study reports the results of an experiment using concept mapping for deepening students’ understanding of teaching materials. it focuses on definitions frequently used in science and hungarian grammar lessons. results indicate that concept mapping training programmes can improve participants’ achievement. experiences show that the technique of concept mapping is frequently unfamiliar to students. more time would be needed for practising and using it in different learning situations. keywords: meaningful learning; concept maps; training programme introduction in the process of teaching, two important questions arise. the first is how much fact and information students should learn. the second is in what way and in what textual form they should be presented. successful learning is a widely researched topic (csapó, 2007; waeytens, lens & vandenberghe, 2002). techniques and strategies that improve student achievement were investigated within the framework of these studies. when these learning methods are compared, the method of concept mapping is unique among visual learning techniques (novak, 1990; 1998; novak & gowin, 1984). concept mapping is widely used and has a variety of applications. among others, it is used to explore prior knowledge (gurlitt, renkl, faulhaber & fischer, 2007), to visually represent texts (hardy & stadehofer, 2006), and to reveal problems (barroso & crespillo, 2008) and misconceptions (berionni & baldoni, 2004, ∗ anita habók, university of szeged, 30-34. petőfi s. sgt., szeged, 6722, hungary, phone/fax: (+36) 62-420-034. e-mail: habok@edpsy.u-szeged.hu international electronic journal of elementary education, 2012, vol.4, issue 3, 459-472 460 himangshu, iuli & venn, 2008). in general, it is a tool that is used to facilitate meaningful learning and help students to represent their knowledge in a visual form. in hungarian schools, grammar is one of the most problematic subjects. one reason for this is that schools place an artificial burden on students by an early introduction of grammatical definitions and rules in mother tongue education. the pisa 2000 results show that hungarian students are relatively weak in using meaningful learning strategies that aid their conceptual understanding (artelt, baumert, julius-mcelvany & peschar, 2003, 40). these results indicate that students have a preference for using memorizing strategies. however, in the long run, rote learning does not lead to recallable knowledge. although rote learning can be useful for some learning purposes, the range of its possible applications is fairly limited. the problem arises when students use rote learning too frequently, and memorize information without seeking to establish connections between concepts. therefore, rote learning should be replaced by meaningful learning, which helps foster a deeper understanding of things. this underlying assumption of the need for meaningful learning serves as the basis for our developmental programme. the goals of our programme are to be accomplished via the use of concept mapping. there are numerous ways to visually construct concept maps. below, i provide an example by novak & canas (2003, figure 1.). figure 1. a concept map based on canas & novak (2008, 27) that shows the key features of concept maps in line with our definition. concept maps represent organized knowledge. they are composed of concepts that are connected by linking words. two concepts and the interconnecting words make up propositions, which comprise units of meaning. in the cognitive structure, knowledge itself is built up of propositions which can be connected to each other. the proposition of concepts in the hierarchical structure depends on our creativity and prior knowledge. concepts are stored in the form of objects and events, and they are labelled as symbols or words, which evaluating a concept mapping training programme by 10 and 13 year-old students / habók 461 may be connected by lines or arrows. in order to express the interrelationships between different levels we may use crosslinks. this kind of knowledge organization helps the process of teaching and learning to be more effective. in addition, concept maps help to answer focus questions that are context dependent. concept maps in practise concept maps have a wide range of uses. they are used in various areas, from kindergarten to higher education. now we will focus on various ways of applying concept maps. for instance, mancianelli et al. (2004) used concept maps to teach scientific language to 4-5 year old children. children were provided with samples related to everyday topics like pumpkins and nests. the process of learning started with the activation of prior knowledge, when students carried out experiments with the objects at their disposal. after discussing the subject’s properties, they drew a concept map. this early developmental technique attempted to acquaint children with concept mapping. similarly, berionni and baldoni (2004) also applied concept mapping in early childhood. berionni’s and baldoni’s concept maps were part of everyday school life and were drawn either individually or within the frame of a group activity. concept maps drawn by students informed teachers about students’ misconceptions and the dynamic processes of their thinking. poveda and oneca (2006) used the technique of concept mapping in elementary schools. their goal was to study the development of the concept mapping technique. in the long run, they set out to gather information on student learning techniques with the help of concept mapping. they intended to explore the efficiency of concept mapping as a learning technique, and its effectiveness as an assessment tool. by comparing concept maps drawn at the beginning and at the end of the learning process, poveda and oneca (2006) concluded that students created more correct propositions and more detailed concept maps by the end of the study. in ahlberg and vuokko’s (2004) study, concept maps were drawn by 20 elementary school students during two periods, each lasting three years. they set out to find out the teachers’ experiences during the study, the reliability of the concept mapping technique in the long term, its effect on learning, and how concept maps and gender differences can affect preschool performance. the two three-year periods encompassed 23 projects. the results revealed that students were able to create significantly more correct propositions by the end of the programme in both learning periods. in the first part of the study, the students’ preschool performance did not affect their achievement in making concept maps. in contrast, the results of the second part of the study showed that preschool achievement had a significant effect on concept mapping. the explanation the researchers provided was that the most important goal of their programme was simply that students learned, used and developed their learning skills. although it had a dramatic effect in each class, no significant differences could be detected concerning gender. halimi (2006) used concept maps to investigate novice translators’ text processing skills. translation is a complex process as it requires many skills, such as understanding written texts, finding the main ideas, expressing content in the second language, problem solving, decision making in various situations and recalling information. halimi (2006) investigated whether concept maps help students comprehend texts, and whether they could easily notice and recall the main ideas and grasp text cohesion. students were divided into two groups. the experimental group was introduced to concept maps as tools of text analysis. then, participants had the possibility to practise following instructions such as ‘select the main ideas from the text’, ‘connect these ideas to others’. in the meantime, the other group analysed texts using traditional text analysis techniques. in the second part of the study the roles of the two groups were transposed. at the end of the study the students in the international electronic journal of elementary education, 2012, vol.4, issue 3, 459-472 462 experimental group were requested to do free recalling on the texts, write down what they could remember and then answer open-ended questions. according to the results, concept maps facilitated the work of students. in conclusion, the group which was first introduced to the concept mapping achieved better results. halimi (2006) drew our attention to the role of practice; 20 hours of practice sessions per month proved to be too short a time for students to become familiar with the technique of concept mapping. vakilifard and armand (2006) combined the activation of text comprehension and prior knowledge in their study. they introduced concept maps to foreign language students. 18 students took part in the study, 9 students were placed the experimental group and 9 in the control group. the students in the experimental group took part in a four-session training course. at the beginning the group got a concept map in which nodes and linking phrases from a given list were to be completed. this exercise sought to activate prior knowledge. later, students were provided with a text. after having read it, students had the possibility of studying the given map and making changes based on their own opinions. during the study more and more parts of the concept maps, which were required to be reconstructed, were modified by the students. before reading the text, the control group was given a multiplechoice test to fill out. this activity was followed by reading the given text, and on the basis of this students could correct their previous answers based on prior knowledge. at the end of the study, the results of text comprehension were analysed in both groups. in one of the exercises it was expected that students would find the answer in the text, while in the other exercise, connections and relationships between concepts within the text had to be found and understood. the students’ performance in the experimental group turned out to be better than that in the control group. the experimental group was given questions at the end of the study to find out whether concept mapping had been helpful for them. almost all the participants reported that they found concept maps useful for exploring the text structure, finding the key ideas and establishing the relationship between them and organizing information. recently, it solutions in concept map research have gained growing popularity besides paper-pencil tests. more concept map software packages are available commercially1. reader and hammond (1994) requested students to create concept maps on pcs. the students’ working process and the alterations they made in a new window were recorded during the practical sessions. the organization of concepts, the place of concepts, the system of grouping and the working process were all analysed. students who connected all the concepts achieved the lowest scores. in this case, the students tried to summarize the ideas in a single figure regardless of their meaning. at the end of the programme, the researchers found that concept maps help students to organize their knowledge and to integrate new information in a meaningful way. they concluded that the learning of a text with a computer is more beneficial if students draw concept maps as opposed to taking standard notes. applying concept maps frequently plays a central role in collaborative learning. immonenorpana and ahlberg (2010) used this technique with physiotherapy students. nine out of 22 potential participants were selected to participate in their study. thus, the training group was composed of only nine students, out of which 3 students were high achieving, 3 students belonged to average achieving students and 3 students were low achieving students. a potential reason for the small size of sample might have been that researchers aimed at a more detailed analysis of thinking processes. besides heuristics, concept mapping was used for evaluating the learning process. the main focus was placed on the analysis of reflective metacognitive competence development. the study analysed concept maps 1 e.g. http://vue.uit.tufts.edu/, http://inspiration.com, http://cmap.ihmc.us/ evaluating a concept mapping training programme by 10 and 13 year-old students / habók 463 constructed during individual and collaborative work and examined the collaborative processes occurring during learning. the results showed that in the first two cases the differences between the individually created concept maps were very small, whereas considerable improvements were registered following shared collaborative learning. it was also observed that the atmosphere was very supportive and students communicated more compared to learning situations relying on traditional methods. concept maps also appeared in teacher training colleges. zanting, verloop and vermut (2003) asked students to use this technique to analyse and assess the practical knowledge of the mentor teachers. the researchers interpreted practical knowledge as complex knowledge that is related to teaching experience, integrates content knowledge and is related to students and the teaching situation. zanting, verloop and vermut (2003) provided a reason for studying practical knowledge from the viewpoint of mentor teachers. by observing them, teachers can profit from their own responses and explanations of their teaching. describing our own ideas and knowing how to formulate assumptions gives an opportunity to reflect on our own activities. for future teachers, this is helpful because this way they can gain access to undocumented knowledge. zanting, verloop and vermut (2003) showed how students could relate practice to theory and form their own opinions about teaching. the researchers were also interested in the students’ opinions regarding concept mapping. they used data gathered in interviews to assess the practical knowledge of mentor teachers. they found that over 70% of the students considered concept maps useful to a greater or lesser degree. one benefit they mentioned was that concept mapping helped them to study different opinions and ideas. furthermore, it helped teachers to reconsider their own views. some of the problems participants faced were that concept mapping did not reveal any new information and it was too artificial and abstract for them. in contrast, concept mapping was useful for mentor teachers because it helped them to clarify their ideas and gave them the opportunity to analyse their own behaviour. the construction of concept maps occurred spontaneously and helped them to analyse new experiences. on the whole, concept maps were more analytical and descriptive than the interviews. in the following section, we present the results of our study related to concept mapping, based on text comprehension and the understanding of definitions. aims and research questions ausubel’s theory (1968) served as the theoretical background for this study. in addition, the technique proposed by novak (1990, 1998) and novak and gowin (1984) was applied in the construction of a concept mapping intervention programme. here we seek to improve the students’ comprehension of rules by means of concept mapping. the main aims are that students should learn to find the main concepts in a text, discover the relationships within the text, and ultimately, learn to draw concept maps unaided. we set out to discover the effect of the concept mapping training programme in hungarian grammar and examine whether there are significant differences between the results of the experimental and control group. our hypotheses were the following: a) we expected to find significant differences between the performance of the experimental and the control group and we predicted that the experimental group would perform better at the posttest. b) as far as the types of concept mapping exercises are concerned, we expected to find significantly higher scores in the posttest when we compared the performance of the experimental group both on the pretest and posttest. international electronic journal of elementary education, 2012, vol.4, issue 3, 459-472 464 c) we expected that students would perform significantly better in completing exercises. we presumed that these types of exercises required only a superficial understanding and more instructions. d) we expected that the variables of the chosen background tests would explain the tasks of concept mapping to a great extent. we hypothesized that the technique of concept mapping would have a positive influence on learning strategies, learning styles, attitude and school marks in different subjects. methods samples the present study was carried out from october to february 2007 and was part of a largescale project on concept mapping with more than 200 participants. the random sample consisted of 10 and 13-year-old students. 10-year-old students are 4th graders and 13-yearold students are 7th graders in hungary. the participant classes were selected from partner schools of the research group on the development of competencies from the university of szeged, which took part in a large-scale project (other studies involved were e. g. antal, 2009; molnár, 2009). the classes chosen included more low-achieving and disadvantaged students. the participants were divided into an experimental group and a control group. in the study 61 10-year-old and 69 13-year-old students were included in the experimental group and 62 10-year-old and 55 13-year-old students formed the control group. the research design was complex since teachers used different course books and workbooks in different schools. there are three versions of learning materials in use in these schools by three different editors, each of which has a different content and structure. instruments and procedures while the pretest was based on students’ prior knowledge, the posttest included the topics of the concept mapping exercises. the subjects used for the intervention material were included in the national curriculum. the learning material was developed for 4th and 7th grade students. our study consisted of 25 sessions by 10-year-old students and 31 sessions by 13-year-old students. in the study, each exercise was constructed by the teachers of the given subject. the research procedure was constructed to serve the aims of the developmental programme by promoting meaningful learning. evaluating a concept mapping training programme by 10 and 13 year-old students / habók 465 figure 2. research design. the students worked on the given exercises, one per lesson, during the lessons. for the completion of each exercise, a time of five to ten minutes was allotted. each student was provided with a workbook containing the exercises. first, the students each had to read the text of the given exercise, then they had to follow instructions of the prompts in the workbook, and after they drew their own concept maps. after students had finished the exercise, they received feedback. the task types included: (1) completing a map by filling in the nodes and defining the linking words, and (2) constructing a map from scratch. below, i provide two examples from the hungarian grammar programme (figure 3. and 4.). pretest pre-tests in the different subjects demonstrated students’ prior knowledge. the control group was taught in the traditional way the experimental group solved the concept mapping exercises posttest post-tests included the issues of the concept mapping exercises. 1. 2. 3. international electronic journal of elementary education, 2012, vol.4, issue 3, 459-472 466 indefinite subject it is used when we cannot or do not wish to specify what the predicate modifies. indefinite pronouns and verbs in 3rd person plural can occur in this position. figure 3. sample task for 7th graders: completing a map by filling in the nodes. the object the suffix “-t” is the formal marker of the object in the sentence. as regards word class, object position is generally occupied by a noun or a member of other word classes which can act as nouns like the question words “whom” and “what” figure 4. sample task for 7th graders: constructing a map. the pretest was based on the students’ prior knowledge in the different subjects, in line with the national core curriculum requirements. after completing the pretest, the experimental group was requested to participate in a series of concept mapping tasks, while the control group was taught in the traditional way. the posttest sought to examine the new knowledge acquired after the pretest. preand posttests were constructed by teachers who were experts in their field. both tests and concept mapping exercises were each corrected by one teacher. 4.3. feedback for the students first, the teachers were given a workbook that also included the completed maps, serving as sample maps and unified starting-points. the students were presented with the solution of the given map after having completed their own. after examining the workbooks, both teachers and students were given the possibility to raise relevant questions. second, the students received their own workbooks without a key. these workbooks were used throughout the concept-mapping programme. written instructions were provided in the workbook, which fostered individual work. after each exercise, feedback was given in evaluating a concept mapping training programme by 10 and 13 year-old students / habók 467 different forms. (1) the teacher and the students discussed the solutions of the mapping task; the students could also consult their own maps. (2) the teacher and the students discussed the solution without the help of the exercise books. (3) there was no feedback – the teacher did not give help until the exercise was completed. having solved their tasks, the students could discuss them with the teacher. results tests students' knowledge was assessed at two different times. their degree of knowledge was measured using a knowledge test at the beginning and the end of the programme. cronbach’s alpha was in most cases very high: 4th graders pretest: 0.95, posttest: 0.95; 7th grader pretest: 0.94, posttest: 0.95. the tests contained different types of exercises. recall, converting and recognition were assessed in the pretest of hungarian grammar by 10-year-old students, while in the posttest recall and converting were assessed. as regards the tests of 13-year-old students, in the pretest recognition, implementation, recall, converting and interpretation were measured, while in the posttest recalling, recognition, implementation, and interpretation were in the focus. constructing the tests was quite a demanding task since teachers in the selected schools use different course books and workbooks. we had to take into account the potential differences in the learning material presented to various classes at a given time. consequently, pretests in every subject were based on students’ prior knowledge, which was common for every student. posttests analysed the learning material presented with concept maps. this was necessary in order to be able to study the results of the experimental group and compare the results the two groups. during the construction of the posttest we also ensured that the learning material was familiar to both of groups. school teachers provided help in the process of the construction of these tests. results of the survey our study was carried out in the subject of hungarian grammar. as stated in the first hypothesis, we assumed that we would find significant differences in the posttest and we presumed that the experimental group would achieve better results. furthermore, we predicted that the experimental group would perform better when we compared the pre and posttest results. figure 5. the results of the preand posttest (%p). international electronic journal of elementary education, 2012, vol.4, issue 3, 459-472 468 we did not discover significant differences between the experimental and control group in the 4th grade classes after comparing the pretest and posttest results. however, we found significant differences between the results of the two groups in the posttest, where the experimental group achieved better in the recall exercises (mexp.gr.=77.5%p, mcont.gr.=70.1%p; p <0.05). in addition, a significant difference was found in the performance of the experimental group for the preand posttest. thus, our hypothesis that the experimental group would perform significantly better in the posttest was confirmed (mexp.gr.=55.0%p, sdexp.gr.=18.2; mcont.gr.=69.6%p, sdexp.gr.=16.7; p=0.001). the performance of the control group changed significantly as compared to the preand posttest results as well. they achievement also showed increase (3.8%p, t=3.304, p=0.001). regarding the results of 7th graders significant differences were found between the experimental and control group on the pretest. here the experimental group performed less successfully in comparison with the control group (mexp.gr.=56.3%p, sdexp.gr.=13.9; mcont.gr.=67.2%p, sdexp.gr.=21.1). however, we did not find any significant differences after comparing the posttest results (mexp.gr.= 61.2%p, sdexp.gr.= 14,6; mcont.gr.= 60.2%p, sdexp.gr.= 23.3). it appears that the experimental group made up for their disadvantage. we investigated the differences between the preand posttest results of the two groups. on the one hand, a significant increase was found for the experimental group (t=-4.950, p=0.001), on the other hand we also observed a significant decrease in the achievement of the control group (t=4.135, p=0.001). overall, the experimental group performed better in the posttest, which supports our first hypothesis. students in the experimental and control group were placed into groups of three according to their results. the first group contained low-achieving students, the second group contained average ability students, while the third group included high-achieving students (table 1.). table 1. the number of the groups. 4th grade experimental 4th grade control 7th grade experimental 7th grade control group 1 (low-achieving) 8 7 6 3 group 2 (average) 36 29 46 18 group 3 (high-achieving) 17 26 17 34 one beneficial effect of the programme was that low-achieving students in the experimental group obtained significantly higher results in the posttest and we even observed a general improvement among them. in contrast, no changes could be detected in the achievement of the control group for group 1. however, group 3 showed a significant increase for 4th graders and a significant decrease for 7th graders. our results suggested that the increase in the students' performance could be attributed to the programme. in most cases we found evidence that the low-achieving students in the experimental group performed better on the posttest, but at the same time we saw no significant change or decrease in the control group. furthermore, we registered a significant increase for high-achieving students from the control group. evaluating a concept mapping training programme by 10 and 13 year-old students / habók 469 4th graders 7th graders figures 6-7. the results of the groups (%p) (significant differences are highlighted in bold and red p≤0.05). by comparing results of the different groups in grade 4, we noticed a significant improvement in the posttest in two cases. concerning the achievement of 7th graders, the lowand medium achievers showed a significant change in the posttest, furthermore, highachieving students reduced their disadvantage and the differences were levelled (figures 67.). we analysed the effect of the programme and got extremely promising results (d4thgrade=0.63; d7thgrade=0.66), which encouraged us to further test the effectiveness of the concept map technique in other school subjects. as stated in the third hypothesis, the two types of concept mapping exercises displayed significant differences. the 'completing a map by filling in the nodes and defining the linking words' type of exercises produced significantly higher results (m4thgraders=83.1%p; m7thgraders=76.1%p) in comparison with constructing a map (m4thgraders=64.2%p; m7thgraders=68.1%p). one possible explanation for this might be that these types of exercises only require a superficial understanding, while the creation of concept maps from scratch demands a deeper understanding and needs more organization. gap filling exercises proved to be more successful, which may be partly due to the fact that more texts provided more input for students. it appears that it was easier for students to look for pieces of information in the text than to draw a concept map on their own. in order to draw a concept map from scratch, the ability to gather information and high level organization skills were required. understanding was assessed on the basis of the layout and complexity of the concept map, which reflected their degree of comprehension of the given reading material. connections with other variables we carried out a regression analysis using the two types of concept maps in each given subject as dependent variables and results from preand posttests, inductive thinking test, learning methods and learning strategies questionnaire, hungarian grammar marks and attitudes as independent variables. in grade 4 we found that grammar pretest (34.5%) and learning activity (6.8%) explained almost than half of the completing exercises as dependent variable. one effect of constructing exercises as dependent variable could be seen in the grammar posttest (20.6%), while the other variables did not show any effect. overall, we may reasonably conclude that applying the concept map technique in the classroom had a positive effect for this age group. in grade 7, the effect of the type of concept mapping exercise was more considerable in both cases, as with completing exercises as dependent international electronic journal of elementary education, 2012, vol.4, issue 3, 459-472 470 variables the type of exercise accounted for 38,2% of the results, and with constructing exercises as dependent variables it accounted for 52.2% of the results. a weaker effect of the grammar posttest (27.2%) could be seen among the independent variables in the completing exercises. the completing exercises had the largest effect and accounted for the results of construction exercises. in summary, it may be concluded that based on our preliminary expectations, different types of concept map exercises had a clearly measurable effect on each other. moreover, we also noticed a beneficial effect in the knowledge tests. conclusions visual organizational tools can be applied in different ways in schools. they provide numerous opportunities to draw attention to various connections. they are mostly used for acquiring content knowledge. however, following the stage of acquisition of knowledge, self-regulated learning plays an important role in organizing newly acquired knowledge. in the concept-mapping programme, alongside with being asked to complete the exercises, students were introduced to a new learning strategy. they were more successful with ‘completing the maps by filling in the nodes’ exercises associated with a superficial understanding than they were with ‘constructing a map’ exercises. these exercises enabled students to work independently and organize information by themselves. the aim of our programme was to provide students with skills that would allow them to map a text and understand it thoroughly. we had favourable results among the students, consequently we claim that low-achieving students also profited from the programme. the concept mapping technique can definitely have a beneficial effect in the classroom and on student achievement. however, its effect depends on the student's abilities. in the present study, it had a positive effect on low-achieving students in hungarian grammar. the results suggest that success in the use of concept maps could be increased by tailoring the use of concept maps to students’ individual needs. unfortunately, acquiring and applying the technique of concept mapping in different fields is rather time-consuming. consequently, this technique should be applied on a longer time-scale and within a wider range of subjects in order to make the differences more apparent. further studies are currently being carried out based on these experiences from the first stage. in addition, research is extended to new areas for example science subjects. the pisa 2000 study found that hungarian students apply rote-learning strategies far too often. unfortunately, a long-term plan is required to change these habits. not only students, but also teachers will need additional support, learning material and practical demonstrations to use new techniques. however, as this study shows, the benefits of applying concept maps in the classroom are evident. . . . references ahlberg, m., & vuokko, a. (2004). six years of design experiments using concept mapping at the beginning and at the end of each of 23 learning projects. in a. j. canas, j. d. novak, & f. m. gonzález (eds.) theory, methodology, technology, proceedings of the first international conference on concept mapping. pamplona: spain. http://cmc.ihmc.us/papers/cmc2004-220.pdf, 2008. 12. 29. antal, e. (2009): the improvement of analogical reasoning of 7u8-years-olds. paper presented at the 13th biennial conference for research on learning and instruction, amsterdam, the netherlands, evaluating a concept mapping training programme by 10 and 13 year-old students / habók 471 august 25u29. 2009. in: g. rijlaarsdam (eds.): fostering communities of learners. 13th biennial conference for research on learning and instruction. programme (p. 85). amsterdam: graduate school of teachinng and learning, uva. artelt, c., baumert, j., julius-mcelvany, n., & peschar, j. (2003). learners for life, student approaches to learning: results from pisa 2000. oecd. ausubel, d. p. (1968). educational psychology: a cognitive view. holt, rinehart and winston, inc. barroso, c., & crespillo, r. (2008). concept maps: tools for understanding complex problems. in a. j. canas, p. reiska, m. k. ahlberg, & j. d. novak (eds.) concept mapping –connecting educators, proceedings of the 3rd international conference on concept mapping (pp. 129-132). vol 3. tallinn: ihmc, tallinn university, university of helsinki. berionni, a., & baldoni, m. o. (2004). the words of science: the construction of science knowledge using concept maps in italian primary school. in a. j. canas, j. d. novak, & f. m. gonzález (eds.) theory, methodology, technology, proceedings of the first international conference on concept mapping. pamplona: spain. http://cmc.ihmc.us/papers/cmc2004-067.pdf, 2008. 12. 29. canas, a., & novak, j. (2008). facilitating the adoption of concept mapping using cmaptools to enhence meaningful learning. in okada, a., buckingham shum, s. j., & sherborne, t. (eds.) knowledge cartography, software tools and mapping techniques (pp. 25-46). london: springer. csapó, b. (2007). research into learning to learn through the assessment of quality and organization of learning outcomes. the curriculum journal, 18(2), 195-210. gurlitt, j., renkl, a., faulhaber, l., & fischer, f. (2007). interactions of expertise and prior-knowledge activation with low-coherent and high-coherent concept mapping tasks. in d. s. mcnamara & g. trafton (eds.), proceedings of the 29th annual conference of the cognitive science society (pp. 10551060). austin, tx: cognitive science society. halimi, s. (2006). the concept map as a cognitive tool for specialized information recall. in a. j. canas, & j. d. novak (eds.) concept maps: theory, methodology, technology, proceedings of the second international conference on concept mapping. san josé: costa rica. http://cmc.ihmc.us/cmc2006papers/cmc2006-p81.pdf, 2008. 12. 29. hardy, i., & stadehofer, b. (2006). concept maps wirkungsvoll als strukturierungshilfen einsetzen: welche rolle spielt die selbstkonstruktion? zeitschrift für pädagogische psychologie, 3, 175-187. himangshu, s., iuli, r. j., & venn, m. (2008). impact concept mapping has on pre-service teachers understanding of sciene content knowledge and development of their science pedagogy. in a. j. canas, p. reiska, m. k. ahlberg, & j. d. novak (eds.) concept mapping–connecting educators, proceedings of the 3rd international conference on concept mapping vol. 2. part two, (pp. 438444). immonen-orpama, p., & ahlbert, m. (2010): collaborative learning by developing (lbd) using concept maps and vee diagrams. in p. l. torres, & r. c. v. marriott (eds.) handbook of research on collaborative learning using concept mapping (pp. 215-237). hershey, new york: information science reference. mancinelli, c., gentili, m., prioiri, g., & valitutti, g. (2004). concept maps in kindergarten. in a. j. canas, j. d. novak, & f. m. gonzáles (eds.) theory, methodology, technology, proceedings of the first international conference on concept mapping. pampolona: spain. http://cmc.ihmc.us/papers/cmc2004-195.pdf, 2008. 12. 29. molnár, gy. (2009): inductive reasoning in the first grade: a training program and its longitudinal effect. paper presented at the 13th european conference for the research on learning and instruction. amsterdam, the netherlands, august 25–29, 2009. in g. rijlaarsdam (eds.): fostering communities of learners. 13th biennial conference for research on learning and instruction. programme (p. 166). amsterdam: graduate school of teachinng and learning, uva. international electronic journal of elementary education, 2012, vol.4, issue 3, 459-472 472 novak, d. j., & gowin, d. g. (1984). learning how to learn. new york: cambridge university. novak, j. d. (1990). concept maps and vee diagrams: two metacognitive tools to facilitate meaningful learning. instructional science. 19(1), 29-52. novak, j. d. (1998). learning, creating and using knowledge. new jersey: lawrence erlbaum associates. poveda, m. r. f., & oneca m. j. t. (2006). a study of links in concept maps constructed by primary school learners. in a. j. canas, & j. d. novak (eds.) theory, methodology, technology, proceedings of the second international conference on concept mapping. san josé: costa rica. http://cmc.ihmc.us/cmc2006papers/cmc2006-p27.pdf reader, w., & hammond, n. (1994). computer-based tools to support learning from hypertext: concept mapping tools and beyond. computers education, 22(1-2), 99-106. vakilifard, a., & armand, f. (2006). the effects of ‘concept mapping’ on second language learners’ comprehension of informative text. in a. j. canas, & j. d. novak (eds.) theory, methodology, technology, proceedings of the second international conference on concept mapping. san josé: costa rica. http://cmc.ihmc.us/cmc2006papers/cmc2006-p79.pdf , 2008. 12. 29. waeytens, k., lens, w., & vandenberghe, r. (2002), ’learning to learn’: teachers’ conceptions of their supporting role. learning and instruction, 12(4), 305–322. zantig, a., verlop, n., & vermut, j. d. (2003). using interviews and concept maps to access mentor teachers’ practical knowledge. higher education, 46, 195-214. microsoft word iejee_5_6_zertuche international electronic journal of elementary education, 2012, 5(1), 79-92. issn:1307-9298 copyright © iejee www.iejee.com how do openers contribute to student learning? ∗∗∗∗ amber zertuche ∗∗∗∗∗∗∗∗ university of california, berkeley, united states of america libby gerard university of california, berkeley, united states of america marcia c. linn university of california, berkeley, united states of america received: 20 august 2012 / revised: 24 october 2012 / accepted: 24 october 2012 abstract openers, or brief activities that initiate a class, routinely take up classroom time each day yet little is known about how to design these activities so they contribute to student learning. this study uses technology-enhanced learning environments to explore new opportunities to transform openers from potentially busy work to knowledge generating activities. this study compares the impact of teacher-designed openers, opener designs based on recent research emphasizing knowledge integration, and no opener for an 8th grade technology-enhanced inquiry science investigation. results suggest that students who participate in a researcher-designed opener are more likely to revisit and refine their work, and to make significant learning gains, than students who do not participate in an opener. students make the greatest gains when they revisit key evidence in the technology-enhanced curriculum unit prior to revision. engaging students in processes that promote knowledge integration during the opener motivate students to revise their ideas. the results suggest design principles for openers in technology-enhanced instruction. keywords: technology, science education, teaching practices, classroom assessment, formative assessment ∗ this article is based on the first author’s unpublished ma thesis entitled “how teacher-led openers contribute to student learning in a web-based inquiry science environment.” this material is based upon work supported, in part, by the national science foundation under grant drl-0733299 [logging opportunities in online programs for science (loops)]. any opinions, findings, and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the national science foundation. ∗∗ amber zertuche, graduate school of education, tolman hall, university of california, berkeley, u.s.a. phone: (415) 656-5840. e-mail: anzertuche@berkeley.edu international electronic journal of elementary education vol 5, issue 1, 79-92, 2012 80 introduction openers, or brief activities that initiate a class, routinely take up classroom time each day yet little is known about how to design these activities so they contribute to student learning. teachers often give mini-lectures to remind students of what they studied in the previous class. or, teachers may assign a short writing assignment. technology-enhanced learning environments provide teachers with new tools to transform openers from busy work to valuable learning activities. openers can engage students in knowledge integration activities such as making predictions, critiquing a claim, assessing peer essays, or reflecting on progress—all activities that have been shown to improve student learning (chiu and linn, 2011; linn and eylon, 2011; white and frederiksen, 1998). this study compares the impact of teacherand researcher-designed openers to no opener in an 8th grade technologyenhanced inquiry science investigation. questions include: how effective are openers? how does the design of the opener contribute to student learning? and, what are effective postopener revision processes? teacher-led openers can play an essential role in the success of technology-enhanced inquiry science instruction. several studies show that the quality of teacher implementation of technology-enhanced instruction predicts the impact of the technology on learning (tamin, bernard, borokhovski, abrami, and schmid, 2011). in a second-order meta-analysis conducted on technology-enhanced instruction over the past 40 years, the authors conclude that the teachers’ ability to monitor student understanding and help students sort out their ideas is essential to successful technology implementation (williams, linn, m, ammon, and gearhart, 2004). this meta-analysis found that as teachers asked more knowledge oriented questions during the course of a technology-enhanced science investigation as opposed to procedural questions, learning gains improved significantly. further, to realize the potential of technology-enhanced instruction, teachers need to help students critically analyze visualizations relative to the conceptual learning goal as students often overestimate their understanding of dynamic computer visualizations, particularly in chemistry (chiu and linn, 2012). technology-enhanced learning environments provide teachers with unique tools to structure effective openers. the computer stores and organizes a record of each student’s work including their multiple revisions, and provides ways to make examples of the student’s work public to the whole class. this means teachers can purposefully select examples of the student’s work for class discussion. effective selection of examples could prompt students (and their teachers) to monitor understanding, sort out ideas about difficult concepts, and revisit and refine their reasoning (izsak, 2012; linn and eylon, 2011). using student work to design openers student work is collected in technology enhanced learning environments and can be used to design openers. black and wiliam (1998) reviewed hundreds of studies and came to the conclusion that the use of student work for formative assessment and refinement of instruction is one of the most powerful ways to increase students’ learning gains, particularly among low achieving students. similarly, shute (2008) reviewed work on formative feedback, defined as information communicated to the learner that is intended to modify his or her thinking or behavior to improve learning, and found that effective feedback is nonevaluative, supportive, timely, and specific. gerard, spitulnik, and linn (2011) showed that when teachers use student work to refine their teaching, students’ learning benefits. these studies distinguish between activities providing verification, where students choose whether an answer is correct or not, and elaboration, where teachers provide cues to guide learners toward a correct answer. gerard et al found that use of student work to design instruction is how do openers contribute to student learning? / zertuche, gerard & linn 81 most effective when the resulting instruction provides advice on what to do to improve performance, rather than on comparisons to other students or on accuracy of responses. one example of a successful use of student work is for selfand peer-assessment. research suggests that students can benefit from self and peer assessment activities when they recognize the desired learning goal, have adequate evidence to determine where the work stands relative to the learning goal, and have an understanding of a way to close the gap between the two (black and wiliam, 1998). under these conditions, students can distinguish between their own ideas and the goal of instruction and strengthen their understanding. taken together, these results suggest that effective openers should elicit student work and provide hints or guidance. what are the challenges of formative assessment? black and wiliam’s review of formative assessment studies point out that, although formative assessment has proven to be useful to student learning, there are concerns about how to enact successful formative assessment (1998). one issue is that in order to implement formative assessment in a useful way, teachers must have easy access to evidence of students’ ideas and efficient routines to elicit students’ ideas during class so they can help students to distinguish among these ideas. this calls for assessment or discussion questions that prompt students to make their reasoning explicit and provide multiple entry points for students with various levels of understanding. students must also be actively involved in the feedback they are receiving in order for it to have an effect on their learning. this requires significant changes in a traditional secondary science classroom. traditional science classroom routines often center on teacher-directed lectures and demonstrations, leaving little space for students to reflect on their understanding and sort out the variety of ideas they hold about the topic gathered from everyday experiences, peers, and school curricula. ruiz-primo and furtak (2007), for instance found that teachers’ formative assessment routines focused on procedural elements of inquiry learning, learning such as checking the students’ knowledge of the correct procedure and asking them to apply a procedure to a new situation, rather than knowledge generation. how to design an effective opener in chemistry? this study investigates the design of knowledge integration openers. typical classroom openers usually reflect the absorption model of instruction and use the question, response, evaluation (qre) approach. in this model, the teacher asks a question with one answer in mind and prompts until a student gives the answer or fills in the response if none is elicited. this theory of learning assumes that the task of the learner is to acquire the body of connections that an expert analysis of the subject matter reveals (greeno, collins, and resnick, 1996). in contrast, the knowledge integration perspective on learning resonates with the black and wiliam’s findings (2011) and guides the design of the openers, curriculum and assessments in this study. the knowledge integration perspective draws on findings from learning sciences research. specifically, learners hold multiple conflicting ideas about scientific phenomenon as has been documented in numerous studies of student intuitions about science topics (disessa, 2000). in addition, learners, often in collaboration with others, can deliberately sort out, link, and critique their ideas when making sense of new scientific phenomena and benefit from encouragement to engage in this process (linn, lee, tinker, husic, and chiu, 2006; linn and hsi, 2000; novak and gowin, 1984; slotta, chi, and joram, 1995). this means that providing opportunities for students to compare alternative ideas to their own, develop criteria for sorting-out and distinguishing among ideas, and reflect on their ideas can help them form coherent hypotheses or explanations. international electronic journal of elementary education vol 5, issue 1, 79-92, 2012 82 the knowledge integration openers in this study were designed to help students make connections among their ideas about chemical reactions. in chemistry, one of the most difficult things for students to learn is how chemicals react. students often have difficulties translating between symbolic representations, molecular representations, and observable phenomena (ardac and akaygun, 2004). particularly, students struggle to make sense of chemical phenomena at the molecular level (johnstone, 1993; krajcik, 1991). for example, many students think of chemical reactions as an instantaneous process without bond breaking and formation, while others think all the molecules break into atoms. in addition, prior studies demonstrate that students often isolate molecular visualizations rather than linking them to existing knowledge or everyday experiences and have difficulty interpreting stand-alone dynamic visualizations (tversky, morrison, and betrancourt, 2002; zhang and linn, 2011). this study addresses this gap in learning chemical reactions with openers that ask students to reflect upon and critique peers’ visual molecular representations of hydrogen and oxygen combustion. methods research design this study investigates how a researcher-designed opener, teacher-designed opener, and a control condition (no opener) contribute to students’ revision of work and understanding of chemical reactions. three central questions guide this research: 1. do openers contribute to student understanding of chemical reactions? 2. how does the opener design influence students’ learning outcomes? 3. what are effective post-opener revision processes? curriculum and assessments the web-based inquiry science environment (wise) is an open-source on-line learning environment that includes multiple standards-aligned science inquiry curriculum units. to engage students in knowledge integration processes, wise projects guide students in collaborative activities with visualizations of scientific phenomena that are difficult to observe, such as molecular views of chemical reactions (figure 1). students investigate hypotheses, design solutions to problems, critique scientific claims, and build scientific models, scaffolded by guidance based on knowledge integration principles. students in this study worked on the wise hydrogen fuel cell car unit. this is a one week unit designed to teach students about chemical reactions, alternative fuels, and energy (http://wise.berkeley.edu/webapp/vle/preview.html?projectid=911). the project begins by asking students if they would rather buy a hydrogen or gasoline powered car. it also elicits their ideas about energy and adds ideas about conservation of energy. then, gasoline combustion in cars is explored including the relationship between carbon dioxide, a product of gasoline combustion, and temperature changes over the last 200 years. students then create their own energy story about cars. this story includes where the energy came from to power the car and any chemical reactions that are involved in their story. hydrogen combustion is then explored using a dynamic visualization of hydrogen combustion (figure 1). then students are taught about the difference between exothermic and endothermic reactions and finally, students investigate a visualization of a hydrogen fuel cell to learn how this technology works. students are then asked which kind of technology they would prefer when buying a car. how do openers contribute to student learning? / zertuche, gerard & linn 83 assessments are embedded throughout the wise projects to help students and teachers monitor student understanding and progress as students interact with visualizations (figure 2). the embedded assessments ask students to make predictions about the visualizations, sort out evidence, and link ideas together to explain their thinking. students can also get hints to help them complete the tasks. figure 1. one of the visualizations in wise’s hydrogen fuel cell cars project models the hydrogen combustion reaction. students can run, stop, reset, and ‘spark’ to start the reaction and explore the nature of chemical reactions. in the hydrogen fuel cell cars unit, an embedded assessment immediately follows the visualization of hydrogen combustion asking students to draw four frames of hydrogen combustion (figure 2). this is meant to help students make sense of the dynamic visualization, recognizing features such as conservation of mass, bonds breaking, and the progression of the reaction. the openers in this study focused on student work from this embedded assessment since in previous years this particular task was particularly challenging to students and yet still, understanding how to draw basic hydrogen combustion is critical to student understanding of chemical reactions. figure 2. the wise project screen has an inquiry map on the left, a navigation bar on top, and an embedded assessment in the center of the screen. international electronic journal of elementary education vol 5, issue 1, 79-92, 2012 84 teacher assessment tools wise provides teachers convenient access to a record of student work as students progress through a wise unit (figure 3). the wise grading tools allow teachers to view embedded assessment data by step, and the flag tool allows teachers to select key student work examples for display. students can view flagged work at any time by clicking on a tab at the top of their wise screen. this allows students to actively see their peers' work and be a part of the feedback process to improve their understanding. students can revise their work based on teachers’ and peers’ comments. all revisions are logged in the grading tool so the teacher can measure the impact of their opener, or comments, by viewing the change in students’ ideas from their original to revised work. figure 3. wise assessment tool with revision number on the left, including time stamp, and checkbox to flag work on the right. study participants two 8th grade teachers and their 236 students from 1 public school participated in this study. the school has medium diversity (22% receiving free lunch, 5% ell, and 27% nonwhite). both teachers had over five years experience teaching with wise and frequently used openers in their regular and wise instruction. students were randomly assigned by class period to one of the three conditions. there were 78 students in the teacher-designed opener, 128 students in the researcher-designed opener, and 30 students in the control group condition. the uneven sample sizes were due to the uneven number of class periods that each teacher taught physical science that year. opener design the researcher-designed opener engaged students in knowledge integration processes, as shown in table 1. activities included a small group discussion, voting, a whole group discussion, and then a closing summary by the teacher. the examples were selected to illustrate the range of conceptual errors in student representations of the chemical reaction. these related to (a) conservation of mass, (b) breaking of bonds, and (c) progression of the reaction. revision number time stamp flag work checkbox how do openers contribute to student learning? / zertuche, gerard & linn 85 the teacher-designed opener alternatively focused on picking out what is good and bad about the student work and a lecture to try to re-teach chemical reactions from a different angle. examples of student work were selected to illustrate responses that could be described as, “the good, the bad and the ugly”. table 1. description of openers used in teacher 1 and teacher 2’s classroom using the knowledge integration framework. teacher-designed researcher-designed teacher 1 elicit ideas: question on the board about how a balanced equation obeys the law of conservation of mass. students fill out a chart on how much mass (amu) exists before and after reaction. add ideas: students use physical model of molecules to break bonds and put back together. teacher shows multiple types of reactions. distinguish ideas: teacher shows four examples of student work and asks students if the example is good or bad. integrate ideas: ____ (~12 min) students open wise project to view four examples of student work in wise project by clicking on the “flagged work” button. each example has one unique link missing (figure 4) elicit ideas: teacher asks students to write down which drawing best represents the visualization of hydrogen combustion and use evidence to explain why. students vote and teacher tallies votes. add ideas: students discuss their choices in groups of 4 and revisit evidence in the visualization. distinguish ideas: students reconsider their initial choice in light of their discussion with peers and revisit the evidence. make a new vote. integrate ideas: teacher tallies new votes and asks students to justify their choice. teacher synthesizes criteria used to evaluate drawings, and instructs students to revise their own drawing. (~20 minutes) teacher 2 elicit ideas: ____ add ideas: used embedded assessment problem and a tree to house analogy to take students through the chemical reaction steps. tree must break into parts, then recombine parts and build a house. uses physical models of hydrogen and oxygen molecules to show students progression necessary to make water. distinguish ideas: ____ integrate ideas: ____ (~8 min) figure 4. examples of student work shown during the researcher-designed opener. link missing: conservation of mass link missing: bonds breaking international electronic journal of elementary education vol 5, issue 1, 79-92, 2012 86 data collection and analysis data sources include: the original student work on the embedded assessment, the revised student work on the embedded assessment after the opener, wise log files illustrating student navigation through wise immediately after the opener, pre and post tests administered immediately before and a 1-14 days after the wise project, classroom video of teacher implementation of the researcher-designed and teacher-designed openers, and teacher interviews. knowledge integration rubrics were used to score the embedded and pre/post assessments. the embedded assessment rubric is shown in table 2. table 2. ki rubric for embedded assessment where students make step-by-step drawings of hydrogen combusting with oxygen ki level score characteristics example invalid 1 blank or i don’t know no links 2 no normative ideas conveyed but work has been done simple, 1 link 3 represents conservation of mass or bonds breaking or progression* advanced, 2 links 4 conservation of mass and progression* or conservation of mass and bonds breaking or progression* and bonds breaking complex, 3 links 5 conservation of mass and bonds breaking and progression* of reaction * progression must be apparent on all 3 transitions pre and post questions were also scored using a ki rubric. the main ideas that students should understand in the preand posttest are that there is a reaction process that occurs and that h2 and cl2 should start breaking bonds before forming new bonds. results we examine the effects of openers on student learning outcomes, and then explicate the contributing factors including opener design and students’ learning practices. we focus on embedded assessments and pretest-posttest performance. we consider the actual implementation of the conditions and student performance as reflected in the log files. how do openers contribute to student learning? / zertuche, gerard & linn 87 embedded assessments to investigate the impact of the opener versus no opener on student understanding we compared students’ original (before opener) and revised (after opener) responses to the embedded assessments. responses were scored using the knowledge integration rubric. both teacher and researcher-designed openers (n=105 pairs) were compared to the no opener condition (n=30 pairs). pairs who did not complete the embedded assessment before the opener were excluded from the analysis (n=26 pairs). although the aim was to facilitate the opener after 75% of students completed the embedded assessment, this number was based on an automated progress screen that only monitored whether or not students submitted work at least once for this assessment. once the researchers looked at the student’s work, it was obvious that, although work was submitted, many students did not finish their drawings and therefore these were not included in the data analysis. responses were scored using the ki rubric. time stamps from the wise log files were used to identify students’ final work immediately before the opener and their revised work on the day of the opener. the analysis suggests that students who had an opener, either teacheror researcherdesigned, made substantially greater learning gains than students who did not have an opener. as shown in table 4, students who had an opener (m = .29, sd = .78) doubled the mean gain score of those students who did not have an opener (m = .13, sd = .51) on the embedded assessment. there was no significant difference between conditions in students’ pre-opener scores. students who had an opener (m = .68, sd = .48) were significantly more likely to revisit and revise their work than students who did not have an opener (m = .33, sd = .48), t(131) = -3.5, p < 0.001. in the opener condition, revision was twice as likely as in the no opener condition. table 3. pre to post gain scores with and without opener. n pairs who revised their work mean gain (sd) rate of revision (n pairs who revised/total pairs) opener 105 .29(.78) 66% no opener 30 .13(.51) 33% the openers were particularly effective for students who demonstrated at least a basic understanding on the embedded assessment prior to the opener (table 7). having an opener had a significant effect on students who began with at least a partial understanding, or level 3 on the knowledge integration scoring rubric (m = .71, sd = .69), t(26) = -2.07, p = .05. students who began with partial understanding developed their ideas into a basic understanding (level 4) after the opener. in contrast, students who began with partial understanding and had no opener continued to demonstrate only partial understanding after revising their work (m = .18, sd = .60). the large effect of the opener on level 3 students is partially due to the examples of student work selected for critique in the researcher and teacher-designed openers. the selected examples that were shown during the opener illustrated characteristics of level 3 understanding, making them most accessible to this population of students. since student work was chosen this way, this opener was unwittingly designed to increase the learning gains of students with an already basic understanding of chemical reactions. students with non-normative ideas, or level 2 understanding, made modest gains with or without an opener, as shown in figure 5. level 2 students may need examples aligned with their own ideas to improve to partial or high level understanding. for instance, showing international electronic journal of elementary education vol 5, issue 1, 79-92, 2012 88 student work with no conservation of mass or bond breaking would lead to a discussion where students point out that both are missing from the drawing. alternatively, showing a common level 2 student work may show this population of students that their attempt is acknowledged and that they have direct feedback for improving their drawing. pre opener: no links present post opener: no links present figure 5. examples of students’ work with a level 2 understanding before and after an opener. table 6. pre to post opener gain scores distributed by pre-opener scores preopener score n pairs who revised their work mean gain with opener(sd) n pairs who revised mean gain without opener (sd) 2 10 .4(.70) 5 .4(.89) 3 17 .71(.69) 11 .18(.60)* 4 42 .21(.42) 10 0(0) *p=.05 pre-test to post-test effects. the preand posttests were administered before students began the week long project and after they finished the project. in spite of the gains found for the specific item addressed by the opener, there was no difference on the pre/post tests between students who had an opener (m = 1.74, sd = 2.50) and those who did not (m = 1.55, sd = 2.16), t(255) = -.511, p = .61. this is not surprising since the opener treatment was only 10-20 minutes out of the 5-7 hour long project. perhaps several openers over the project length would have been able to affect the pre test to post test results. also, one of the teachers in the study did not give the post test until 2 weeks after the project was completed. although the schedule was controlled, the teacher ultimately has the authority on when to give the post test. this delay may have masked any immediate effects of the opener on the post test. because of these time related anomalies, student performance on the embedded assessment is a better representation of the immediate effects of the openers. how does the opener design influence students’ learning outcomes? the opener designs, described in table 2, differed primarily in their support for distinguishing and integrating ideas. in both openers, students were presented with evidence regarding a chemical reaction and prompted to think about conceptual features of a chemical reaction. the researcher-designed opener had additional components to support integration of ideas. after considering the new evidence, students were guided to reconsider how do openers contribute to student learning? / zertuche, gerard & linn 89 their initial views in light of the evidence presented, and refine their criteria for conceptual features of a chemical reaction. these additional components made the researcherdesigned opener longer (approximately 20 min) than the teacher-designed opener (approximately 10 min). since the length of both openers was difficult to predict ahead of time, the students in the teacher-designed opener did not have an additional task to account for the 10 minute different time on task. students who participated in the researcher-designed opener were significantly more likely to revise their work on the embedded assessment (m = .75, sd = .44) compared to students who participated in the teacher-designed opener (m = .56, sd = .50), t(101) = -1.98, p < 0.05 (table 8). there was no significant difference in learning gains between the two conditions (table 9). the researcher-designed opener was not completely implemented as planned. the voting process in the researcher-designed opener was a new activity for students and did not work as anticipated. in some classes, students directly copied the examples of student work that received the most votes even though none of the student work examples were in fact “correct”. although teachers reminded students that none of the student work examples were correct, this could have been emphasized and the teacher could have checked that students truly did understand that examples were not to be directly copied (have a student repeat it back, whole class response etc). table 7. student ki score (maximum score 5) of embedded assessment before and after the opener treatment n revised total teams avg rate of revision pre opener post opener ki score gain teacher designed opener 22 39 .56(50) 3.88 4.26 0.38 researcher designed opener 48 64 .75(.44) 3.59 3.95 0.36 table 8. student pair ki score (out of 5) for pre and post test n pre test post test ki score gain teacher designed opener 78 2.69 3.54 0.85 researcher designed opener 121 2.58 3.30 0.72 log files. to analyze post-opener learning practices we looked at log files. two students from each period, one with the highest gain and one with the lowest gain, were selected for log file analysis in order to get an equal distribution of student learning practices from each condition. students’ learning practices may explain why some students made greater gains on the embedded assessment post opener than others. the wise log files show that the biggest contribution to learning gains was the time students spent revising their initial work, and revisiting relevant evidence in the wise unit. the students who made the greatest improvement revisited an evidence page immediately after the opener or spent more time revising their original work than other students (table 10). an evidence page could be the dynamic visualization of hydrogen combustion or notes that students wrote about what a international electronic journal of elementary education vol 5, issue 1, 79-92, 2012 90 good drawing of hydrogen combustion should include. this suggests the value of providing students with access to evidence to sort out their ideas during an opener. table 10. comparison of students who had a gain in their ki score by at least 1 and those who had no gain. after the opener, students with a gain either spent more time on the embedded assessment or revisited an evidence page after the opener. percentage results are aggregated for each condition. learning practice gain (n=6) no gain (n=6) revisited evidence page 67% 33% revisited embedded assessment for more than 1 minute 100% 50% both 67% 33% discussion this study illustrates how openers can improve student learning compared to not using openers. both the teacher-designed and researcher-designed openers used in this study encouraged students to review their ideas. the researcher-designed openers encouraged students to distinguish ideas and reflect, and resulted in a greater propensity to revise answers than did the teacher-designed openers. these results for openers resonate with research showing the benefit of giving students feedback based on their responses to assignments (black and wiliam, 1998; shute, 2008). openers supported students to reflect upon their initial ideas, reconsider evidence, and refine their views. openers were particularly useful for students who began with a partial understanding of a concept. when students benefitted from openers, they took advantage of the evidence presented in the opener, revisited a dynamic visualization or another evidence source, and reflected on the new information. the findings from this study suggest the following design principles for openers: 1) openers can reinforce normative conceptual ideas by drawing attention to the distinction between student ideas and normative views. getting evidence from peers, visualization, teacher, or other classroom resource can help students understand complex ideas. this evidence helps students close the gap between what they know now and the normative view of the phenomena studied. 2) openers help students when they occur soon after a topic is introduced and direct attention to specific ways to improve their ideas. 3) openers succeed when they are non-evaluative and support students to explore evidence or views of their peers. for example, teachers can support students by giving them the opportunity to distinguish among the ideas held by the group of students in the class. giving students a chance to appreciate conflicting views held by classmates and use evidence to sort them out helps students integrate their ideas. 4) openers should be short. the openers in this study were much longer and had more teacher involvement for most openers. it may work better to have students review examples of student work as homework, then discuss with their partner at the very start of class. this would free the teacher up to take attendance and give more time for the peer and class discussion. • • • how do openers contribute to student learning? / zertuche, gerard & linn 91 amber zertuche. received masters degree and single subject credential in science and math education from university of california, berkeley in 2012. she is a first year physics teacher at burton high school in san francisco unified school district since 2012. before becoming an educator, she helped engineer the james webb space telescope’s primary mirrors. libby gerard. postdoctoral fellow with the technology-enhanced learning in science center in the graduate school of education at the university of california, berkeley. she completed her phd from mills college in 2008. she studies the relationship between professional development, technologyenhanced instruction and student knowledge integration in inquiry science. libby is particularly interested in how to use technology-enhanced assessment data to support teacher and principal decision-making to improve inquiry instruction and student learning. libby leads the tels professional development activities including summer institutes, sustained inquiry based communities, and teacher mentoring. marcia linn. professor, graduate school of education, university of california, berkeley. mbr: national academy of education. fellow: american association for the advancement of science, the american psychological association. elected: chair of the aaas education section, president of the international society of the learning sciences. awards: national association for research in science teaching award for lifelong distinguished contributions to science education, council of scientific society presidents first award for excellence in educational research. references ardac, d. & akaygun, s. (2004). effectiveness of multimedia-based instruction that emphasizes representations on students’ understanding of chemical change. journal of research in scienceteaching, 41(4), 317-337. berland, l.k. & reiser, b.j. (2011). classroom communities’ adaptations of the practice of scientific argumentation. science education, 95(2), 191–216). black, p. & wiliam, d. (1998). inside the black box: raising standards through classroom assessment. phi delta kappan, 80(2), 139-148. chiu, j. & linn, m. (2012). the role of self-monitoring in learning chemistry with dynamic visualizations. in metacognition in science education (pp. 133–163). dordrecht: springer. disessa, a. (2000). changing minds: computers, learning and literacy. cambridge, ma: mit press. eylon, b. & linn, m. (1988). learning and instruction: an examination of four research perspectives in science education. review of educational research, 58(3), 251–301. gerard, l.f., spitulnik, m., & linn, m.c. (2011). teacher use of evidence to customize inquiry science instruction. journal of research in science teaching, 47(9), 1037-1063. greeno, j.g., collins, a.m., & resnick, l.b. (1996). cognition and learning. in d.c. berliner & r.c. calfee (eds.), handbook of educational psychology (pp.15-46). new york: macmillan. johnstone, a.h. (1993). the development of chemistry teaching: a changing response to changing demand. journal of chemical education, 70(9), 701-704. krajcik, j. (1991). developing students' understandings of chemical concepts. in s. glynn, r. yeany, & b. britton (eds.), the psychology of learning science (pp. 117-147). hillsdale, nj: erlbaum. linn, m.c., clark, d., & slotta, j.d. (2003). wise design for knowledge integration. sci ed, 87, 517-538 linn, m.c. & eylon, b.s. (2011). science learning and instruction: taking advantage of technology to promote knowledge integration. new york: routledge. linn, m. c. & hsi, s. (2000). computers, teachers, peers. hillsdale, nj: erlbaum. linn, m.c., lee, h.s., tinker, r., husic, f., & chiu, j.l. (2006). teaching and assessing knowledge integration in science. science, 313, 1049-1050. international electronic journal of elementary education vol 5, issue 1, 79-92, 2012 92 novak, j., & gowin, d. (1984). learning how to learn. new york: cambridge books. slotta, j. d., chi, m. t. h., & joram, e. (1995). assessing the ontological nature of conceptual physics: a contrast of experts and novices. cognition and instruction, 13(3), 373–400. tversky, b., morrison, j. b., & betrancourt, m. (2002). animation: can it facilitate? international journal of human-computer studies, 57, 247-262. tamin, r., bernard, r., borokhovski, e., abrami, p., & schmid, r. (2011). what forty years of research says about the impact of technology on learning: a second-order meta-analysis and validation study. review of educational research, 81(1), 4-28. white, b.y. & frederiksen, j.r. (1998). inquiry, modeling, and metacognition: making science accessible to all students. cognition and instruction, 16(1), 3-118. williams, m., linn, m. c., ammon, p., & gearhart, m. (2004). learning to teach inquiry science in a technology-based environment: a case study. journal of science education and technology, 13(2), 189-206. zhang, z. & linn, m. c. (2011). can generating representations enhance learning with dynamic visualizations? journal of research in science teaching, 48(10), 1177-1198. microsoft word iejee_4_3_kolovelonis_goudas_dermitzaki international electronic journal of elementary education, 2012, 4(3), 507-517. issn:1307-9298 copyright © iejee www.iejee.com students’ performance calibration in a basketball dibbling task in elementary physical education athanasios kolovelonis ∗∗∗∗ university of thessaly, greece marios goudas university of thessaly, greece irini dermitzaki university of thessaly, greece received: february, 2012 / revised: march, 2012 / accepted: april, 2012 abstract the aim of this study was to examine students’ performance calibration in physical education. one hundred fifth and sixth grade students provided estimations regarding their performance in a dribbling test after practicing dribbling for 16 minutes under different self-regulatory conditions (i.e., receiving feedback, setting goals, self-recording). two calibration indices, calibration bias and calibration accuracy, were calculated. the results showed that students who practiced dribbling under different self-regulatory conditions (i.e., receiving feedback, setting goals) did not differ in calibration bias and accuracy. regardless of the group, students were overconfident. moreover, sixth grade students were more accurate compared to fifth grade students. these results were discussed with reference to the development of performance calibration and self-regulated learning in physical education. keywords: calibration, physical education, grade/gender differences, self-regulation, basketball dribble introduction the development of self-regulated learners is a major educational goal (boekaerts, 1997). self-regulated learning is an active, self-directive process whereby students monitor, regulate, and control their cognition, motivation, affect, behavior, and environment to achieve their goals (efklides, niemivirta & yamauchi, 2002). self-regulated students see ∗ athanasios kolovelonis, department of physical education and sport science, university of thessaly, 42 100 karies, trikala, greece. phone: +30 24310 47045. e-mail: sakisanta@hotmail.com international electronic journal of elementary education, 2012, vol.4, issue 3, 507-517 508 themselves as agents of their own behavior, believe that learning is a proactive process, are self-motivated and use strategies to achieve their academic goals. ιt has also been supported that the development of self-regulated learning is associated with positive learning outcomes in both academic and physical education settings (kitsantas, steen & huie, 2009; zimmerman, 2002). in fact, recent research in elementary physical education has shown that students can effectively use self-regulatory processes, such as goal setting, self-recording, and self-talk, to enhance their performance in motor and sport skills (kolovelonis, goudas & dermitzaki, 2010, 2011a, 2011b, in press; kolovelonis, goudas, hassandra & dermitzaki, 2012). to become self-regulated learners, students need to monitor accurately their ongoing cognitive states and processes, and to use the information obtained from this monitoring to regulate those processes (hacker, bol & keener, 2008). thus, a factor associated with selfregulated learning is calibration which is the degree to which a person’s perception of performance corresponds with his or her actual performance (keren, 1991). calibration is considered one component in the process of developing self-regulatory competency and is a metacognitive skill for monitoring one’s performance (pieschl, 2009; zimmerman, 2008). learners make judgments about what knowledge or skill they have learned, and those judgments are compared to an objectively determined measure of that knowledge or skill (winne, 2004). the more closely students’ predicted performance matches their actual performance, the better calibrated they are (hacker, bol & bahbahani, 2008). the underlying psychological process reflected in calibration entails a person’s monitoring of what he or she knows about a specified topic or skill and judging the extent of that knowledge in comparison to some criterion task, such as an examination or a test (hacker et al., 2008). calibration is educationally important because of its implications regarding students’ motivation (schunk & pajares, 2009) and metacognitive control processes and self-regulation (efklides & misailidi, 2010). in particular, students who overestimate their capabilities may attempt challenging tasks and fail, which would decrease their subsequent motivation, whereas those who underestimate their capabilities may avoid challenging tasks limiting their potential development of necessary skills (schunk & pajares, 2004). moreover, overconfidence may decrease effort exertion when needed (efklides & misailidi, 2010), provide a false sense of the strategy’s effectiveness (hacker, 1998), whereas students who underestimate what they can do may be reluctant to try the task and thereby retard their skill acquisition. furthermore, more accurate monitoring has been shown to lead to improved self-regulation and thus to higher performance (thiede, anderson & therriault, 2003). calibration research has shown that students are often inaccurate in judgments of their capability on a task or test (chen, 2003; hacker & bol, 2004) with a tendency to overconfidence (keren, 1991). in general, underconfidence is associated with higher performance and overconfidence with lower performance (hacker et al., 2008). stone (2000) hypothesized that self-regulated learners are well calibrated. however, it seems that calibration accuracy is hard to learn or resistant to change. previous efforts to improve calibration accuracy have shown mixed results. some studies have found modest gains in participants’ ability to predict and postdict performance (hacker, bol, horgan & rakow, 2000; nietfeld & schraw, 2002) but some other have reported no significant change in calibration accuracy after practice or other interventions (bol & hacker, 2001; bol, hacker, o’shea, allen, 2005). zimmerman, moylan, hudesman, white and flugman (2008) found that an intervention designed to improve students’ self-reflection improved the accuracy of students’ self-monitoring of their problem-solving performance. performance calibration in physical education / kolovelonis, goudas & dermitzaki 509 in sport and physical education research using the calibration paradigm to judge metacognitive bias normally varies from calibration testing in the cognitive domain, in that participants are immediately aware of their result in physical tasks (fogarty & ross, 2007). in particular, sport activities differ from academic ones by the presence of readily observable performance feedback (i.e., knowledge of results). for example, a basketball player can see after the execution if he was successful in the shot (feltz & magyar, 2006). thus, in motor and spots skills where immediate feedback is provided in the form of success or failure calibration becomes an integral part of learning the task and environmental cues are always available to ensure the accuracy of calibration (horgan, 1992). furthermore, competence is not wholly dependent on knowledge but also depends on actual physical skill. calibration research in sport and physical education is limited. fogarty and ross (2007) asked participants to estimate how many tennis serves out of the 10 they could hit into the target area. results showed that players were well calibrated on the easier task, but overconfident on the more difficult task (i.e., smaller target area). in a similar study (fogarty & else, 2005) golfers completed a putting and a chipping task after first estimating how well they would perform on each of these tasks. it was found that golfers were well calibrated on easier tasks (putting) and overconfident on more difficult tasks (chipping and pitching). participants were also overconfident on the golf rules test, a result which was consistent with results in cognitive calibration research. mcgraw, mellers, and ritov (2004) found that most recreational basketball players were overconfident regarding their shooting performance, but those who were more overconfident experienced less enjoyment. in a recent study in physical education, kolovelonis, goudas, dermitzaki and kitsantas (in press) found that performance calibration did not differ between students who practiced dribbling receiving social feedback and setting process or performance goals and control group students. undoubtedly, students’ performance calibration in physical education is unexplored. therefore, considering the important implication of students’ performance calibration regarding their motivation and self-regulation (efklides & misailidi, 2010; schunk & pajares, 2009) further research is needed to examine the status of students’ performance calibration in elementary physical education. furthermore, no study to our knowledge has examined grade and gender differences in students’ performance calibration in physical education. it has been theorized that capability self-beliefs become more accurate and specific with age and cognitive maturity (schunk & miller, 2002) because students become more realistic about their capabilities, are better equipped to interpret multiple sources of information about competencies, and have a more differentiated view of their abilities (eccles, wigfield & schiefele, 1998). thus, in the present study differences between fifth and sixth grade students in performance calibration were examined. regarding gender differences, previous research in academic settings has shown contradictory results regarding the role of gender in self-efficacy judgments as well as in calibration (chen, 2003). some research has reported gender differences in calibration among fifth graders, but not among middle school or high school students (chen, 2003). furthermore, some research has reported gender differences in students’ beliefs regarding their capabilities (i.e., self-efficacy) favoring adolescent boys, some have reported differences favoring girls, and others has revealed no gender differences (schunk & meece, 2006). in view of these mixed findings, the role of gender in students’ performance calibration requires further examination, particularly in physical education where this kind of research is limited. the aim of this study was to explore fifth and sixth grade students’ calibration regarding their dribbling performance in physical education. moreover, grade and gender differences in international electronic journal of elementary education, 2012, vol.4, issue 3, 507-517 510 performance calibration were examined. we hypothesized that students would overestimate their dribbling performance, and sixth grade students would be better calibrated compared to fifth grade students. no specific hypothesis for gender differences was established due to previous mixed results. method participants participants were 100 students (40 boys and 60 girls) between 11 and 12 years of age, who attended two fifth grade (40 students) and three sixth grade (60 students) physical education classes from two elementary schools located in a medium-sized city in central greece. students participated in the study voluntarily. no student refused to participate. students had little previous experience in the basketball dribble and none of them participated in basketball clubs out of school. students were randomly assigned to five groups using the proportional stratified sampling method and practiced dribbling under different conditions (see procedure section). measures basketball dribble. students had to dribble among five cones that had 3.05 m distance between each other. the distance between the first cone and the starting line was also 3.05 m. the test lasted 30 seconds and each student’s score was the total number of cones that he or she dribbled successfully. students were asked not to touch the cones during dribbling, to change the dribbling hand in each cone and to collect the ball by themselves in the case of losing its control. high test-retest reliability (r : .95) has been reported for this test (barrow & mcgee, 1979). calibration. prior to the dribble post-test students were asked the question: “how many cones will you dribble in the post-test?” based on this estimation and students’ actual scores in dribbling post-test two calibration indices were computed, the bias and the accuracy score (hacker et al., 2008). calibration bias was computed as students’ estimated performance score minus the actual performance. calibration bias is an index of the direction of the calibration. positive bias indicates overestimation of performance and negative bias underestimation. the absolute values of the bias scores resulted in the accuracy index which reflects the magnitude of calibration error. values closer to zero indicate higher calibration accuracy. procedure permission to conduct the study was obtained from the greek ministry of education lifelong learning and religious affairs and the school principals. students participated in the study in groups of four, in the school gym, with the presence of a trained experimenter who was a physical education teacher blind to the aims of the study. students were told that the purpose of the study was the improvement of their dribbling skill. initially, students were informed about the procedure of the study which consisted of the dribbling pre-test, the dribbling instructions and modeling, the 16-minute practice phase, and the post-tests. after the initial guidelines, students were informed about the scoring system, performed a trial run and then they were pre-tested in dribbling. then, students were provided with oral dribbling instructions and observed the experimenter’s dribbling demonstration. next, all students practiced the dribble for 16 minutes following different self-regulatory conditions, which included either the practice with social feedback or the simple practice in the first 8 minutes and setting process or performance goals in the next 8 minutes. in particular, in the first 8 minutes group 1 and 2 students practiced dribbling receiving social performance calibration in physical education / kolovelonis, goudas & dermitzaki 511 feedback (affirmative responses, performance reminders, and reinforcement). in the next 8 minutes group 1 students set process goals (i.e., focus on dribbling low with fingers-wrist) and group 2 students set performance goals (i.e., improving 20% their pre-test scores in dribbling test). students of both groups self-recorded their performance. group 3 and 4 students practiced dribbling without receiving social feedback in the first 8 minutes, but they set the same process (group 3) or performance (group 4) goals and self-recording their performance in the next 8 minutes. control group students practiced the dribble without receiving social feedback, setting goals or self-recording. after the end of the practice students answered the calibration question and were post-tested in dribbling. statistical analyses group differences in calibration bias and accuracy were examined with separate analyses of variance. grade and gender differences on calibration bias and accuracy were examined with separate 2 (grade) x 2 (gender) analyses of variance. effect sizes of partial η2 and cohen’s d were also calculated (cohen, 1988). results means and standard deviations of students’ calibration bias and accuracy scores separate for each grade and gender are presented in table 1. table 1. means and standards deviations for calibration bias and accuracy scores fifth grade sixth grade total boys girls total boys girls m sd m sd m sd m sd m sd m sd calibration bias 5.30 7.13 5.85 7.65 4.75 6.71 2.97 5.75 2.22 4.77 3.33 6.21 calibration accuracy 6.70 5.79 7.85 5.45 5.55 6.03 4.73 4.38 4.25 3.01 4.98 4.94 note: positive calibration bias scores indicate overestimation of performance. calibration accuracy scores closer to zero indicate higher accuracy. the one-way anova showed a nonsignificant difference between groups in calibration bias, f(4, 95) = 0.73, p = .57, and accuracy, f(4, 95) = 0.99, p = .41. thus, students of all groups were pooled to examine grade and gender differences in calibration bias and accuracy in the total sample. in calibration bias, the 2 (grade) x 2 (gender) anova showed a nonsignificant main effect for grade, f(1, 95) = 3.55, p = .063, and gender, f(1, 95) = 0.00, p = .99, and a nonsignificant grade x gender interaction, f(1, 95) = 0.67, p = .42. in calibration accuracy, the 2 (grade) x 2 (gender) anova showed a significant main effect for grade, f(1, 95) = 4.01, p = .048, partial η2 = .04, a nonsignificant main effect for gender, f(1, 95) = 0.57, p = .45, and a nonsignificant grade x gender interaction, f(1, 95) = 2.11, p = .15. that is, regardless of the gender, sixth grade students were more accurate (m = 4.73, sd = 4.38, d = 0.38) compared to the fifth grade students (m = 6.70, sd = 5.79). discussion the aim of this study was to examine students’ performance calibration in physical education. students provided estimation measures regarding their dribbling performance after they had practiced dribbling for 16 minutes following different self-regulatory conditions. two calibration indices were calculated, the bias and the accuracy index (hacker et al., 2008). the results showed no difference among groups in calibration bias and accuracy. next, data from all groups were pooled and analyzed to examine grade and gender international electronic journal of elementary education, 2012, vol.4, issue 3, 507-517 512 differences in students’ performance calibration. the results showed a main effect for grade in calibration accuracy. in particular, sixth grade students were more accurate in estimations regarding their posttest dribbling performance compared to fifth grade students. this result is consistent with views that capability self-beliefs become more accurate and specific with age and cognitive maturity (schunk & miller, 2002). growing up students become more realistic about their capabilities, are better equipped to interpret multiple sources of information about competencies, and have more differentiated views of their abilities (eccles et al., 1998). it has been supported that younger students typically overestimate how much they can remember or learn, whereas older students’ estimates of memory and learning are much closer to their actual performance on academic tasks (lan, 2005). moreover, another possible explanation of this result maybe the fact that sixth grade students displayed higher competence in dribbling performance compared to fifth grade students. calibration accuracy correlates positively with performance (bol & hacker, 2001; chen, 2003). furthermore, it has been found that low-achieving students are less accurate and have a greater tendency towards overconfidence than high-achieving students who are more accurate tending to be slightly underconfident (bol & hacker, 2001; hacker & bol, 2004; hacker et al., 2008; horgan, 1992; keren, 1987; kruger & dunning, 1999). however, this interpretation needs further examination because other research did not support it (fogarty & else, 2005). no difference was found between genders in calibration accuracy. previous research in academic settings has shown contradictory results (chen, 2003; schunk & meece, 2006). girls often perform as capably as boys in various academic domains but they may report lower self-efficacy, especially at higher academic levels (schunk & pajares, 2009). in sports setting, females may report lower levels of self-efficacy probably because their performance is usually inferior compared to males. moreover, males may underestimate the demands of a task, and females may devalue their abilities (feltz, short & sullivan, 2008). however, in the present study, the levels of calibration accuracy regarding dribbling performance were similar in both boys and girls. undoubtedly, the role of gender in students’ performance calibration in physical education requires further examination. regarding calibration bias no grade or gender difference was found. calibration bias is an index of the direction of the calibration. positive values in this index were found in both grades and genders indicating that students overestimated their dribbling performance. this result is consistent with previous findings in sport settings showing that athletes were overconfident regarding their performance, especially in more difficult tasks (fogarty & else, 2005; fogarty & ross, 2007; mcgraw et al., 2004). probably, students might have perceived the dribbling test as difficult because they were not accustomed to estimate how many cones they could dribble in a specific time. it has also been supported that students may deliberately overestimate their performance to look good to the experimenter or even to themselves (baumeister, 1998). however, these interpretations need further examination. no difference was found among students who practiced dribbling under different selfregulatory conditions. this result is consistent with kolovelonis, goudas, dermitzaki, and kitsantas’ (in press) findings that students who practiced dribbling with social feedback and set process or performance goals did not differ in calibration accuracy from control group students. it seems that calibration accuracy is hard to learn or resistant to change. probably, feedback and practice alone are insufficient for improving calibration accuracy (hacker et al., 2008). previous efforts to improve calibration accuracy have shown mixed results. zimmerman et al. (2008) found that an intervention designed to improve students’ selfreflection improved students’ self-monitoring accuracy. thus, when a self-regulated learning performance calibration in physical education / kolovelonis, goudas & dermitzaki 513 intervention designed explicitly to improve students’ self-reflection can improve their accuracy in self-monitoring. however, in the present study, none explicit technique for improving calibration was adopted. therefore, self-regulatory practice does not automatically increase calibration accuracy, unless this is explicitly pursued through a welldesigned and specific intervention. such an intervention should enhance students’ selfreflection processes regarding their achievement, help them to self-reflect on their errors, and encourage them to seek assistance from their teacher or a peer (zimmerman et al., 2008). performance calibration has implications regarding the development of self-regulated learning. it has been supported that self-regulated learners are well calibrated students (stone, 2000). students need to accurately monitor their performance and the effectiveness of the processes they use during practice because this information would be used to regulate these processes and performance (hacker et al., 2008). more accurate monitoring has been shown to lead to improved self-regulation and higher performance (thiede et al., 2003). calibration has also implications regarding students’ motivation (schunk & pajares, 2009). horgan (1992) has suggested that good calibration has motivational benefits. students with good calibration tend to make controllable attributions for both success and failure. these attributions have fewer harmful effects regarding future participation because are adaptive for students’ learning and can help them to focus on improving themselves, persisting in their learning efforts and mastering the new skill (schunk, 2008). conversely, poor calibrated students, especially those who are overconfident, are unlikely to learn from their mistakes, may suffer frustration, and lack of motivation to continue their efforts. wellcalibrated people are accurate in judging their capability to perform a task, and thus they learn more effectively (schunk & pajares, 2004). students’ performance calibration can have practical implications regarding learning skills in physical education. overconfident students may believe that they have master a skill, and thus they may put less effort during practice and may lack motivation to continue to strive for the highest levels of achievement (horgan, 1992). on the other hand, students who underestimate their capabilities may avoid challenging tasks limiting their potential development of necessary skills (schunk & pajares, 2004). thus, physical educators should help students to become well calibrated in physical education. students should be helped to fully understand the demands of the task, and what success at the tasks requires in order to prevent them from overestimating or underestimating what they can do. greater experience with various tasks informs students regarding the skills needed to succeed (schunk & pajares 2009). moreover, good calibration may facilitate effective goal setting. in particular, the calibration methodology is an effective way of assessing the realism of goals that students set (fogarty & else, 2005). furthermore, performance calibration is associated with some teaching styles. the reciprocal and the self-check styles are two teaching styles that can enhance students’ performance in physical education (kolovelonis, goudas & gerodimos, 2011). these styles involve students in the process of observing and recording their peers or their own performance. consistent with the results of the present study, kolovelonis and goudas (2012) found moderate levels of accuracy with a tendency to overestimation in students’ recordings of their peers and their own chest pass performance. training students in using these styles giving emphasis on improving their accuracy on discriminating their own or their peers’ performance may enhance students’ performance and calibration accuracy in physical education (byra, 2004). a possible limitation of this study concerns the fact that students predicted their dribbling performance in a single trial. future research should involve students to estimate their international electronic journal of elementary education, 2012, vol.4, issue 3, 507-517 514 performance in more than one single trial and examining students’ performance calibration including various types of motor and sport tasks (e.g., open or closed, gross or fine, discrete or continuous skills) from various sports. furthermore, factors associated with students’ performance miscalibration should be explored. the difficulty of the task and the provided feedback as well as students’ self-perceptions and personality characteristics may affect the status of their performance calibration in physical education. finally, although students in this study practiced dribble under different self-regulatory conditions, they were not explicitly trained to estimate their dribbling performance during practice. thus, the development and evaluation of interventions designed to improve students’ performance calibration in physical education could be a fruitful area for future research (fogarty & else, 2005). conclusions the findings of this study showed that students were overconfident when estimated their basketball dribbling performance. however, sixth grade students were more accurate compared to fifth grade students. considering the important implications of performance calibration regarding students’ motivation (schunk & pajares, 2009) and self-regulation (efklides & misailidi, 2010), interventions to improve students capability to accurately evaluate their performance are warranted. . . . references barrow, h. m., & mcgee, r. (1979). a practical approach to measurement in physical education. philadelphia: lea & febiger. baumeister, r. (1998). the self. in d. t. gilbert, s. t. fiske, & g. lindzey (eds.), the handbook of social psychology (4th ed., pp. 680-740). new york: oxford university. boekaerts, m. (1997). self-regulated learning: a new concept embraced by researchers, policy menders, educators, teachers, and students. learning and instruction, 7, 161-186. bol, l., & hacker, d. j. (2001). a comparison of the effects of practice tests and traditional review on performance and calibration. the journal of experimental education, 69, 133-151. bol, l., hacker, d. j., o’shea, p., & allen, d. (2005). the influence of overt practice, achievement level, and explanatory style on calibration accuracy and performance. the journal of experimental education, 73, 269-290. byra, m. (2004). applying a task progression to the reciprocal style of teaching. journal of physical education, recreation, and dance, 75(2), 42-46. chen, p. p. (2003). exploring the accuracy and predictability of the self-efficacy beliefs of seventh grade mathematics students. learning and individual differences, 14, 79-92. cohen, j. (1988). statistical power analysis for the behavioral sciences (2nd ed.). hillsdale, nj: lawrence erlbaum. eccles, j. s., wigfield, a., & schiefele, u. (1998). motivation to succeed. in w. damon (series ed.) & n. eisenberg (vol. ed.), handbook of child psychology, vol. 3: social, emotional, and personality development (5th ed., pp. 1017-1095). new york: wiley. efklides, a., & misailidi, p. (2010). introduction: the present and the future in metacognition. in a. efklides & p. misailidi (eds.), trends and prospects in metacognition research (pp. 1-18). new york: springer. performance calibration in physical education / kolovelonis, goudas & dermitzaki 515 efklides, a., niemivirta, m., & yamauchi, h. (2002). introduction: some issues on self-regulation to consider. psychologia, 45, 207-210. feltz, d. l., & magyar, t. m. (2006). self-efficacy and youth in sport and physical activity. in f. pajares & t. urdan (eds.), self-efficacy beliefs and adolescent development and education (pp. 161-180). greenwich, ct: information age publishing. feltz, d. l., short, s. e., & sullivan, p. j. (2008). self-efficacy in sport. champaign, il: human kinetics. fogarty, g., & else, d. (2005). performance calibration in sport: implications for self-confidence and metacognitive biases. international journal of sport and exercise psychology, 3, 41-57. fogarty, g., & ross, a. (2007). calibration in tennis: the role of feedback and expertise. in k. moore (ed.), proceedings of the 2007 conference of the australian psychological society (pp.148-152). brisbane, australia. hacker, d. j. (1998). metacognition: definitions and empirical foundations. in d. j. hacker, j. dunlosky, & a. c. graesser (eds.), metacognition in educational theory and practice (pp. 1-23). mahwah, nj: erlbaum. hacker, d. j., & bol, l. (2004). metacognitive theory: considering the social–cognitive influences. in d. m. mcinerney & s. van etten (eds.), big theories revisited: vol. 4: research on sociocultural influences on motivation and learning (pp. 275-297). greenwich, ct: information age. hacker, d. j., bol, l., & bahbahani, k. (2008). explaining calibration accuracy in classroom contexts: the effects of incentives, reflection, and explanatory style. metacognition and learning, 3, 101-121. hacker, d. j., bol, l., & keener, m. c. (2008). metacognition in education: a focus on calibration. in j. dunlosky & r. a. bjork (eds.), handbook of metamemory and memory (pp. 429-455). new york: psychology press. hacker, d. j., bol, l., horgan, d., & rakow, e. (2000). test prediction and performance in a classroom context. journal of educational psychology, 92, 160-170. horgan, d. d. (1992). children and chess expertise: the role of calibration. psychological research, 54, 44-50. keren, g. (1991). calibration and probability judgements: conceptual and methodological issues. acta psychologia, 77, 217-273. kitsantas, a., steen, s., & huie, f. (2009). the role of self-regulated strategies and goal orientation in predicting achievement of elementary school children. international electronic journal of elementary education, 2, 65-81. kolovelonis, a., & goudas, m. (2012). students’ recording accuracy in the reciprocal and the self-check styles in physical education. manuscript submitted for publication. kolovelonis, a., goudas, m., & dermitzaki, i. (2010). self-regulated learning of a motor skill through emulation and self-control levels in a physical education setting. journal of applied sport psychology, 22, 198-212. kolovelonis, a., goudas, m., & dermitzaki, i. (2011a). the effect of different goals and self-recording on self-regulation of learning a motor skill in a physical education setting. learning and instruction, 21, 355-364. kolovelonis, a., goudas, m., & dermitzaki, i. (2011b). the effects of instructional and motivational selftalk on students’ motor task performance in physical education psychology of sport and exercise, 12, 153-158. kolovelonis, a., goudas, m., & dermitzaki, i. (in press). the effects of self-talk and goal setting on selfregulation of learning a new motor skill in physical education. international journal of sport and exercise psychology. international electronic journal of elementary education, 2012, vol.4, issue 3, 507-517 516 kolovelonis, a., goudas, m., dermitzaki, i., & kitsantas, a. (in press). self-regulated learning and performance calibration among elementary physical education students. european journal of psychology of education. kolovelonis, a., goudas, m., & gerodimos, v. (2011). the effects of the reciprocal and the self-check styles on pupils’ performance in primary physical education. european physical education review, 17, 35-50. kolovelonis, a., goudas m., hassandra, m., & dermitzaki, i. (2012). self-regulated learning in physical education: examining the effects of emulative and self-control practice. psychology of sport and exercise, 13, 383-389. kruger, j., & dunning, d. (1999). unskilled and unaware of it: how difficulties in recognizing one’s own incompetence lead to inflated self-estimates. journal of personality and social psychology, 77, 11211134. lan, w. (2005). self-monitoring and its relationship with educational level and task importance. educational psychology, 25, 109-127. mcgraw, a. p., mellers, b. a., & ritov, i. (2004). the affective cost of overconfidence. journal of behavioral decision making, 17, 281-295. nietfeld, j. l., & schraw, g. (2002). the effect of knowledge and strategy training on monitoring accuracy. the journal of educational research, 95, 131-142. pieschl, s. (2009). metacognitive calibration–an extended conceptualization and potential application. metacognition and learning, 4, 3-31. schunk, d. h. (2008). attributions as motivators of self-regulated learning. in d. h. schunk & b. j. zimmerman (eds.), motivation and self-regulated learning: theory, research, and applications (pp. 245-266). new york: erlbaum. schunk, d. h., & meece, j. l. (2006). self-efficacy development in adolescents. in f. pajares & t. c. urdan (eds.), self-efficacy beliefs of adolescents (pp. 71-96). greenwich, ct: information age publishing. schunk, d. h., & miller, s. d. (2002). self-efficacy and adolescents’ motivation. in f. pajares & t. c. urdan (eds.), academic motivation of adolescents (pp. 29-52). greenwich, ct: information age. schunk, d. h., & pajares, f. (2004). self-efficacy in education revisited: empirical and applied evidence. in d. m. mcinerney & s. van etten (eds.), big theories revisited, vol. 4: research on sociocultural influences on motivation and learning (pp. 115-138). greenwich, ct: information age. schunk, d. h., & pajares, f. (2009). self-efficacy theory. in k. r. wentzel & a. wigfield (eds.), handbook of motivation at school (pp. 35-53). new york: routledge. stone, n. j. (2000). exploring the relationship between calibration and self-regulated learning. educational psychology review, 12, 437-475. thiede, k. w., anderson, m., & therriault, d. (2003). accuracy of metacognitive monitoring affects learning of texts. journal of educational psychology, 95, 66-73. winne, p. h. (2004). students’ calibration of knowledge and learning processes: implications for designing powerful software learning environments. international journal of educational research, 41, 466-488. zimmerman, b. j. (2002). achieving academic excellence: a self-regulatory perspective. in m. ferrari (ed.), the pursuits of excellence through education (pp. 85-108). mahwah, nj: lawrence erlbaum. zimmerman, b. j. (2008). investigating self-regulation and motivation: historical background, methodological developments, and future prospects. american educational research journal, 45, 166-183. performance calibration in physical education / kolovelonis, goudas & dermitzaki 517 zimmerman, b. j., moylan, a., hudesman, j., white, n., & flugman, b. (2008). enhancing self-reflection and mathematics achievement of at-risk students at an urban technical college: a self-regulated learning intervention. u.s. department of education. microsoft word 8ejee_7_1_rasinski international electronic journal of elementary education, 2014, 7(1), 3-12. issn:1307-9298 copyright © iejee www.iejee.com fluency matters timothy rasinski ∗∗∗∗ kent state university, kent, oh, usa received: 23 october 2014 / revised: 29 october 2014 / accepted: 31 october 2014 abstract although reading fluency has been dismissed and overlooked as an important component of effective reading instruction, the author makes that case that fluency continues to be essential for success in learning to read. moreover, many students who struggle in reading manifest difficulties in reading fluency. after defining reading fluency, the article explores proven methods for improving reading fluency, and finally explores questions regarding fluency that when answered may lead to a greater emphasis on and understanding of reading fluency as a necessary part of teaching reading. keywords: fluency, reading, struggling readers, automaticity, prosody introduction in the late 1970s i was working as an intervention teacher, providing instruction mainly to primary grade students who were experiencing difficulty in reading. for many of these students simply helping them master and put into practice their knowledge of soundsymbol relationships was sufficient to move the students forward. for a fairly significant number of students, more and different phonics instruction was not enough. they were already fairly good at sounding out written letters and decoding words. however, reading orally was clearly a painful experience. although most of the words they encountered were read correctly, their reading was marked by excessively slow, letter by letter and word by word reading, lengthy pausing, and lack of expression. and, of course, this sort of reading also resulted in poor comprehension. it was clear that these students were not enjoying the experience nor were such experiences advancing their growth in reading for my part, i did not know exactly what else i should be doing. i had been doing instruction that was conventional for the day – language experience approach, phonics, read aloud to students, discussions of the texts had read. yet, none of these approaches seemed to tap into the needs that were manifested in these students. fortunately i had been working on my masters’ degree at the time and one professor had us reading some professional articles that were beginning to appear on this concept called reading fluency. one piece in particular by carole chomsky (1976) entitled “after decoding: what? “ described an intervention where students were asked to read a text repeatedly while simultaneously listening to a fluent oral rendering of the text until they were able to read the text well on ∗ timothy rasinski, reading and writing center, kent state university, kent, oh 44242 usa. phone: 330-672-0649. e-mail: trasinsk@kent.edu international electronic journal of elementary education vol.7, issue 1, 3-12,2014 4 their own without the assistance of the recording. then students would continue the routine using a new text. the approach seemed deceptively simple and since i was out of instructional ammunition i decided to give chomsky’s approach a try with my own students. remarkably, my students began to make significant progress in reading. moreover, i found that as students achieved levels of reading performance that was the equal of their more normal developing classmates, they began to see themselves as readers and were developing confidence in themselves as readers. although i had stumble on an approach to improving reading through reading fluency instruction, i discovered that fluency was not all that popular a topic in reading education. i recall digging through the teacher’s edition of the reading series we used in school, looking for reading fluency and how it was taught. although i found detailed strands of instruction for phonics, vocabulary, and comprehension, i found very little that dealt with fluency and the development of fluency in students. i decided then that fluency was a topic i need to learn more about. defining fluency i have found that reading fluency can mean different things to different people. so, i would like to share my understanding of reading fluency. reading fluency is made up of two distinct components at two ends of the reading spectrum – automaticity in word recognition and expression in oral reading that reflects the meaning of the text. in a sense, reading fluency is the essential link between word recognition at one end of the spectrum and reading comprehension at the other. automaticity in word recognition refers to the ability to recognize or decode words not just accurately, but also automatically or effortlessly. in their seminal article on reading fluency, laberge and samuels (1974) noted that all readers have a finite amount of attention or cognitive energy to accomplish two essential tasks in reading – word recognition and comprehension. attention expended for one task cannot be applied to another, it is used up. and so, when readers have to use excessive amount of their cognitive energy for word recognition, even if they are able to decode the words accurately, they have reduce the amount of cognitive energy available for comprehension and thus, comprehension suffers. these were the readers i was working with in my intervention class. they were able to decode most of the words, but simply listening to the excessive slowness of their word decoding, it was not difficult to tell that they were using up plenty of their cognitive resources analyzing and decoding the individual words in the text, they had little attention left for making sense of what they were reading. automatic word recognition takes phonics to the next level. automatic readers not only recognize words accurately, they do it with minimal employment of their cognitive resources. the best examples of automatic readers are you, the person reading this article. as you read this piece, how many of the words did you have to analyze in order to sound out correctly? my guess is few if any. most of the words you encountered in this article were identified by you instantly and effortlessly. your minimal employment of attention means that you can reserve your attention for making meaning, or understanding the text itself. expression in oral reading, or prosody, is fluency’s connection to meaning or comprehension. in order to read something with appropriate expression that reflects the author’s purpose and meaning, the reader must have some degree of comprehension of the passage itself. indeed, when reading orally with appropriate expression the reader is enhancing his or her own comprehension by using various prosodic elements (volume, pitch, phrasing, etc.) to expand on the meaning. again, as i reflect on the students i had been fluency matters / rasinski 5 working with many years ago, their lack of expression and confidence in their oral reading was clearly apparent. why reading fluency matters fluency matters simply because it is an essential element of proficient and meaningful reading. in his “interactive compensatory model” of reading fluency, stanovich (1980) argued that the automaticity component of fluency is a distinguishing factor between good and struggling readers. good readers are so automatic or effortless at the bottom up word processing requirement for reading, they can use employ their finite cognitive resources for the more important top-down requirement for reading – comprehension. struggling readers, on the other hand, are not automatic in their word recognition, so they must use their cognitive resources for the more basic bottom-up of word recognition, thereby depleting what they will have available for more important top-down task – making meaning. in offering an alternative explanation of reading fluency that focused on prosody, schreiber (1980) suggested that good readers employ prosody in their reading to phrase text into syntactically appropriate and meaningful units that are not always explicitly marked by punctuation. additionally, the oral emphasis placed on particular words or phrases in a written text create inferences that allow readers to understand text at level deeper than literal comprehension. over the past 30+ years, a growing body of evidence has demonstrated the link between both components of fluency and proficient and meaningful reading (rasinski, reutzel, chard, & linan-thompson, 2011). moreover, research into students who are identified as struggling readers or who perform poorly on high stakes silent reading comprehension tests has found that poor reading fluency appears to be a major contributing factor to their poor reading (rasinski, & padak, 1998; valencia & buly, 2004). further, although reading fluency is identified as a foundational reading competency in the united states by the common core state standards (common core state standards initiative, 2014), an expanding body of research has shown that many students in the upper elementary, middle, and secondary grades have not achieved adequate levels of fluency in their reading and thus experience difficulty in others of reading, including silent reading comprehension (rasinski, et al, 2009; rasinski, rikli, & johnston, 2010; paige, rasinski, & magpuri-lavell, 2012). although reading fluency has been studied extensively for readers of english, the concept of fluency in reading should apply to the reading of other written languages and at least one study has demonstrated a relationship between reading fluency and proficiency in reading among fifth-grade turkish readers (yildirim, ates, rasinski, fitzgerald, & zimmerman, 2014). despite the growing evidence of the importance of fluency in reading, it is ironic that in the united states its perceived importance among literacy scholars and educators has been on the decline. for the past several years, annual surveys of literacy experts have consistently identified reading fluency as of one of the few topics that is considered “not hot” (rasinski, 2012). moreover, the same respondents also indicated strongly that reading fluency should not be considered a hot topic in reading. this disconnect may be due to the way reading fluency is commonly assessed and taught in many schools and in many commercial instructional programs aimed at teaching fluency. assessing and monitoring reading fluency in order to determine if fluency is a concern among readers and how progress in fluency can be monitored, we need to have methods of assessing fluency. since automaticity refers to the ability to recognize words instantly and effortlessly, reading speed or rate offers a simple international electronic journal of elementary education vol.7, issue 1, 3-12,2014 6 approach to measuring this component of fluency. readers who are automatic in their word recognition tend to read at a faster rate than readers who are less automatic; moreover readers who are automatic in word recognition should also be better in reading comprehension. research has consistently demonstrated significant and substantial correlations between measures of reading rate and reading comprehension or other general measures of overall reading proficiency at a variety of grade levels (e.g., deno, 1985; rasinski, reutzel, chard, & linan-thompson, 2011). the most common protocol for assessing reading rate automaticity is to have a student read a grade level text for 60 seconds and simply count the number of words read correctly in that minute. the reading rate score can then be compared against grade level norms for students in the elementary and middle grades (e.g. hasbrouck & tindal, 2006). students who fall significantly below the 50%ile score may be considered at risk in terms of the automaticity component of fluency. while reading rate has been established as a strong measure of automaticity, a major potential problem occurs when reading fluency becomes instruction on how to increase one’s reading rate. such an approach seems to have dominated reading fluency instruction over the past decade in the united states. the unintended consequence of such instruction is the development of readers who understand reading to be all about reading as fast as possible. of course, reading becomes the quest for speed, reading comprehension often falls by the wayside (rasinski & hamman, 2010). reading speed is an outcome of automaticity, it is not the cause of automaticity. automaticity in word recognition, as described in the next section is developed through extensive practice of authentic reading experiences. as i mentioned earlier in this article, i think most readers of this piece would consider themselves fluent in terms of their word recognition automaticity; yet i would conjecture that few, if any, of you experienced the kind of reading speed instruction that seems to have dominated reading fluency. rather, we developed our automaticity in reading simply by reading extensively. plenty of exposure to words and word patterns caused those words and patterns to become fixed in our memories and easily retrieved when exposed to them in subsequent readings. prosody or expressiveness in reading is the other component in reading. while there have been recent studies that have used high tech methods for assessing components of prosody, the most practical approach for assessing prosody is for informed teachers to simply listen to students read orally and to rate the students’ expressiveness on a guiding rubric (e.g., zutell & rasinski, 1992). although a subjective measure of prosody, studies have found that such approaches are valid, reliable, and well correlated with other general measures of reading proficiency. still, the subjective nature of assessments of prosody means that such assessments are often ignored or employed on a limited basis in schools. as a result, since prosody is not overly emphasized in summative or formative assessments it is often not taught or emphasized in instructional environments. as a result, an important aspect of fluency instruction is often minimized, thus also leading to its identification as “not hot” among reading experts. teaching reading fluency think of how you became fluent at any task and you’ll probably get a good sense of how reading fluency can be taught. i consider myself a fairly “fluent” driver – despite driving over 12,000 miles per year, i have not been in an accident in over 20 years not have i received a traffic violation ticket over that same period. how did i become the fluent driver that i am today? first, i watched my parents, and other adults in my life, drive during the first 16 years of my life. i observed the protocol my parents used for starting, backing, driving, and parking fluency matters / rasinski 7 the family car in various scenarios. i also became acquainted with the various controls on different cars and the rules for driving as well as the signs that help to direct drivers. when i turned 16 and received my learners’ permit to drive, i was finally able to get behind the wheel and drive the family car on my own. however, i was never alone in these situations. i always had one of my parents or another adult sitting next to me, offering me instructions, guidance, and encouragement as i gradually learned the skill of driving. as i became more and more proficient in my driving, my parents continued to sit next to me, but they offered less and less guidance. finally, my driving skills were tested at the local drivers’ licensing station, i was found to be competent to a minimally acceptable level, and was issued a state drivers’ license that allowed me to drive by myself, without the guidance or support of an adult passenger sitting next to me. i must admit that even though i had my license to drive, i was not a skilled driver. i had several minor accidents and also was issued a few warnings and traffic tickets by the local police who observed me making deriving errors. however, i continued to “practice” my driving, driving a variety of automobiles over the course of several years. today, i consider myself a “fluent” or very competent driver of nearly any type of conventional automobile. what i find interesting is that i am so competent (accurate and automatic) in my driving ability i am able to engage in some other tasks while driving – i can listen to the radio, chat with a passenger, or even talk on the cell phone while driving legally and safely. this analogy also applies to reading where fluent readers are able to multi-task – they are able to read the words in the text so accurately and automatically that i can, at the same time, focus my attention on making meaning from the text. essentially my road to fluency in driving began with modeling of fluent driving by my parents, supported driving where my parents or other competent adult driver sat next to me while i drove to offer guidance, and finally independent practice in driving. the independent practice involved repeated practice on my parents’ car at first, but as my driving proficiency increased i was also able to drive a wider variety of automobiles, from my brother and sister’s cars to cars owned by other relative and friends. learning to become a fluent reader is in many ways analogous to learning to drive. model fluent reading just as i spent a significant amount of time observing my parents drive during my early years, children need to observe fluent reading by adults and other fluent readers. the value of adults reading to children is compelling (rasinski, 2010). reading to children increases children’s motivation for reading, enlarges their vocabulary, and also improves their comprehension. reading to children also provides children with a model of what oral reading should sound like – embedded with expression that helps to enhance the listener’s understanding of the text. often when i read to students we will follow up a quick discussion of the story itself with a discussion of how “dr. rasinski read the story.” i will try to make note of various prosodic features i embedded in my reading (e.g. “did you notice how i changed my voice when i became a different character?” “what were you thinking when i made my voice louder and faster as this particular point in the story?) and help them see that these features helped with their satisfaction with and understanding and enjoyment of the text itself. occasionally when i read to students i will purposely start by reading in a less-than-fluent manner (too fast, too slow, too much of a monotone). after a couple sentences i stop and ask them what they noticed in my reading. they are not generally impressed with this sort of reading. their satisfaction and understanding of the text was impaired by such disfluent reading. of course, my message to the students is that they do not understand well or have international electronic journal of elementary education vol.7, issue 1, 3-12,2014 8 much satisfaction with texts read in such a manner, they should try not to read in such a manner themselves when reading independently as it will limit their understanding and enjoyment of their texts. provide fluency support through assisted reading when i first began driving, i had the assistance of an adult who sat next to me in the car and provided expert support while driving. support or assistance can also be made available to students while reading in order to improve their fluency. essentially assisted reading involves the novice reader reading a text while simultaneously listening to a fluent oral rendering of the text. as you may recall, carole chomsky’s research that was so influential to me involved a form of assisted reading – students read a text while listening to a pre-recorded version of the same text. assisted reading provides support in at least two essential ways. first it allows the students to decode all the words in the text successfully, even those that they would not be able to decode if reading on their own. second, by listening to a fluent reading of the text, students are provided with a positive model of an expressive and meaningful reading of the text. students hear prosody in action while reading the same text. assisted reading, then, essentially supports both word recognition accuracy and automaticity as well as prosodic reading. assisted reading can take a variety of forms. one of the most common is a novice reader sitting next to a more fluent partner reader, with both readers reading the same text together. various names and protocols have been used and developed to operationalize partner reading. in their review and summary of research on partner reading rasinski, reutzel, chard, and linan-thompson report that the various iterations of this form of assisted reading to result in positive reading outcomes for students. technology offers some interesting assisted reading applications. students reading a text while listening to a fluent recorded version of the same text are engaged in assisted reading. recent developments in technology have freed students from cassette tapes, tape recorders, and compact disc recordings. using readily available voice recording applications, teachers (or others) can record their reading of a text, save the recording as a digital file, provide access to the recording via the internet, and have students read while listening to the digital recording on a mobile device. although the studies using technology –assisted reading is limited, the results of the existing studies demonstrate great potential for improving students’ fluency and overall reading achievement (rasinski, reutzel, chard, & linanthompson, 2011). reading practice once i had developed a minimally acceptable level of proficiency in my driving to be permitted to drive independently, i needed to continue practice my driving in order to achieve a level of high fluency. similarly, developing readers need opportunities to read independently in order to achieve high levels of fluency, both automaticity and prosody, in their own reading. reading practice can actually take two general forms. the first and most common form of practice is wide reading. this is the type of reading that adults typically engage in and it is also the type of reading that usually occurs in school settings. students read a text, discuss the reading with the teacher and/or classmates, perhaps engage in some extension activities related to the text, and then move on to the next text or book chapter. wide reading is essentially on reading after another. clearly this form of reading is important, in both silent and oral forms. perhaps one of the most common forms of wide reading is found in the daily independent reading or sustained silent reading time often to students. the cliché, “the fluency matters / rasinski 9 more you read, the better reader you will become” has a lot of surface level truth to it. it is difficult to imagine a person becoming a proficient reader without practicing the craft of reading independently. although not universally endorsed as an instructional activity (e.g. national reading panel, 2000), a growing body of scholarly writing (e.g. stanovich, 1986; morgan, mraz, padak, & rasinski, 2008) and research (e.g. allington, et al, 2010) suggests that increasing the volume of students’ independent reading will yield improvements in students’ reading fluency and other measures of reading proficiency. reutzel, jones, fawson, and smith (2008) argue that students often do not have much guidance or accountability in many independent riding protocols and suggest that providing greater structure and accountability during independent reading will yield even more positive results in students’ reading outcomes. when learning to drive i found that i practiced only on my family’s car for several weeks before moving on to driving other cars. reflecting back on this experience, it seems to me that if i had moved from one car to another after only driving each car once i would have not achieved a sense of mastery over the first car and experienced considerable difficulty quickly switching to new cars as new each car would be somewhat different from the others. by practicing only on one car for a period of time, i was able to master that car. then, when i finally transferred by driving skills to other cars, what i had learned on that initial car was able to be transferred to other automobiles. i think this repeated practice analogy also applies to reading. many of our struggling readers read a text only once during wide reading and they do not read it well. yet, they move on to a new text and read it once (and not very proficiently) as well. it will be difficult for these students to achieve fluency in general, if they are not given opportunities to achieve fluency over particular texts. repeated practice on the same text (or car when learning to drive) allows students to achieve this form of fluency or mastery than can easily transfer to new, never-before-read texts. in his landmark study on repeated readings (samuels, 1979) had struggling readers read a text repeatedly until they achieved a certain level of proficiency on that text. of course with practice they demonstrated improvement on the text practiced. the more interesting finding from samuels’ research was that when students moved on to new texts that were as or more difficult than the previous text, there were vestiges of improvement on the new text as well. in the same way that i was able to transfer skills from one automobile to another after repeated practice of the first car, so to students are able to transfer competencies in reading fluency from one text to another by engaging in repeated reading of the original text. in their review of subsequent research on repeated reading with guidance and feedback provided to students, rasinski, reutzel, chard, and linan-thompson (2011) conclude that such practice “has been shown to effective in promoting fluency growth among a variety of students across differing reading levels and text levels” (p. 301). while repeated readings has been shown to be effective in improving reading, a problem in implementing repeated reading has caused some educators to question its value. in many programs for developing fluency, because automaticity is often measured by reading rate or speed, the goal of the repeated reading is to increase students’ reading rate from one reading to the next. this is not a terribly authentic reading experience as very few adult reading experiences requires adults to practice a text repeatedly for the purpose of reading the text fast. as mentioned earlier, the result of such overt emphasis on reading speed is a diminished focus by students on prosody and meaning while readings. it seems that a more authentic approach to repeated reading where adults do, indeed, practice or rehearse a text. rehearsal is truly a form of repeated reading where the rehearsal international electronic journal of elementary education vol.7, issue 1, 3-12,2014 10 is aimed at developing a prosodic and meaningful oral interpretation of the text. texts that are often rehearsed and then preformed for a listening audience include scripts, poetry, song lyrics, speeches, and more. several classroom-based studies have found that when students engage in a more authentic repeated reading and performance experience they make exceptional gains on various dimensions of reading, including measures of reading fluency (griffith & rasinski, 2004; martinez, roser, & strecker, 1999; young & rasinski, 2009). next steps it is clear that reading fluency is an important competency that needs to be addressed in the literacy classrooms around the world. research and scholarly writing have demonstrated that fluency is conceptually an important reading competency, that it can be measured relatively easily and quickly, and that instructional methods have been developed that have shown to be effective in improving students’ fluency. still, although much is known about fluency, there are many questions and concerns that remain. here are just a few based on my own understanding of the concept. first, the concept of fluency itself may be a source of confusion as it appears to include two separate subordinate competencies (automaticity in word recognition and prosody) and related to a second major competency (word recognition). for some scholars and practitioners fluency in reading is automaticity, for others it is word recognition accuracy, for still others it is prosody, and for some it simply means generally proficient reading. it may be helpful if scholar began to sort these concepts out for clarity sake. one possibility would be to simply refer to word recognition accuracy, word recognition automaticity, and prosody as three distinct reading competencies. reading fluency could then be used as a synonym for overall proficient reading. the role of text type and text difficulty clearly needs further consideration for fluency development. in many existing programs for teaching fluency informational text is the primary text students used. the rationale for using such texts is that greater emphasis is being placed on students engaging in informational text reading, even in the primary grades. while there are compelling reasons for students to read more informational texts, i wonder if reading fluency instruction is good place for such texts to be used. informational texts are generally rather lengthy. if students are asked to engage in repeated readings, the texts used cannot be excessively long as the repeated reading of a lengthy text would take more time than what would normally be allotted for fluency instruction. secondly, the nature of informational texts does not easily lend themselves to expressive oral reading (prosody). it may be wise to consider other text genres, genres that are meant to be performed orally. if texts are meant to be read orally for an audience they need to rehearsed (repeated reading) with the purpose of the rehearsal being expressive reading to aid the understanding of the audience. as mentioned earlier, texts that are meant to be rehearsed and performed include scripts, poetry, and song lyrics among others. poetry and song lyrics also have the added feature of being relatively short, making them ideal for repeated reading over a short period of time. interestingly though, these genres of texts have been regularly reduced in terms of their perceived importance and inclusion in the elementary grades. text level of difficulty is another issue that needs to be considered as we move forward in fluency. should students be asked to read easy texts or texts that considered more challenging. on the surface it would seem that easier texts or texts that are within students’ instructional levels would be the appropriate choices as students are more likely to achieve fluency more quickly on such texts. there is a body of scholarly thought and evidence to support the use of such text levels especially with struggling readers (hiebert & mesmer, fluency matters / rasinski 11 2013). however, in their review of fluency instruction, kuhn and stahl (2004) noted 6 studies that found that students experienced greater benefits when the reading texts were somewhat above the students’ instructional reading levels as opposed to when the materials were below their instructional levels. is it possible to accelerate students’ reading fluency progress by providing them with materials to read, along with appropriate support, that are above the level they normally would be asked to read instructionally? clearly, this is an area of great importance. the issue of stamina in reading is one that has not been addressed sufficiently in fluency research. in most studies fluency is assessed during the first minute of reading a text. moreover, fluency instruction generally occurs using relative short passages that can be read in less than five minutes. we do not know the impact on fluency or fluency’s impact on comprehension as students become more involved in a text at one setting. does fluency improve or decline in the 20th minute of reading? finally, reading fluency has been identified as a foundational reading competency that should be mastered no later than grade 5 or below (common core state standards initiative, 2014). yet, a growing body of research is demonstrating that significant numbers of students have yet to achieve sufficient levels of fluency, both automaticity and prosody, in the middle and secondary grades. moreover, these students are likely to manifest difficulties in other areas of reading including silent reading comprehension. how is it that so many students appear to flow through the cracks? what can be done to assure that students attain and maintain adequate levels of reading fluency beyond the primary grades? i truly believe that reading educators can make a significant impact on student reading achievement and academic achievement in other areas that require fluency when answers to these and other questions can be found. despite the rocky road that reading fluency has traversed over the past several decades, many reading scholars continue to view it a critical foundational competency for students to achieve. instructional methods and materials have been identified to improve fluency in students, especially those students who struggle in gaining fluency. not only can fluency instruction be effective in improving students’ reading proficiency, it can also be an authentic, engaging, and pleasurable experience for students. as omar, a student whose teacher used readers theatre scripts to improve his reading fluency and overall reading performance, indicated, “readers theatre is the funnest reading i ever did before” (martinez, roser, & stecker, 1999, p. 333). • • • timothy rasinski is a professor of curriculum and instruction in the reading and writing center at kent state university, kent, oh, usa. his scholarly interests include reading fluency and students who struggle in learning to read. he has written extensively on reading fluency and is author of the fluent reader (scholastic), a prominent professional book on fluency. international electronic journal of elementary education vol.7, issue 1, 3-12,2014 12 references allington, r., mcgill-franzen, a., camilli, g., williams, l., graff, j., zeig, j., zmach, c., nowak, r. (2010). addressing summer reading setback among cconomically disadvantaged elementary students. reading psychology , 31(5), 411-427. chomsky, c. (1976). after decoding: what? language arts, 53, 288-296. common core state standards initiative (2014). common core state standards initiative. retrieved june 24, 2014 from: http://www.corestandards.org/. griffith, l. w., & rasinski, t. v. (2004). a focus on fluency: how one teacher incorporated fluency with her reading curriculum. the reading teacher, 58(2), 126-137. hasbrouck, j., & tindal, g. a. (2006) oral reading fluency norms: a valuable assessment tool for reading teachers. the reading teacher, 59(7), 636-644. hiebert, e. h., & mesmer, h. a. (2013). upping the ante of text complexity in the common core state standards: examining its potential impact on young readers. educational researcher, 42, 44-51. martinez, m., roser, n., & strecker, s. (1999). “i never thought i could be a star”: a readers theatre ticket to reading fluency. the reading teacher, 52, 326-334. laberge, d. & samuels, s.a. (1974). toward a theory of automatic information processing in reading. cognitive psychology, 6, 293-323. morgan, d., mraz, m., padak, n., & rasinski, t. (2008). independent reading. new york, ny: guilford. paige, d. d., rasinski, t. v., & magpuri-lavell, t. (2012). is fluent, expressive reading important for high school readers? journal of adolescent & adult literacy, 56(1), 67–76. rasinski, t., & hamman, p. (2010). fluency: why it is “not hot.” reading today, 28, 26. rasinski, t. v., & padak, n. d. (1998). how elementary students referred for compensatory reading instruction perform on school-based measures of word recognition, fluency, and comprehension. reading psychology: an international quarterly, 19, 185–216. reutzel, d. r., jones, c., fawson, p., & smith, j. (2008), scaffolded silent reading: a complement to guided repeated oral reading that works!. the reading teacher, 62, 194–207. samuels, s. j. (1979). the method of repeated reading. the reading teacher, 32, 403-408. schreiber, p. a. (1980). on the acquisition of reading fluency. journal of reading behavior, 12, 177-186. stanovich, k. e. (1986). matthew effects in reading: some consequences of individual differences in the acquisition of literacy. reading research quarterly, 21, 360-407. stanovich, k. e. (1980). toward an interactive-compensatory model of individual differences in the development of reading fluency. reading research quarterly, 16, 32-71. valencia, s. w., & buly, m. r. (2004). behind test scores: what struggling readers really need. the reading teacher , 57, 520-531. yildirim, k., ates, h., rasinski, t., fitzgerald, s., & zimmerman, b. (2014). the relationship between reading fluency and comprehension in fifth-grade turkish students. international journal of school and educational psychology, 2(1), 35-44. young, c. and rasinski, t. (2009), implementing readers theatre as an approach to classroom fluency instruction. the reading teacher, 63, 4–13. international electronic journal of elementary education, 9(3), 681-692, march 2017 issn:1307-9298 copyright © iejee www.iejee.com use of information and communication technologies in primary education – a case study of the czech republic pavla hlásná a blanka klímová a  petra poulová a a university of hradec kralove, czech republic received: 8 november 2016 / revised: 12 january 2017 / accepted: 17 january 2017 abstract the aim of this research study is to explore the use of information and communication technologies (ict) in classes at the first stage of primary schools, specifically in the czech republic. firstly, the authors discuss the current state of this research issue, and secondly, they describe their own research which should clarify how, why and how often the teachers at the first stage of primary schools use ict in their teaching. the findings indicate that although more than 50% of the teachers at primary schools use ict in their teaching on a daily basis, they need further continuous methodological training which would contribute to their effective use of ict in classrooms. the results also showed that the teachers who had participated in a methodological course on ict use appeared to implement ict in their classes more than those who had not attended such training. in addition, the results revealed that the use of ict was not influenced by the length of the teaching practice and that the use of ict still did not have any impact on the relationship between the teacher and his/her pupils and among pupils themselves. nevertheless, the results suggest that there should be ample and continuous trainings which would ensure that teachers have relevant competences for using ict in their classrooms. keywords: education, teachers, ict, primary schools, case study introduction at present, modern information and communication technologies (ict) penetrate in all spheres of human activities, including education, in order to increase effectiveness and quality of teacher’s work. in fact, ict are now an inseparable part of children’s lives. they consider them as natural as breathing (prensky, 2004). therefore, schools at all educational levels try to incorporate them in their curricula. research connected with the use of ict in education has already run for more than 40 years (cf. baytak, tarman, & ayas, 2011; brdicka, 2003; collins, 1996; domingo & gargante, 2015; murphy et al., 2003). research evidence reveals that ict have a positive impact on learning, especially in primary education (badia et al., 2014; inan & lowther, 2010; or van braak et al., 2004).  corresponding author: blanka klímová, faculty of informatics and management, university of hradec kralove, rokitanskeho 62, hradec kralove, 500 03, czech republic. phone: +420 493 332 813 e-mail: blanka.klimova@uhk.cz http://www.iejee.com/ international electronic journal of elementary education, 9(3), 681-692, march, 2017 682 frydrychova klimova (2012) lists several benefits of the use of ict on learning, which is also true for primary education:  learning concentrates rather on the learner than a teacher;  learning becomes more personalized;  learning becomes on the one hand more independent, on the other more collaborative and interactive;  learning can happen at any place and any time;  learning is enriched with more up-to-date materials, which can be tailored according to students’ immediate needs;  thanks to multimedia activities, learning becomes more varied and dynamic;  learning requires critical thinking;  learning becomes more culture conscious. the importance, efficacy and usefulness of ict in primary education have been also reflected in many cross-national and national comparative research studies such as computers in education (comped, 1989-1992), second information technology in education study (sites, 2006), or study of the impact of technology in primary school (steps, 2007). the latest international research study conducted in the area of primary education was international computer and information literacy study (icil, 2013). in the czech republic, the most significant project in this field was the project on information and communication technologies in everyday teacher’s work (zounek & sedova, 2009). the research project was performed only in one region of the czech republic and focused on the teacher’s attitude towards technologies. the results of the project were as follows:  teachers perceive ict in their teaching as a good opportunity or as a recognized necessity;  teachers used ict in a traditional way, i.e., for testing and explanation of the study material;  relationship between pupils and teachers still remain traditional in the new technological environment;  teachers incorporate ict into their teaching due to their popularity among pupils, not for the didactic purpose;  teachers admit not having relevant ict skills and an overview of materials and applications used in the teaching enhanced by ict. research (kiper & tercan, 2012) also indicates that especially the present novice teachers, at primary schools are willing to use ict in their teaching, however, they need a systematic methodological input on the use of ict in primary education. in addition, most of the schools still have a lack of teachers who would possess superior user or administrator skills (annual report of the czech school inspection, 2011; 2012). thanks to the eu projects, basic and secondary schools are now relatively well equipped with modern ict. this is also confirmed by annual reports of the czech school inspection (annual report of the czech school inspection, 2011) and european schoolnet (czech republic country report on ict in education, 2011). ict form an inseparable part of basic education, used as an instructional tool, s learning tool and as a school subject. the subject ict is taught once a week for 45 minutes (1 lesson) at primary schools (rvpzv, 2007). the purpose of this study is to explore teachers’ attitude towards the use of ict at the first stage of primary schools in the czech republic. firstly, the authors discuss the current state of this research issue in the countries of the european union and secondly, they describe their own research which should clarify how, why and how often the teachers at the first stage of primary schools use ict in their teaching. use of information and communication technologies in primary education / hlásná, klímová & poulová 683 research questions, hypotheses, phases and methods of the research the aim of this research study was to find answers to the following three research questions: 1. does the length of teaching practice of the teachers at the first stage of primary schools influence the frequency of the use of ict in teaching? 2. does further education of the teachers at the first stage of primary schools have an impact on the frequency of the use of ict in teaching? 3. do the relationships between the teacher and pupil and among pupils themselves change as a consequence of the use of ict during lessons? on the basis of the research questions stated above, the following two hypotheses were defined: hypothesis a: younger teachers (less experienced) use ict more often than older teachers (with a longer teaching practice). hypothesis b: teachers of the first stage of primary school who participated in an ict training use ict in teaching more than the teachers who did not participate in such a training. the research started in september of 2010 and finished at the end of the year 2015. consult fig. 1 below for the research timeline. figure 1. research timeline (authors’ own processing) the research was performed in five phases, which are described together with the methods used for the analysis of the research issues. 2010 2011 2012 2013 2014 2015 literature review setting methodology and tools own research analysis of questionnaire interpretation of results international electronic journal of elementary education, 9(3), 681-692, march, 2017 684 phase 1 phase 1 involved a comparative analysis of the use of ict in the subject man and his world in the fourth and fifth grades of the basic school. the teachers of this subject at five basic schools in hradec kralove were interviewed and on the basis of these interviews, a pilot questionnaire was made. phase 2 the pilot questionnaire was finalized. this questionnaire was partially based on already validated and used questionnaire made by zounek and sedova (2009). the questionnaire as a research tool was statistically processed and its reliability was set at 0.71 (cronbach coefficient alfa). the pilot version of the questionnaire was distributed among randomly selected 15 parttime students (future teachers of the first stage of primary education, out of which 13 were females and 2 were males at the age of 25-51 with the teaching practice between 2 and 26 years) at the university of hradec kralove. the questionnaire consisted of 15 closed and 4 open questions. smaller insufficiencies in the questionnaire were removed and the newly modified questionnaires could be distributed. phase 3 the questionnaire was distributed into 230 basic schools in the region of hradec kralove. altogether 135 respondents returned the filled in questionnaire. 129 were females and 6 were males at the age of 26 – 60 years with the teaching practice between 3 months and 40 years. the aim of the questionnaire was to discover:  in what ways the teachers use ict;  how often they use ict in their classes;  how ict are applied in classes;  to what extent ict influence teaching;  to what extent ict penetrate into the relationship between the teacher and pupil and among the pupils themselves;  what factors influence the teachers of the first stage at primary schools to implement ict into their teaching;  what kind of training on ict the teachers have already had;  whether the age of the teacher influences his/her attitude towards ict or not. phase 4 phase 4 involved a standardized observation which was done with the help of interactive analysis based on the observation and evaluation of communication and interaction in the classroom. out of 53 available observation systems, the authors selected flanders system of interactive analysis (flanders, 1970). flanders set basic activities of a teacher and a pupil. these activities repeat during classes and their mutual proportion during classes characterizes this process. svatos and dolezalova (2011) expanded and slightly modified these categories. in addition, manenova (2012) added one more category which describes a form of pedagogical interaction – a description of silence, chaos in the classroom. all these categories are defined below:  t1 – teacher accepts pupil’s feelings, behavior, s/he tries to show empathy in a constructive way.  t2 – teacher evaluates pupils positively, s/he praises pupil’s performance. use of information and communication technologies in primary education / hlásná, klímová & poulová 685  t3 – teacher uses, explains, develops or accepts pupil’s ideas.  t4 – teacher asks pupils questions which are connected with the study material, working methods or organizational matters.  t5 – teacher informs, explains or states his/her opinions.  t6 – teacher gives instructions, orders.  t7 – teacher criticizes performance, answers or activities of pupils or their behavior.  p1 – pupil asks questions, s/he seeks support and help at his/her teacher.  p2 – pupil asks questions, s/he seeks support and help at his/her peers.  p3 – pupil informs, explains, states his/her opinions – under pressure or influence of his/her teacher.  p4 – pupil informs, explains, states his/her opinions – out of his/her own activity and motivation.  p5 – pupil directs, modifies activities of other pupil(s), provides help to other pupil(s).  p6 – goup work when pupils communicate with each other.  p7 – all class discussion.  p8 – pupils work independently, without apparent interaction.  o1 – silence or chaos in the classroom, pauses (slurred communication). explanation: t – teacher, p – pupil, o organization these data were evaluated both in a quantitative and statistical way. a special program codenet developed in the department of pedagogy and psychology of the faculty of education at the university of hradec kralove was used for the acquisition and quantitative processing of the data. the research ran from september of 2011 till january of 2012. the aim was to discover characteristics of four lessons with the use of interactive board. phase 5 phase 5 included the evaluation of the results and their comparison with other research studies. results results of the questionnaire the findings showed that 99% of the respondents actively used ict in classes, mainly to stimulate pupils to learn, to test and evaluate pupils and to explain new subject matter. almost three quarters (57%) of the respondents use ict very often, 27% several times a week, 15% only several times a months and only 1% never uses ict in classes. furthermore, the findings also demonstrated that pupils considered the use of ict in their classes as an inseparable, normal part of teaching to which they had already got used to. the results of the questionnaire also revealed the fact that 90% of the respondents did not think that the use of ict would cause a loss of personal, face-to-face contact between the teacher and his/her pupil or among the pupils themselves. 62% of the respondents also reported that they did not believe that the pupils would learn faster with the help of ict and 51% did not consider their teaching more effective when using ict. as the findings indicate, the teachers most often exploit ict for the following activities:  92% of the respondents use ict to practice and revise the study material;  88% of the respondents use ict to access pictures, music or demonstration of various phenomena;  66% of the respondents give pupils tasks through ict; international electronic journal of elementary education, 9(3), 681-692, march, 2017 686  65% of the respondents use ict as a tool for the explanation of the new subject matter;  37% of the respondents use ict to test pupils;  4% of the respondents let pupils study the new material with the help of ict (e.g., with the help of computer tutorials, the internet or a video film). table 1 below then illustrates the type of ict the teachers use in their preparation for classes as well as during classes. table 1. ict used in the preparation for classes and during classes interactive board 83 % internet 91 % data projector 43 % cd player 84 % television 25 % video 41 % mp3 player 14 % printer connected to the computer 64 % scanner 33 % video camera 9 % dictaphone 1 % computer 80 % tutorials and encyclopedias on cd or dvd 84 % digital camera 51 % programs for the preparation of presentations 64 % in addition, 70% of the respondents stated that they considered ict as a nice change in their teaching, 36% of the respondents as a necessity and 2% of the respondents as an unnecessary tool for their teaching. the findings also showed that the teachers were mostly stimulated to use ict by their family (50% of the respondents) or by the training paid by their employer (64% of the respondents). only 7% of the respondents were motivated by their study at a university and 17% of the respondents attended the ict course at their own expense. moreover, almost 86 % of the respondents reported that they developed all teaching materials (presentation, text files) themselves. 89% of the teachers used the internet to search for new materials. 90% of the teachers use materials bought by their school. interestingly, 41 % of the teachers did not use the materials developed by their colleagues 95% of the teachers use ict for communication with their colleagues, pupils and their parents. more than 60% of the respondents participated in some training on the use of ict. results of the standardized observation the findings of the standardized observation confirmed that the teachers actively used ict in their teaching. as far as the interaction is concerned, the teachers mostly organize and direct, while pupils mostly state their own opinions; it is pupils’ own activity and motivation – s/he raises his/her hand or freely enters discussion. statistical verification of the hypotheses hypothesis a null hypothesis: age (length of teaching experience) does not influence the frequency of the use of ict during classes. use of information and communication technologies in primary education / hlásná, klímová & poulová 687 alternative hypothesis: the frequency of the use of ict during classes depends on the length of teaching practice. the research sample was formed by the inexperienced teachers (length of the practice – 1 year) and by the experienced teachers (length of the practice up to 40 years). the respondents were divided according to their teaching practice into five groups (table 2). table 2. frequency of the use of ict on the basis of the teachers’ experience up to 6 years 6 – 12 years 13 -19 years 20 -27 years over 27 years total every lesson 5 4 11 9 3 32 once a day 8 4 7 12 11 42 three times a week 9 2 2 15 8 36 less often 3 3 4 9 6 25 total 25 13 24 45 28 135 independence hypothesis was tested by chi-square distribution. value 14.749 for the monitored data was determined. at the chosen significance level of 5%, the null hypothesis is not cast off, i.e., the frequency of the use of ict in education does not depend on the length of teaching experience. classic pearson's correlation coefficient is observed for two variables value of r = -0.096, therefore slightly negative. this would mean that the lower length of teaching experience (which usually means a lower age teacher), the higher the frequency of the use of ict. p-value for this test of the zero coefficients is 0.227, however, that the hypothesis that these two variables are independent cannot be dismissed. when used, agresti approach (agresti, 2002) is m = 1.248. monitored data includes 135 individuals, which is enough to use asymptotic distribution of this statistic. the calculated p-value equals to 0.264. at the 5% level the independence of the length of practice and frequency of the use of ict by teachers are not rejected. the findings thus show that the hypothesis has not been proved. hypothesis b null hypothesis: participation in an ict training does not have any impact on their use in teaching. alternative hypothesis: frequency of the use of ict in classes depends on the fact whether the teacher participated in the ict training or not. the respondents could choose from three areas of education in the field of ict.  sipvz (state information policy in education) – basic (b)  sipvz – basic (b) and advanced (a)  other (most of the respondents had some sort of training with interactive board or methodological training in the field of ict) in the first phase the respondents were divided into two groups on the basis of their participation in an ict course. table 3. frequency of the use of ict depending on passing or not passing the ict course no course course total every lesson 4 27 31 once a day 8 35 43 three times a week 2 34 36 less often 2 23 25 total 16 119 135 international electronic journal of elementary education, 9(3), 681-692, march, 2017 688 apart from the main hypothesis, the authors set three partial hypotheses: b1) null hypothesis: the teacher who participated in the basic training (sipvz – b), uses ict as often as the teacher who did not participate in any training. b2) null hypothesis: the teacher who participated in the basic and advanced training (sipvz – b and a) uses ict as often as the teacher who did not participate in any training. b3) null hypothesis: the teacher who participated in the basic and advanced training (sipvz – b and a) or in other training, uses ict in classes as often as the teacher who participated in the basic training (sipvz – b). all the hypotheses described above were processed in the same way as hypothesis a. tests of hypothesis b and all three partial hypotheses are presented in table 4 below. the chosen significance level was 5%. table 4. hypothesis b and its tests hypothesis b hypothesis b1 hypothesis b2 hypothesis b3 chi-square 3.631 3.449 1.443 10.425 p-value 0.304 0.327 0.695 0.015 yarnold’s criterium fulfilled fulfilled fulfilled fulfilled correlation coefficient r2 -0.100 -0.214 -0.066 0.255 p-value 0.248 0.131 0.540 0.005 m2 statistics 1.342 2.293 0.381 7.688 p-value 0.247 0.130 0.537 0.006 n 135 51 88 119 the main hypothesis about the frequency use of ict in classes is not influenced by the participation in the ict course cannot be rejected. however, the partial hypothesis b3 is right since applied tests rejected the null hypothesis. therefore the teacher who participated in the basic training only uses ict in his/her classes less often than the teacher who participated in other training. summary of the results the research showed that more than a half of the respondents used ict on a daily basis, mainly in the teaching of czech language and mathematics. the most exploited ict included the interactive board, the internet and tutorials. the teachers perceived ict as a nice change in their traditional classes and as an interesting activation and motivation for their pupils. only 2% of the respondents considered ict useless for their teaching. 36% of the respondents perceived ict a necessity due to the present pressure on information literacy. 98% of the teachers evaluate the use of ict in education positively. therefore it is surprising that they did not think that ict had a significant effect on the increase of the effectiveness of teaching. the survey also revealed that ict would negatively influence their attitude to pupils or among pupils themselves. the results also showed no impact of the length of the teaching practice on the use of ict in teaching. nevertheless, those who attended the methodological course on ict used ict in their classes more often than those who participated only in the basic or advanced ict training. discussion the aim of this research study was to discover teacher’s attitude towards the use of ict at the first stage of primary schools. the authors researched several variables: age, the length of the teaching practice, and education in the field of ict. the questionnaire survey revealed that the teachers were positive about the use of ict in their classrooms. they most often use the interactive boards, computers and cd players. use of information and communication technologies in primary education / hlásná, klímová & poulová 689 91% of the respondents use the internet in their work. during the lesson they exploit ict to broaden the knowledge and practise the study material (92% of the respondents), zounek & sedova (2009) in their study present that 41% of their respondents use ict for testing, 38% as a background and complement and 24% of the respondents use ict to explain the new study material. similar results have been confirmed by tezci (2009), who also argues that the teachers’ use of ict in classrooms is fairly limited. they mainly exploit the internet, e-mails, word processing and educational cds. sanchiz ruiz et al. (2011) say that the teacher should be able to use ict especially in the following areas of education: web pages, emails, videoconferences, telephone, chats, news, cell phone messages, and software. in fact, it is a must since students who are now the so-called digital natives want to interact (communicate, share, exchange), create, meet, coordinate, evaluate, learn, search, analyse, report, socialize and evolve differently (prensky, 2004). the authors also compared the frequency of the use of ict with the results of zouenk & sedova (2009). consult table 5 below: table 5. comparison of the findings of two research studies on the frequency of the use of ict in teaching frequency of the use of ict authors’ study zounek and sedova (2009) every hour 24 % 0 % once a day 33 % 12 % three times a week 27 % 52 % three-four times a months 18 % 31 % never 1 % 5 % the findings indicate that currently, the teachers use ict more frequently than it used to be in 2009. in addition, in comparison with zounek & sedova (2009), the findings of this study also show that present primary schools are much better equipped with ict thanks to the eu projects and therefore the frequency of the use of ict can increase. on the basis of the interviews, it can be said that the use of ict has changed teacher’s role as well. the teachers have become mediators, which means that they are a kind of organizers, moderators; they interfere into the interaction pupil – ict if a pupil faces a problem and needs the teacher’s help. some teachers also think that they are more partners to their pupils since the use of ict enables them to work together with their pupils and solve join problems or similar tasks. the findings also revealed that the teachers did not consider that the use of ict would have any impact on the relationship between the teacher and the pupil. however, they are a bit skeptical about the communication among the pupils themselves. they pointed out at the fact that ict enabled children to play with their help, which does not make them move much and thus, it has an adverse impact on their health. the questionnaire survey also showed that the most stimulating factor supporting the use of ict in classes is the training paid by the employer, followed by colleagues and then own children. table 6 below compares the authors’ findings with the findings of zounek & sedova (2009). table 6. comparison of the stimulating factors influencing the frequency of the use of ict in teaching stimulating factor authors’ study zounek and sedova (2009) partner 14 % 22 % own children 28 % 33 % relatives 8 % 4 % international electronic journal of elementary education, 9(3), 681-692, march, 2017 690 table 6 (cont.). comparison of the stimulating factors influencing the frequency of the use of ict in teaching stimulating factor authors’ study zounek and sedova (2009) friends 14 % 11 % colleagues 36 % 59 % training paid by the employer 64 % 56 % university study 7 % 9 % pupils 4 % 4 % training at one’s own expense 17 % 7 % the person alone 17 % 40 % table 6 illustrates that the teachers more often take an advantage of a possibility of further education in the field of ict although they have to pay for it themselves. the survey also confirmed that the teachers who participated in the ict training use ict in their teaching more than those who did not attend any course. this is also confirmed by manenova (2012; 2009) that education in the field of ict has an impact on the use of these technologies. in his research study tezci (2009) claims that the more experience and knowledge of ict the teachers have the more positive they are about their use in teaching. interestingly, there is a low percentage of the impact of the university study on the use of ict in teaching. therefore more research should be done on this issue. do future teachers get acquainted with ict as didactic tools during their university studies? do they pass any course on the use of ict in teaching during their university studies? the statistical analysis also did not prove the impact between the age (the length of the teaching practice) and the frequency of the use of ict. the same findings were also confirmed by manenova (2009), tezci (2009) or umar and yusoff (2014) who also found no correlation between the teachers’ years of service/experience and the use of ict. this can be connected with the fact that current novice teachers are much more digitally literate than it used to be 20 years ago. nevertheless, even these novice teachers lack a systematic, continuous, methodological training which would contribute to the improvement of effective use of ict in primary education. for example, the teachers should be advised on how to use ict to promote autonomous learning as well as collaboration among their pupils. as smeets (2005) claims, most teachers still do not make use of the potential of ict to contribute to the power of learning environments. computers are used mainly to complement rather than change the existing pedagogical practice. ersay and bozkurt (2015) suggest that teachers should reduce the prejudice about the use of ict and utilize the processes that increase and facilitate the learning. on the basis of these findings, the authors set the main pedagogical approaches to the use of ict in primary education, which should be followed. these include a collaborative use of ict (in pairs or small groups), but younger pupils may need some guidance in how to collaborate effectively and responsibly; a short but focused use of ict in classrooms (three times a week); use of ict as a supplement to traditional teaching; and support teachers in developing their use of digital technology to ensure it improves teaching and learning aims (cf. higgings et al., 2012). conclusion both theoretical and practical findings indicate that the teachers at primary schools flexibly react to the present changes in education which are connected with the use of ict in teaching. in fact, ict have become an inseparable part of their everyday educational practice. they offer teachers a number of new didactic tools of high quality. nevertheless, there should be more training on the effective use of ict in primary education which would ensure that the teachers have relevant competences for using them in their use of information and communication technologies in primary education / hlásná, klímová & poulová 691 classrooms. in addition, there should be more encouragement and support to increase teachers’ motivation to improve the level and quality of ict in their teaching (uluyol & sahin, 2016) since it is the teacher who by his/her approach and professionality decides about the benefits and their impact on the quality of teaching. as higgins et al. (2012) say, it is the pedagogy of the application of ict in the classroom which is important: the how rather than what. acknowledgements the paper is supported by the spev project 2017 at the faculty of informatics and management of the university of hradec kralove, czech republic. the authors thank the spev students for their help in the collection of data. conflict of interest the authors declare that they have no conflict of interest. • • • references agresti, a. (2002). categorical data analysis, second edition. new york: john wiley and sons. annual report of the czech school inspection. (2011). [vyrocni zprava ceske skolni inspekce za skolni rok 2010/2011.] praha: csi. annual report of the czech school inspection. (2012). [vyrocni zprava ceske skolni inspekce za skolni rok 2011/2012.] praha: csi. badia, a., meneses, j., sigales, c., & fabregues, s. (2014). factors affecting school teachers' perceptions of the instructional benefits of digital technology. procedia social and behavioral sciences, 141, 357-362. baytak, a., tarman, b., & ayas, c. (2011). experiencing technology integration in education: children’s perceptions. international electronic journal of elementary education, 3(2), 139-151. brdicka, b. (2003). the role of the internet in education. [role internetu ve vzdelavani.] kladno: aisis. chraska, m. (2007) methods of pedagogical research: basics of quantitative research. [metody pedagogickeho vyzkumu: zaklady kvantitativniho vyzkumu.] praha: grada. collins, b. et al. (1996). children and computers in school. new jersey: lawrence erlbaum associates. comped. (1989-1992). http://www.iea.nl/comped.html. accessed 5 october 2016. czech republic country report on ict in education. naep (2011). http://cms.eun.org/shared/data/pdf/czech_republic_report.pdf. accessed 5 october 2016. domingo, m. g, & gargante a. b. (2016). exploring the use of educational technology in primary education: teachers´ perception of mobile technology learning impacts and application´ use in the classroom. computers in human behavior, 56, 21-28. ersay, a., & bozkurt, m. (2015). understanding an elementary school teachers’ journey of using technology in the classroom from sand table to interactive whiteboard. international electronic journal of elementary education, 8(1), 1-20. flanders, n. a. (1970). analyzing teaching behavior. london, addison-wesley. international electronic journal of elementary education, 9(3), 681-692, march, 2017 692 frydrychova klimova, b. (2012). impact of ict on foreign language learning, awer procedia information technology and computer science, 2, 180-185 higgings, s., xiao, z.m., & katsipataki, m. (2012). the impact of digital technology on learning: a summary for the education endowment foundation. full report. uk, durham university. inan, f. a. y., & lowther, d. l. (2010). factors affecting technology integration in k-12 classrooms: a path model. educational technology research and development, 58(2), 137-154. kiper, a., & tercan, s. s. (2012). the usage of information technologies in classroom environment among primary school teachers and their perception on in-service training programs on it (sample of sakarya). tojet, 11(3), 386-392. manenova, m. et al. (2009). ict and the teacher of the first stage of basic school. [ict a ucitel 1. stupne zakladni skoly.] brno, computer press. manenova, m. (2012). the impact of ict on teacher’s job at the first stage of primary schools. [vliv ict na praci ucitele 1. stupne zakladnich skol.] praha, extrasystem. murphy, k. l., depasquale, r., & mcnamara, e. (2003) meaningful connections. using technology in primary classrooms. young children, 58(6), 12-18. prensky, m. (2004). the emerging online life of the digital native: what they do differently because of technology and how they do it. http://www.bu.edu/ssw/files/pdf/prenskythe_emerging_online_life_of_the_digital_native-033.pdf. accessed 5 october 2016. rvpzv. (2007). http://www.vuppraha.cz/wp-content/uploads/2009/12/rvpzv_2007-07.pdf. accessed 5 october 2016. sites. (2006). http://www.iea.nl/sites_2006.html. accessed 5 october 2016. steps. (2007). http://eacea.ec.europa.eu/ llp/studies/ documents/ study_ impact_ technology_ primary_school/ 01_executive_summary_steps_en.pdf. accessed 5 october 2016. smeets, e. (2005). does ict contribute to powerful learning environments in primary education? computers & education, 44, 343-355. svatos, t. (2009). new technologies in education. [nove technologie ve vzdelavani.] in prucha, j. (ed.) pedagogicka encyklopedie. praha, portal. svatos, t., & dolezalova, j. (2011). pedagogical interaction and communication with respect to the system of categories. [pedagogicka interakce a komunikace pohledem vyvoje kategorialniho systemu.] hradec kralove: gaudeamus. tezci, e. (2009). teachers’ effect on ict use in education: the turkey sample. procedia social and behavioral sciences, 1, 1285-1294. uluyol, c., & sahin, s. (2016). elementary school teachers’ ict use in the classroom and their motivators for using ict. bjet, 47(1), 65-75. umar, i. n., & yusoff, m. t. m. (2014) a study on malaysian teachers’ level of ict skills and practices, and its impact on teaching and learning. procedia social and behavioral sciences, 116, 979-984. van braak, j., tondeur, j. y., & valcke, m. (2004). explaining different types of computer use among primary school teachers. european journal of psychology of education, 19(4), 407-422. zounek, j., & sedova, k. (2009). teachers and technologies. [učitelé a technologie.] brno, paido. students’ performance calibration in a basketball dibbling task in elementary physical education international electronic journal of elementary education, 2015, 7(2) v dear iejee readers, international electronic journal of elementary education (iejee) offers you a new issue with eleven articles. twenty three researchers from different research institutions address and explore different educational issues. it is iejee’s first time that it comes out with a so expanded issue, a new face and advanced technological solutions, but also with a new policy. there are several factors that caused for this expansion and other changes at iejee. factor 1: iejee is getting more and more attention from database and indexing agencies. the good news is that eric started to index iejee. eric is an education resources information center (eric). it is an online digital library of education research and information. eric is sponsored by the institute of education sciences of the united states department of education. the mission of eric is to provide a comprehensive, easy-to-use and searchable database. we are proud of being a part of eric’s database. therefore we decided to improve our digital solutions and make indexing and searching easier for our users. factor 2: iejee is receiving more and more attention and thus more submissions from researchers from all over the world. many accepted papers had to wait due to technical constrains. now iejee has developed a new face, an opening page and an easy to use type of menu. we upgraded our technical solutions and expanded our publication capacity. the aim is to provide our researchers and readers a better and effective scientific service. factor 3: all the mentioned increased interest for iejee made it necessary to increase the number of our annual issues. starting from this year, 2015, iejee will come out with three ordinary issues (march, june and september) and a special issue in december. improving technology, increasing capacity, and continuous maintenance and upgrading need funding. all these factors made it necessary to ask for a symbolic publication fee for the accepted papers. we hope researchers will show us understanding. i would like to express my deep gratitude to dr. turan temur; dr. gökhan özsoy, dr. hayriye gül kuruyer, doctoral candidate hasan tabak, doctoral canditate mustafa bakir, ulaş yabanova and to all our contributors in this issue. sincerely, editors in chief dr. kamil özerk, university of oslo, norway international electronic journal of elementary education, 2015, 8(2), 199-210 issn:1307-9298 copyright © iejee www.iejee.com the transition year: a unique programme in irish education bridging the gap between school and the workplace joseph a. moynihan  university college cork, ireland abstract transition year is a unique and exciting programme situated in the middle of the six year second level education system in ireland. since its introduction in 1974, the programme has experienced unprecedented growth now being offered in over 80% of schools on the island. transition year seeks to emphasize alternative learning methodologies including self-directed learning and experiential learning; intellectual, social and personal development; community and inter-school communications; exposure to careers education and the world of work. the latter occurs predominantly through actual work experience. this paper situates and explains the transition year programme in irish education as well as outlining the significance of the essential work experience component. transition year work experience is the bridge that connects young people in the classroom with adult life and the world of work. keywords: transition year, experiential learning, work experience, socio-cultural learning, career path. introduction the transition year programme (typ), also known simply as transition year (ty) is truly a unique feature in irish education. based on its significant success over a 40-year period in ireland, it is surprising that currently, there does not appear to be a programme that compares adequately with it anywhere else in the world. the typ is “pioneering in an international context with no comparable educational intervention existing in similarly standardised systems” (smyth , byrne and hannan, 2004, p. 7). these authors make reference to the “seconde” in france which does have some similarities, as an orientation year, to the typ. they add though, that the french system differs considerably due to nationally prescribed coursework in core academic subjects as well as national evaluation in some subject fields. the irish department of education (de) describes the typ as offering pupils  joseph a. moynihan, school of education, university college cork, ireland. phone: +353 87 2397468. e-mail: joseph.moynihan@ucc.ie or jagmoy@gmail.com http://www.iejee.com/ international electronic journal of elementary education vol.8, issue2, 199-210, 2015 200 … a broad educational experience with a view to the attainment of increased maturity, before proceeding to further study and/or vocational preparation. it provides a bridge to help pupils make the transition from a highly-structured environment to one where they will take greater responsibility for their own learning and decision making (de, 1993c, p. 1) ireland has a long and noble tradition of respect and concern for education (coolahan, 2009). this, coolahan adds, is reflected in the monastic schools, which were seen as the “lights of the north” (p. 8) during the dark ages of europe. o’buachalla (1988) describes the developments in irish education through the 19th century as remarkable particularly when set against the backdrop of economic deprivation and deep political unrest of the times. educational innovation has been part of irish education throughout its long and vibrant history. the irish education system was traditionally divided into three distinct sectors: 1) primary education (8 years, age 5-12) 2) secondary education (5 or 6 years, age 12-18) 3) third level education (age 17+) in the latter decades of the twentieth century and the early twenty first century, the system expanded to include:  pre-school education (age 3-5)  adult and further education these additions came about as the concept of life-long learning began to gain momentum and the de tried to meet the demands of a population that was becoming more academically aware and intent on responding to market demands in a rapidly growing knowledge economy. second level education in ireland caters for students from around age 12 years to 18 years. students commence year one at the age of 12/13 years. the next three years are spent pursuing a programme which ultimately prepares the students for their initial state examination, namely the junior certificate. on completion of the junior certificate examination students can choose one of the following routes: a) participate in the transition year programme (a one year programme in year four of a six year cycle) b) advance to fifth year and begin the terminal leaving certificate programme including its variants leaving certificate applied (lca) and leaving certificate vocational programme (lcvp). c) leave school, by law having reached the age of 16 years, the current legal minimum age to exit compulsory education in ireland. ever since its introduction over 40 years ago, an increasing number of students are choosing the ty route. the transition year programme the typ first appeared in 1974 when it was offered in just three schools as a pilot programme. it experienced its greatest expansion throughout the 1990s (smyth, byrne & hannan, 2004). its introduction was brought about by the recognition of the irish de, at that time, of the overly academic nature of the senior cycle. the then minister for education, richard burke, summed it up when he said: because of growing pressures on students for high grades and competitive success, educational systems are becoming increasingly academic treadmills. increasingly, too, because of these pressures the school is losing contact with life outside and the student the transition year / moynihan 201 has little or no opportunity “to stand and stare”, to discover the kind of person he is, the kind of society he will be living in and, in due course, contributing to, its shortcomings and its good points. the suggestion was made that perhaps somewhere in the middle of the course we might stop the treadmill and release the students from the educational pressures for one year so that they could devote time to personal development and community service (burke, 1974). the transition year can be seen as transitional in two specific ways: 1) the transition between school and the world of work for those who were ready or wanted to leave school. 2) the transition between the junior cycle and the senior cycle for those who wished to remain in second level education. today, the transition year programme involves an impressive number of students across ireland (table 1). the table clearly shows how the programme has gathered serious momentum and has become a significant feature on the irish educational landscape. with over 600 schools now offering the programme, it continues to grow and would appear to have a bright future. the programme continued to attract students right through the economic crash of the 1980s and the unrelenting recession experienced in ireland and many other nations since 2007. despite direct financial cuts to the programme as a result of restricted de spending during the economic downturn, the typ continued to develop and grow. this was largely due to the commitment of schools, teachers, students and parents to a programme that has a truly astounding track record of success. table 1. number of schools and students in transition year 2001-2013 school year no. of schools no. of students 2001/2002 500 22,773 2002/2003 510 23,299 2003/2004 526 23,772 2004/2005 522 24,798 2005/2006 528 25,862 2006/2007 524 27,090 2007/2008 546 27,760 2008/2009 544 27,800 2009/2010 560 28,635 2010/2011 574 30,536 2011/2012 589 32.673 2012/2013 601 data source: department of education & skills, database section, december 2012 students find themselves immersed in an alternative learning environment where the focus is no longer on rote learning but on more socio-cultural and experiential learning styles. young people in ty have the opportunity to participate in various mutual arrangements where they have much to learn from experienced others, but they too, have much to share with their older counterparts. they are coming from an intensive education setting. in most cases, today’s students are quite comfortable with technology and innovation. this new way of learning for them does not just apply to work experience, but also to the many other tasks and challenges that they will encounter during the ty experience. international electronic journal of elementary education vol.8, issue2, 199-210, 2015 202 the actual mission of transition year is “to promote the personal, social, educational and vocational development of pupils and to prepare them for their role as autonomous, participative, and responsible members of society” (de, 1993 p. 3). the overall aims of the typ are to educate young people for maturity, promote general, technical and academic skills and support learning through the experience of adult and working life (de, 1993). there was a clear desire on the part of the de to bridge the gap between school and working life. transition year curriculum in terms of curriculum, the de offers ample suggestions with an open invitation for each school to take the lead in designing its own, based on the needs of its student population. the teaching and learning methodologies proposed are varied, exciting and very achievable in any school with a well-structured programme and a sound core team in place. the assessment of the programme is intended to be diagnostic and formative with the opportunity for the learner to participate in the procedure via dialogue with the tutor and, in some instances, self-rating. the de suggests that this should lead to greater selfawareness and an increased potential to take responsibility for both performance and personal learning. a whole school approach is highly recommended along with the necessity for clear co-ordination and teamwork on the part of the staff charged with the responsibility of ty. the de has produced very detailed guidelines to facilitate this process within schools. any written programme devised by a school is based on four key layers: 1) the core subjects layer 2) the subject sampling layer 3) the ty specific layer 4) the calendar layer schools have the freedom to decide what specific subjects to offer and what modules to develop depending on the skills of the teaching staff, the general resources of the school and the access it has to outside expertise in the community. the ty specific layer and the calendar layer are subject to some extent to the level of funding available in each school. while schools offering the typ receive an annual per capita payment from the department of education and skills, the vast majority of schools also charge a ty participation fee to offset the additional cost of engaging specialist speakers and presenters as well as organising off campus events and activities. the following chart outlines the possible content of each specific layer: the transition year / moynihan 203 (based on professional development for teachers (pdst) website, 2015) ty and work experience a key constituent of twenty first century schooling in ireland, and in many other countries across the globe, is the availability of work experience schemes to students who attend formal second level education. such schemes evolved throughout the twentieth century and now have become quite sophisticated programmes which, in ireland, include career investigation programmes, work shadowing programmes and work experience placements. the work experience programme (wep) is a fundamental component of ty. work experience has been described as a paradoxical phrase in that it is clearly distinguished from actual work and is used to describe schemes where students are only exposed to part of the full experience of work (miller, watts & jamieson, 1991). these authors add that a key difference is that the role of the student on work experience is that of a learner and not of an employee. miller et al. (1991) state that a further paradox is that the learning yield will be “substantial and distinctive” (p. 16) only if the experience gets the student as close as possible to actually being an employee. the key objectives of ty for students participating in a work experience programme are to: figure 1. suggested areas of study international electronic journal of elementary education vol.8, issue2, 199-210, 2015 204  link the core academic curriculum with the world of work  promote the school-to-career transition for students  increase students’ confidence levels  assist students in the development of attitudes, skills and habits conducive to job success and personal development  increase student understanding of enterprise and working life  use the local business community as an incubator to help students develop a positive work ethic  help students to make better informed decisions about their future  help students acquire, or indeed refine, work related skills and job performance in a real work environment. work experience in ireland presents young people with “engaging and relevant learning opportunities” (jeffers, 2006, p. 409). jeffers comments that following the restructuring of senior cycle education in ireland in the 1990s, preparing young people for adult and working life became a more explicit goal within the new programmes developed. work experience is an important part of ty, offering students personal growth, maturity and preparation for adult and working life (smyth et al., 2004). in the 2011 longitudinal study entitled from leaving cert to leaving school, work experience was highlighted as a very valuable component of the school lives of irish adolescents. in this study, 6th year students “highlighted the need for greater preparation for adult and working life, suggesting that a life skills course and access to work experience for all students [emphasis added] would enhance their overall educational development” (smyth, banks & calvert, 2011, p. 192). students generally spend an average of 15 days on work experience, with most schools involved in the programme usually offering a minimum of five days. a minority of schools offer three or even four weeks in the workplace. the department of education evaluation of the transition year 1994-1995 (de, 1996) points out that the variation in duration seems to be influenced, in some instances, by the ethos and traditions of schools as well as the long term intentions of the students: “schools whose curricula had traditionally an academic orientation having shorter periods with those of a more vocational bent opting for longer periods” (p.15). the report claims that pupils were found to be well prepared for their work experience placements. the three key partners in this preparation were usually the ty coordinator, the guidance counsellor and the potential employers. employers were found to be co-operative and very willing to provide the schools with feedback on completion of the placement (de, 1996). preparation of students for work experience varies from school to school. some schools have no formal training programmes while others have work experience preparation time-tabled on a weekly basis. the time allocated for that preparation is subsequently used for monitoring while students are on placement, and de-briefing when students return to school. ty and socio cultural learning experiential learning, as defined by any of the major theorists in this field (kolb, 1984; dewey, 1991; mcelhaney, 1999) effectively underpins the transition year programme. contemporary conceptualisations of socio-cultural learning theories draw heavily on the work of russian psychologist lev vygotsky who believed that social events have learning embedded within them (vygotsky, 1978). this is a fundamental concept underpinning the work experience programme in ty. vygotsky proposes that what a child can learn with the assistance of others (social learning) is more indicative of his/her mental development than what s/he can learn alone. while he acknowledged the role of the teacher as vital to a child’s learning, he also saw the classroom itself as a social setting and quite the transition year / moynihan 205 representative of larger society. the department of education (de) saw ty as a means of varying the learning environment and “helping to dispel the notion that learning is something that happens only, or most effectively, within the classroom” (de 1993b, p. 3). they perceived work experience as one method of achieving this. ty has many other examples of experiential and socio-cultural learning in the form of young entrepreneur schemes (yes) where students set up and run small companies; school musicals where students take on all sorts of roles from acting and singing to design, production, and direction; social innovation programmes where students work with others in the community dealing with social issues; foreign travel offering students an opportunity to interact with other cultures and in some cases total immersion to develop their language skills. an article by barbara scully (2015) in the irish independent recently described ty as a “bright light” (p. 31) in irish second level education. this writer offers a contemporary description of ty saying it “embodies what education should be an opening of minds, an exploration of our world, a pause in which to think and debate and sing and climb and challenge” (scully, 2015, p. 31). ty, as the name suggests, is a year of transition. perhaps a further conceptualisation of this could be to see ty as a vehicle for the transition of irish classrooms from traditional centres of learning to active centres of learning, as suggested by the oecd (1991). transition year, with its focus on active learning and experiential learning, combines the best of both these worlds in terms of how students learn. they have the benefit of the teacher in the classroom while, at the same time, having substantial access to the many more experienced or more knowledgeable others within the school or the wider community. the work experience programme gives students the opportunity to access the world of work and expose them to adult and working life. moving from the safe and familiar school environment into the world of work can be a daunting experience for students. whatever the school can do to facilitate this and make the transition as comfortable as possible for the student can be crucial to the success of the programme. the change and development in young people that can arise out of the move from one learning context to another, in this instance from the school to the workplace, is an important outcome of any work experience programme. engeström (1996, as cited in guile & young, 2003), describes this “boundary-crossing” as a form of “horizontal development” (p. 70). by boundary-crossing he means “learning how to cross the social and cultural borders between different activity systems” (p. 70). the idea of horizontal development was a significant advance in socio-cultural theory contrasting with the traditional concept of vertical development, which was more concerned with intellectual development. each person’s growth was based on navigating a hierarchical body of knowledge and skills which gick (1995, as cited in guile & griffiths, 2001) described as the cornerstone of most cognitive development theories. vertical and horizontal learning tend to be treated in isolation from one another which, really is a reflection of the institutional separation of formal learning from informal learning (guile & griffiths, 2001). such separation had been “a feature of most academic communities for many years” (bourdieu, 1988, as cited in guile & griffiths, 2001, p. 3). engeström argues that learning how to cross social and cultural borders is one of the main challenges faced by people and groups in modern society. ty offers an appropriate and safe space where students can encounter and face up to such a challenge. a well organized and successful placement can be an immersive experience, a metaphor boud (2008) defines, in the positive sense, as “being deeply absorbed or engaged in a situation or problem that results in mastery of a complex and demanding situation” (p. 1). while every student may not necessarily experience mastery of the work undertaken by them during their placement, there is still, nonetheless, significant learning involved. when transition year students enter the workplace, they engage in a new international electronic journal of elementary education vol.8, issue2, 199-210, 2015 206 community of practice, a term developed by wenger and lave (1991), describing the social learning process that takes place when people are actively involved in a common pursuit. interaction is the key to a true community of practice where individuals build relationships and learn from one another (wenger, 1999). such interaction is reflected throughout much of transition year as well as specifically during work placements. this same practice is now set to play a significant role in the new junior certificate and revised leaving certificate programmes currently being developed by the irish department of education and skills (des). such developments in education indicate that the next major transformation in irish education may well be the transition from traditional individual learning to a more participative, interactive process where students will play a more active role in a more experiential learning environment. boud (2008) who prefers the term experience-based learning over experiential learning posits that “all learning needs to be seen as coming from experience” (p. 2). he argues that learning can only take place through engagement by the learner with whatever activity is being conducted inside or outside of the classroom. however, true experience-based learning, he adds, is concerned with instances where learning activities are arranged with the clear goal of substantial engagement with the learner’s actual experience and where the learner occupies the central place in all aspects of teaching and learning. transition year students have many opportunities to experience this through general classroom activities specific to a well written and well implemented ty programme, along with other activities such as the aforementioned yes, musicals and community and socially based innovation. more specifically, work based learning (wbl) occurs through participation in the work experience programme. raelin (2000) argues that there are clear distinctions between traditional classroom learning and wbl. a significant example of this is that wbl should be centred around reflection on the work practices experienced. primarily, he stresses the importance of reviewing and learning from what has actually been experienced by the participant. boud (2008) too attaches major importance to reflection portraying debriefing and reflective thought as crucial to the process. ty work experience and possible career choices moynihan (2013) conducted a mixed methods study of work experience in ty and its effect on subject and career choices. the research was carried out across three schools involving 323 students. the schools consisted of a single sex male school, a single sex female school, and a co-educational school. the quantitative element of the study showed students’ attitudes to work experience to be very positive. the general feeling among students was that the time spent on work experience was not only very worthwhile, but actually a highlight of the entire ty programme, with females shown to be more likely than males to believe this. those who had existing career ideas, especially the female students, agreed that their career choices had been reinforced by the opportunity to participate in adult life and the world of work. this proved to be particularly true of those who had ideas about their future careers for a long time and even dating back to early childhood. the very narrow margin of students who believed that work experience was a waste of time for them, were predominantly those who expressed low self-esteem, had poor academic performance records, and whose placement choices were either poor or careless. the majority of students however, did make careful and valuable selections with those in the former category openly admitting that they should have been more vigilant in their selection. while the majority of students either had their career choices confirmed or found new career directions, a significant minority were put off their career idea. this was more applicable to boys than girls. in general, most students agreed that the work experience process, at the very least, crystallized their career ideas for the future. the transition year / moynihan 207 the analysis of the qualitative interview data generally supported the findings from the quantitative analysis. again, it showed students to be very positive about the work experience element of the ty programme. even those who believed that they had made poor selections agreed that there was significant learning there for them nonetheless. many students talked about being put off a particular career while the majority believed that they either had their career idea confirmed or that they were switched on to a new career goal. boys were shown to be more likely than girls to actually be put off a career but this is not that surprising when one considers that boys proved to be more careless in their placement selections in the first instance. a significant number of students, especially girls, had career ideas dating back to quite a young age. super (1957) and ginzberg (1966) refer to the time when this occurs as the fantasy stage in their career development theories. the interview results highlighted a more significant number of students claiming a link between their subject choices and work experience. this time it proved to be a majority of students who agreed there was a definite influence. the figure was highest in the co-educational school with a significant majority of those interviewed identifying a link. another interesting development from the qualitative interviews was the extent to which students agreed that, subsequent to work experience, they had an increased awareness of the importance of study in the following years. students, right up to their terminal year in school, still agreed that their transition year work experience caused them to work harder in school thereafter. students responded very positively to questions on their experience of adult life and the world of work. they expressed a strong sense of satisfaction and excitement when talking about the experience. they contrasted the independence and freedom of working life with the overly controlled environment of the classroom. they used words like restricted to describe learning in school while words like insightful, responsible and enjoyable, were associated with work-based learning. similarly, positive words like confident, mature and independent were used when students spoke about themselves and how they were personally affected by their time in the world beyond the school walls. they were clearly aware of the development within themselves as young adults and the ever-changing identity that defines them. overall, students were overwhelmingly in support of the work experience programme and the transition year as a whole. the vast majority agreed that age 15 or 16 was an ideal time to develop some sort of career goal. they also saw significant value in discovering that a career was not right for them. when it came to advising others who are about to embark on a work experience programme, they were most willing to give sound advice with the very strong recommendation to choose something in which one is genuinely interested. they openly cautioned against convenience and careless placements. conclusion transition year is a widely accepted success in irish schools, with much research supporting the predominantly positive attitudes towards the programme from all of the education partners. it is still difficult to comprehend that, based on its phenomenal success in ireland, a transition year model, or similar programme, has not appeared in any other education systems. work experience is evidenced throughout the research of moynihan (2013) as a powerful programme within the overall ty experience. that research has shown a clear and important link between participation in the work experience programme and the subjects chosen thereafter by students for their senior cycle education as well as their potential career path. this contributes to previous studies on subject choice and decision making theories. while there are many factors that influence subject choice, work experience has been proven to be one worthy of serious consideration. it is a factor that has a knock-on or domino effect. the subject combinations fashioned by international electronic journal of elementary education vol.8, issue2, 199-210, 2015 208 students in the senior cycle inevitably lead them in a specific direction in terms of further study and a possible career. slaney (1988) claimed that most college students already have expressed vocational interests. decisions that carry that much weight need to be well thought out and this is where the education system, along with its various actors, can seize the opportunity to exercise their significant role in this process. students participating in work experience need to be provided with adequate advance preparation, well planned placements and efficient monitoring. post placement, they also require appropriate guidance in terms of debriefing and reflection. most career development and occupational choice theories are relevant in part or in whole to how students develop their career ideas and aspirations. most researchers agree that the study of career theory started with trait and factor theory. this was built upon by a host of theorists working in the field. lent, brown and hackett’s social cognitive career theory (scct) serves to bring together the various threads of career theory. it incorporates the evolutionary process dealt with in detail by ginzberg (1966) and super (1957) while having much in common with the work of krumboltz (1976), which like scct, is fundamentally based on bandura’s social cognitive theory. in summary, brown et al. advocate that beginning in early childhood, and extending through adulthood, people narrow their scope with a view to forming or pinpointing a career goal or, indeed, a career choice. they add that the likelihood of success can reinforce career choices. this was a significant outcome for students who participated in work experience (moynihan, 2013). the vast majority of those who participated in that research had their career goals reinforced if their placement proved to be an agreeable experience. where barriers are perceived, according to lent et al., then the interest and choice actions are much weaker. some of the barriers perceived by a minority of students in moynihan (2013), were things like students’ inability to attain grades, not liking the work, the difficulty of the work, low self-esteem and lack of confidence in their ability to pursue a third level qualification. moynihan (2013) also demonstrated the significant effect of work experience, and the broader ty programme, on students developing identities and their emerging vocational identities. holland’s career typology (1959) illustrates a strong relationship between personality and the occupational environment. this suggests that one’s identity, ultimately, has a significant effect on the selection of a suitable work environment. transition year assists with maturation and the development of personal identity. vocational identities emerge or are honed by participation in the work experience programme. through this combination, a career path, leading to the development of a potentially successful future self, is laid. in sum, the ty work experience programme influences senior cycle subject choices by students. it helps students fashion a clear focus on their future career possibilities while in some cases causing students to generate entirely new career paths or directions. work experience contributes to improved academic performance. a significant number of students, who participate in a work experience programme while still at school, can have existing career ideas reinforced, including ideas they may have held since childhood, or even early childhood. students experience personal growth during the ty experience with work experience contributing to the development of their confidence and their selfesteem. following on from this, students are helped with the clarification and the further development of their gradually emerging vocational identities. all of this contributes to their developing overall identity. finally, and of great significance, young learners are exposed to alternative ways of learning both within the school community and outside in the adult world of work. ty and its crucial work experience component form a gateway to effective subject choices and a pathway to a positive career trajectory. the transition year / moynihan 209 • • • joseph a. moynihan is currently lecturing in the school of education at university college cork, ireland. formerly, he was a teacher of french, geography and art, craft & design in second level education for 20 years. this was followed by four years working as a guidance counsellor prior to working as a deputy principal and principal for eight years. his research interests are in socio-cultural learning, work based learning and education in general. references boud, d. (2008). locating immersive experience in experiential learning. [online ebook]. retrieved from http://learningtobeprofessional.pbworks.com/w/page/21763862/david %20baud burke, r (1974) address to the annual conference of the teachers union of ireland. 17 april, 1974. in j. eggleston (ed.) work experience in secondary schools. london: routledge & kegan paul. coolahan, j. (1981). irish education: its history and structure. dublin: institute of public administration. department of education (1993b). circular letter m47/93. department of education (1993c) transition year programme: guidelines for schools. dublin: department of education. department of education (1996). transition year programme 1994-1995 an evaluation by the inspectorate of the department of education. dublin: department of education. dewey, j. (1991). how we think. buffalo, ny: prometheus books. ginzberg, e., ginsburg, s.w., axelrad, s., & herma, j.l. (1966). occupational choice. new york: columbia university press. guile, d., & young, m. (2003). transfer and transition in vocational education: some theoretical considerations. in t. tuomi-grohn, & y. engestrom (eds.) between school and work: new perspectives on transfer and boundary-crossing. oxford: elsevier science. guile, d., & griffiths, t. (2001). learning through work experience. london: university of london. jeffers, g. (2006). school guidance counsellors’ learning through work experience placements. reflective practice, 2, 3, 409-422. kolb, d. a. (1984). experiential learning: experience as the source of learning and development. new jersey: prentice-hall. krumboltz, j.d., mitchell, a.m. & jones, g.b. (1976) a social learning theory of career selection in the counselling psychologist, 6, 1, pp71-81. holland, j. l. (1985). making vocational choices: a theory of vocational personalities and work environments (2nd edition). englewood cliffs, nj: prentice hall. mcelhaney, k.a. (1999). student outcomes of community service learning: a comparative analysis of curriculum-based and non curriculum-based alternative spring break programs. michigan: unpublished dissertation. miller, a., watts, a.g. & jamieson, i., (1991). rethinking work experience. london: the falmer press. international electronic journal of elementary education vol.8, issue2, 199-210, 2015 210 moynihan, j. (2013). work experience in transition year: a mixed methods study of subject and career choices. (unpublished doctoral thesis). university college cork. pdst (2015). ty curriculum. [online webpage] retrieved from http://cmsnew.pdst.ie/ty/curriculum o’buachalla, s. (1988). education policy in twentieth century ireland. dublin: wolfhound press. oecd (1991). review of national policies for education, ireland. paris: oecd. raelin, j.a. (2000) work-based learning: the new frontier of management development new jersey: prentice hall. scully, b. (2015, may 6). our rigid education system is an unfair points game largely unchanged since '70s. the irish independent, p. 31. slaney, r.b. (1988). the assessment of career decision making. in w. b. walsh & s. h. osipow (eds.), career decision making. new jersey: lawrence erlbaum associates. smyth, e., banks, j., & calvert, e. (2011). from leaving cert to leaving school: a longitudinal study of sixth year students. dublin: the liffey press. smyth, e., byrne, d. & hannan, c. (2004). the transition year programme: an assessment. dublin: liffey press. super, d. e. (1957). the psychology of careers. new york: harper and row. vygotsky, l.s. (1978). mind in society. cambridge, ma: harvard university press. wenger, e. & lave, j. (1991). situated learning: legitimate peripheral participation. cambridge: cambridge university press. wenger, e. (1999). communities of practice: learning, meaning, and identity. cambridge: cambridge university press. students’ performance calibration in a basketball dibbling task in elementary physical education international electronic journal of elementary education, 2013, 5(2), 143-156. issn:1307-9298 copyright © iejee www.iejee.com questioning the validity of a state reading list gina berridge  university of southern indiana, evansville, indiana, usa jeff thomas university of southern indiana, evansville, indiana, usa received: 11 december 2012 / revised: 20 february 2013 / accepted: 5 march 2013 abstract quality incorporation of children’s literature has long been supported to help students learn. the purpose of this study was to examine the 79 fiction selections of our state’s sponsored reading list for grades k-2. the list was examined for original publication date, selections recognized as award winners, sex of authors, sex of protagonist, race of protagonists, and the presence of gender or racial bias. results indicate the sponsored reading list is out of date, needs to contain more award winning selections, and fails to represent a balance of women authors and selections containing female protagonists. gender bias was minimal and no presence of racial bias was observed. keywords: children’s literature, primary students, sponsored curricula, bias, reading. introduction in humankind’s earliest days people shared stories by word of mouth with children to pass along knowledge and values and also for mere entertainment. in time, printed characters were used. later, after the invention of the printing press books were printed and read as a way to communicate with children and capture their imaginations. aside from the recent advances of technology and the impact of the internet upon children learning to read (called the new literacies), books are still a significant way we pass along the knowledge and values among generations. educators know that books excite children and prompt them to imagine possibilities, ask questions, and enhance or teach literacy skills. the importance of quality children’s books in today’s classroom is critical for student learning and motivation to read. selecting quality literature for the classroom is a challenging task. resources that help facilitate this process should be reliable, updated and readily available to educators, librarians and parents. the purpose of this study was to read and evaluate the 79 fiction books on the suggested indiana reading list (irl) for grades k-2 for teachers and parents. the importance of children’s books in the primary setting  gina berridge, university of southern indiana, 8600 university boulevard, evansville, in, usa, phone: 812.465.7043. e-mail: ggberidge@usi.edu http://www.iejee.com/ international electronic journal of elementary education vol.5, issue 2, 143-156, 2013 144 good quality children’s literature inspires, motivates and teaches. for this reason, most if not all, primary teachers read fiction books to explicitly teach and model the scientifically based reading elements of phonemic awareness, phonics, fluency, vocabulary and comprehension. norton (2011) reports that teachers play a major role in developing a love for reading books in the lives of their students. selecting quality literature for the primary classroom is essential to engage students. this is no easy task since there are 5000 new books published for young children in the united states each year (temple, martinez, & yokota, 2006). criteria for selecting children’s books selecting from the many new publications is daunting since teachers must attempt to include representations from many cultures and peoples that view their cultures and themselves as equal members of society (tunnel & jacobs, 2008). maintaining a balance of male and female protagonists in book selections is important to help students understand each other and learn about the perspectives, problems, and feeling of the opposite sex (lynch-brown, tomlinson, & short, 2011). quality children’s literature is also gender sensitive showing the multidimensional roles of men and women in our society (temple et al., 2006). selecting books for primary age children is key to helping children foster an increased sense of self, to develop fundamental reading skills, and a love of reading. as a source for classroom selections, “teachers and librarians often rely on the professional judgment of committees that choose what they consider to be the most outstanding picture books published each year in this country and aboard” (lynch-brown & tomlinson, 2005, p. 77). reliable resources for book selections include award-winning titles that honor authors and illustrators whose work is judged by experts to be the best. according to tunnell, jacobs, young, & bryan (2012) there are over 200 different awards and prizes for children’s books presented by a variety of organizations in the united states. there are credible awards bestowed by professional organizations. two noteworthy examples include the international reading association’s award for new authors and the national council of teachers of english excellence in poetry for children award. however, for this study, the researchers focused on the five awards given by the american library association (ala). for the purposes of this study, the authors chose the following five awards given to specific picture books that best fit the study. these awards are sponsored and administered by the association for library service to children, which is an arm of the ala. these awards are appropriate for the study because: a) the ala is a recognized leader in recommending quality children’s literature among the variables used in the study; b) the awards are seen as prominent awards in their areas of focus; and c) the awards are based upon the quality of the book as defined by the awards’ criteria.  caldecott medal. an award given to a united illustrator for the most distinguished picture book for children published in the previous year (established 1938).  coretta scott king award for writing. an award given to an african american author for the outstanding inspirational and educational contribution to literature and children published in the previous year (established 1970).  coretta scott king award for illustration. an award given to an african american illustrator for the outstanding inspirational and educational contribution to literature and children published in the previous year (established 1974).  mildred l. batchelder award. an award given to a united states publisher for the most distinguished translated work for children published in the previous year (established 1968). questioning the validity of a state reading list / berridge & thomas 145  pura belpré awards for writing and illustration. an award given to a latino/a writer and illustrator whose work best portrays, affirms, and celebrates the latino/a culture experience literature for youth published in the previous year (established 1996) (lynch brown, 2011). using children’s literature in an evolving society. as our culturally diverse, elementary classrooms armed with netbooks, promethean boards, and ipads instruct our children in a global world, authors of children’s books are addressing audiences very differently from those even as fewer than five years ago (lukens, smith, & coffel, 2013). as david elkind, author of the hurried child suggests, “children today experience time, space, social relationships-even reality itself-differently from the ways earlier generations did” (as cited in lukens, et al., 2013, p.15). eliza t. dresang in her book, radical change: books for youth in a digital age, points out that more recent books respond to children in our digital age in three ways (as cited in lukens, et al., 2013). first, she notes that more recent books provide children with connections to the world by linking them to an expanding community. second, dresang asserts that more recent books provide the reader more variability in subject matter and type of book. finally, she claims that today’s children have more access to subjects and styles of language than did previous generations of children. examples of such books are genre blending, wordless picture books, and graphic and verse novels. lukens, et al. (2013) also addresses the changing boundaries of children’s books. topics are now addressed that were formerly off-limits for children’s books. picture books that include topics about homosexuality, divorce, illness, and disabilities are among the more recent books being published. though there remains continuing dispute as to the suitability of these books for the elementary classroom, but many view their inclusion as important ways to academically connect to all learners. cultural diversity in children’s literature. according to temple et al. (2006), quality children’s books involve students emotionally. “schools can be instrumental in providing opportunities for students to read and discuss from multiple viewpoints” (p. 90). they emphasize that selections of quality literature should include picture books of people from many different cultures so students will understand people who are different from them. they state that good books broaden children’s understanding and empathy for other children around the globe. children can identify with the story’s character(s) and “walking in someone else’s shoes is a way that often helps them develop a greater capacity to empathize with others” (lynch-brown & tomlinson, 2011. p. 5). norton (2007) agrees that children’s literature plays a vital role in helping children understand their own heritage. she makes the following statement: carefully selected literature can illustrate the contributions and values of the many cultures. it is especially critical to foster an appreciation of the heritage of the racial minorities in american society. a positive self-concept is not possible unless we respect others as well as ourselves; literature can contribute considerably toward our understanding and thus our respect (p. 3). also, the lack of culturally diverse literature in the classroom may be a subtle form of racial bias. lynch brown et al., (2011) make the following point, “even when schools and communities are culturally homogeneous, librarians and teachers should select books that reflect the diversity of the greater world” (p. 214). literature can be a powerful tool for international electronic journal of elementary education vol.5, issue 2, 143-156, 2013 146 combating mistrust or fear of people who are foreigners (tunnel, jacobs, young, & bryan, 2012). gender bias in children’s books. another criterion when selecting high quality picture books for the primary classroom should be the absence of gender bias in the text and illustrations (norton, 2011). according to narahara (1998), teachers need to be critical when making selections to avoid books containing stereotypical gender roles. narahara (1998) further states, “gender role stereotypes affect how children perceive themselves, especially young children (p. 4). studies show that females are likely to be portrayed as dependent human beings and submissive in nature, which is not an accurate representation of women today (morgan, 2009). the selections should also possess protagonists equally representing both sexes in their storylines. a study by davis and mcdaniel (1999) that examined male and female appearances in caldecott-winning books from 1972 to 1997 shows that males represented 61% of the protagonists while females accounted for only 39% (as cited in morgan, 2009). in conclusion, as children learn about authors of picture books and their writing styles, primary teachers’ selections should have a balance of books that have been written by both male and female authors so that children can recognize and visualize their own writing aspirations by the books’ authors. indiana reading list the k-12 indiana reading list (irl) arose out of the desire to help teachers and parents identify literature selections that align to the reading level and state standards in the content areas. the state’s standards were first designed through stakeholder committees in the late 1990s and published in 2000. the irl then followed and is divided into four grade level categories: k-2, 3-5, 6-8, and 9-12. it was last updated in 2008. the instructions accompanying the k-2 irl state: these lists provide lots of good choices for reading across all grade levels. there is something here for everyone, but they are not all inclusive. many of the authors on the list have written other interesting works. many titles are the first in a series that may spur readers into reading the other books about the same characters. parents and families will want to preview every title for appropriateness of content, interest, and reading level before selecting it for reading for their children (indiana department of education, 2008). additionally, when reviewing the k-2 irl one also finds the statement: designed as a companion piece to indiana’s academic standards in english/language arts, the following selections of the irl illustrate the quality and complexity of the suggested reading materials for students in grades k – 2. the irl is not required reading nor is it meant to be all-inclusive. teachers and parents are encouraged to review the selections to ensure suitability for the individual student (indiana department of education, 2008). the disclaimers are noteworthy and transparent in that these selections need scrutiny by educators and that other selections might be useful to educate children. to this end, the researchers set out to explore the k-2 irl selections for children’s literature in an attempt to answer six research questions. 1. does the k-2 irl of fiction books contain current selections? 2. does the k-2 irl of fiction books contain a balance of authors from both sexes (male and female)? questioning the validity of a state reading list / berridge & thomas 147 3. does the k-2 irl of fiction books contain at least half its titles as ala award winning selections? 4. does the k-2 irl of fictions books contain selections that have equal representation of male and female protagonist(s)? 5. does the k-2 irl of fictions books contain selections that have balanced representation in the racial composition of protagonist(s)? 6. does the k-2 irl of fiction books contain gender bias? 7. does the k-2 irl of fiction books contain racial bias? method to address the research questions this study evaluated the 79 fiction books listed on the k-2 irl. that meant that for this study fiction books were limited to the titled sections on the k-2 irl of fiction books (50 titles) and folklore/fairy tales/mythology (15 titles), and poetry (14 titles). general fiction in this list is defined by the sate as picture books, chapter books, illustrated books where the pictures do not tell the story and concept books that focus on shapes, colors and sizes. while the genre containing specific titles in the k-2 irl may differ among researchers, we have adhered strictly to how they are categorized (and in essence coded) by the state’s department of education and listed specifically in the k-2 irl. since the k-2 irl is already published, the study assumes a sample of convenience. it is important to note that a purposeful sampling of the k-2 irl would not be adequate for the study. the remaining k-2 irl sections of nonfiction books (32 titles), biographical books (13 titles), magazines (11 titles), reference books (4 titles), and technical documents (4 titles) are outside the scope of this study. evaluation utilized six variables to reveal findings about the research questions. the variables included the 1) publication date, 2) sex of author, 3) selection as a caldecott, coretta scott king writing, coretta scott king illustrating, mildred batchelder, or pura belpré award winning book, 4) protagonist’s expressed or implied sex, 5) protagonist’s race, and 6) presence of gender or racial bias. the term sex is used as the appropriate term for research questions two and four to identify whether one is male or female. the term gender is used in research question six as the appropriate way to describe the perceived roles of men or women in social groups. seven books had no stated protagonist and focused on a concept or idea (e.g., ten black dots). no frequency tally was recorded for such a book when this occurred. when a book had both a male and female protagonist then a frequency count was recorded for both sexes. to begin the process the researchers randomly selected 15 books from the list on which to perform an item analysis to establish inter-rater agreement. both researchers took notes on each of the fifteen books as to the protagonist’s sex and race. if the researchers detected any gender or racial bias the page(s) was documented for further review. the researchers decided to record gender bias based on two distinct criteria from russell’s (2012) description. they include dominating roles for men and subservient roles for women and/or casting women as the weaker sex and the male as the physically stronger sex. racial bias was based on russell’s (2012) criteria of portraying characters as an ethnic or cultural group with similar habits or traits, with the same socioeconomic status and occupations, and not portraying a culture respectfully and accurately. international electronic journal of elementary education vol.5, issue 2, 143-156, 2013 148 the researchers discussed their initial reviews and came to agreement on all 15 books. the same evaluation method was then performed on the remaining 64 books. one researcher recorded the publishing date and author’s gender. the other researcher completed the review for award winning books. after all the books were reviewed, the researchers met, came to agreement if discrepancies existed, and created a final entry for tabulation. results following are the results for the analysis of the 79 books on the indiana reading list for primary grades that were classified as fiction. research question 1 asked, “does the k-2 irl of fiction books contain enough current selections?” item analysis revealed an average publication date of 1971 and a median publication date of 1984. the earliest original publication date was 1843 and the most recent was 2002. interestingly, the 2002 publication was the only selection originally published in this 21st century (see table 1). table 1. item analysis results for the k-2 indiana reading list fiction sections book title year of 1st publication author award sex of protagonist(s) race gender or racial bias the adventures of old mr. toad 1916 m x male x x amelia bedelia 1963 f x female white gender angela weaves a dream 1997 f x female latina x anno’s journey 1977 m x male white x arthur 1995 m x male x x baseball in the barrio 1997 m x male latino x a bear called paddington 1958 m x male x x brown bear, brown bear… 1967 m x x x x butterfly boy 1997 f x male latino x caps for sale 1940 m x male white x the cat in the hat 1957 m x male white x corduroy 1968 m x male x x dinosaur dream 1974 m x male x x the doorbell rang 1989 f x male/ female white x flossie and the fox 1986 f x female black x frog & toad … 1970 m caldecott honor male x x the gingerbread.. 1998 m x male white x the giving tree 1964 m x male white x goodnight moon 1947 f x x x x green eggs & ham 1960 m x 2 males /female white x questioning the validity of a state reading list / berridge & thomas 149 harold & the purple crayon 1955 m x male white x table 1. (continue) item analysis results for the k-2 indiana reading list fiction sections book title year of 1st publication author award sex of protagonist(s) race gender or racial bias hattie and the fox 1987 f x female x x henry and mudge 2002 f x male white gender if you give a mouse a cookie 1985 f x male white x just so stories 1912 m x male white x lily’s purple plastic purse 1996 m x female x x the little engine that could 1930 m x female x x lyle, lyle crocodile 1965 m x male white x make way for ducklings 1941 m caldecott medal female white x grandfather’s journey 1993 m caldecott medal male asian x millions of cats 1928 f x male white x the mitten 1989 f x male white x a mother for coco 1992 m x male x x the mouse and the motorcycle 1965 f x male x x now one foot, now the other 1981 m x male white x the polar express 1985 m caldecott medal male white x rosie’s walk 1968 f x female x x the snowy day 1962 m caldecott medal male black x song of the swallows 1949 m caldecott medal male latino x stone soup 1986 f x male white x sylvester and the magic pebble 1970 m caldecott medal male x x the tale of peter rabbit 1902 f x male x gender tar beach 1991 f coretta scott king female black x ten back dots 1968 m x x x x international electronic journal of elementary education vol.5, issue 2, 143-156, 2013 150 there’s an alligator under my bed 1987 m x male x x table 1. (continue) item analysis results for the k-2 indiana reading list fiction sections book title year of 1st publication author award sex of protagonist(s) race gender or racial bias the three little pigs 1997 m x male x x the ugly duckling 1843 m x male x x the very hungry caterpillar 1969 m x x x x where the wild things are 1963 m caldecott medal male white x winnie the pooh 1926 m x male x x the golden goose 1995 m x male white x goldilocks and the three bears 1996 f x female white x john henry 1987 m x male black x la cucaracha martina: the caribbean folktale 1997 m x female x x little red riding hood 1982 m x female white x lon po po 1989 m caldecott medal male x x mufaro’s beautiful daughters 1987 m caldecott honor female black x paul bunyan 1984 m x male white x pecos bill 1992 m x male white x the selkie girl 1986 f x female white x song to demeter 1987 m f x female white x the story of johnny appleseed 1971 m x male white gender the story of jumping mouse 1984 m caldecott honor male x x the village of round and square houses 1986 f caldecott honor female black x three billy goats gruff 1973 m x male x x the big red 1989 f x x x x questioning the validity of a state reading list / berridge & thomas 151 barn chicken soup with rice 1962 m x male white x table 1. (continue) item analysis results for the k-2 indiana reading list fiction sections book title year of 1st publication autho r award sex of protagonist(s) race gender or racial bias child’s garden of verses 1885 m x male/female white x the dragons are singing tonight 1993 m x x x x eats; poems 1979 m x male x x every time i climb a tree 1925 m x male/ female white x flicker flash 1999 f x male/female white/ black x i know an old lady who swallowed a fly 1990 m caldecott honor female white x james marshall’s mother goose 1986 m x male/female white x orchard book of nursery rhymes 1990 f x male/female white x owl moon 1987 f caldecott medal male white x sheep in a jeep 1988 f x x x x the wheels on the bus 1990 m x male/female white x you read to me, i’ll read to you 1962 m x male white x mean 1971 median 1984 earliest 1843 most recent 2002 research question 2 asked, “does the k-2 irl of fiction books contain a balance of authors from both sexes (male and female)?” the 79 books on the list included 80 different authors since one of the selections was coauthored. there were 55 books with male authors (68.8%). that was approximately twice the number of 25 (31.3%) books written by female authors (see table 1). research question 3 asked, “does the k-2 irl of fiction books contain at least half its titles as ala award winning selections?” analysis indicated 14 (17.8%) of the identified books on the list were selected as caldecott award winning titles and that one (1.2%) of the books was selected as a coretta scott king award selection. there were no books on the list that had won either a mildred batchelder or pura belpré award (see table 1). research question 4 asked, “does the k-2 irl of fiction books contain selections that have equal representation of male and female protagonists?” table 2 identifies the percent sex of international electronic journal of elementary education vol.5, issue 2, 143-156, 2013 152 protagonists in the books on the k-2 irl. the number of books that contained a male protagonist was 56 (69.1%) and the number of books that contained a female protagonist was 25 (30.9%). the sex demographics for indiana and the united states are also included for comparison (u.s. census, 2010). chi-square analysis yielded significant differences between the k-2 irl percent sex of protagonists and indiana populations (x2 = 15.8, df = 1, p < 0.0001) and the united states (x2 = 15.8, df = 1, p < 0.0001). table 2. percent sex expressed in protagonists in the indiana reading list sex frequency percent of protagonist(s) indiana demographics united states demographics female 25 30.9 50.8* 50.8* male 56 69.1 49.2* 49.2* totals 81 100.0 100.0 100.0 *united states census bureau (2010) research question 5 asked, “does the k-2 irl of fiction books contain selections that have a balanced representation in the racial composition of protagonists?” table 3 identifies the percent composition of the racial identities of protagonists in the books on the k-2 irl. the racial demographics of indiana and the united states are also included for comparison (u.s. census, 2010). only the races found in the k-2 irl analysis are included in table 3 for indiana and the united states. since some books were of animals or abstract concepts only, not every k-2 irl selection included a data point for this variable. the data revealed 38 (76%) of the selections were white, seven (14%) were black, four (8%) were latino/a and one (2%) was asian. remarkably, there were no selections that showed the american indian race. chisquare analysis yielded no significant differences between the k-2 irl percent race of protagonists and indiana populations (x2= 13.6, df = 5, p = 0.299) or the united states (x2 = 12.39, df = 6, p = 0.054). table 3. percent race expressed in protagonists in the indiana reading list race frequency percent of protagonists indiana demographics* united states demographics* american indian 0 0 0.3 0.9 asian 1 2 1.6 4.8 latino/a/ hispanic 4 8 6.0 16.3 black 7 14 9.1 12.6 white/not hispanic 38 76 81.5 63.7 native hawaiian 0 0 0 0.2 two or more 0 0 2 2.9 total 50 100 100.5 101.4 * united states census bureau (2010)census totals are not equal to 100%* research question 6 asked, “does the k-2 irl of fiction books contain gender bias?” only 4 (5.1%) of the 79 books contained gender bias (see table 1). the observations for bias in all four books were illustrations perceived against women. women were observed wearing aprons and completing household chores like cooking and cleaning. research question 7 asked, “does the k-2 irl of fiction books contain racial bias? there were no observations for racial bias. questioning the validity of a state reading list / berridge & thomas 153 discussion we found in the k-2 irl that the selection of literature ranged from 1843 to 2002 for year of first publication. some of the book jackets and illustrations were quite old and not as attractive as newer selections. many of the older books were small and not suitable to read to a large group of children. new forms of children’s picture books including the writing and illustrations were not included in the k-2 irl selections even though some of these books had been published with awards since 1991. there was only one book on the list published since 2000. and, only 18 of the 79 fiction books were published in the 1990s. the researchers understand that a book’s worth cannot be determined by the year in which it was published; however, with the changing landscape of children’s books and new printing technologies that create more sophisticated illustrations and fonts, it would seem that there would be more current selections. most of the books on the list were dated and may not be relevant or engaging for today’s primary students. therefore, we answered research question one by stating the k-2 irl did not contain enough current selections. there were twice as many male authors as females. the researchers felt that a balance of book authors in the classroom is more beneficial to best promote equal perspectives from both male and female writers. before reading a picture book, teachers often identify the author and illustrator and may even give a brief biography before or after reading the book. sometimes students write letters to the author, see an author in person or skype with them online to discuss ideas for characters or the writing style of the book. if there is not a balance of male and female authors, young girls may feel that authoring a book is mainly a man’s job and a teacher may not be aware of the developing stereotype. therefore, we answered research question two by stating the k-2 irl of fiction books did not contain a balance of authors from both sexes (male and female). only 19% of the books selected for this list were award-winning books using our selected criteria, which the researchers felt was far too few. since librarians and teachers looked to the experts for books to use in their classroom, it seemed that this list should contain a robust number of the selected children’s book awards. the award winning book selections are published online and are easy to access. therefore, we answered research question three by stating the k-2 irl of fiction did not contain at least half its titles as ala award winning selections? nearly 70% of the books contained a male protagonist, a finding that perpetuates a male dominant world. both boys and girls need to “see themselves as human beings with an equal right to all benefits and choices” (narahara, 1998, p. 13). according to russell (2012), stereotypes and gender bias still plague our society. the findings supported those of the sociologists for women in society (2011) that found similar results for about 6000 leading children’s books that were published in the 20th century. therefore, we answered research question four by stating the k-2 irl of fiction books did not contain selections that have equal representation of male and female protagonists. of the seven demographic categories reported in the u.s. census, four were represented in the k-2 irl. the 79 picture books included protagonists from the following races: white/not hispanic, black, latino/a/hispanic and asian. three race categories, american indian, two or more races, and hawaiian, were not represented. as a global society changes the makeup of cultures and races for indiana and the united states, it is essential for schools to include books about children from many races. lukens et al. (2013) recommended that teachers include in their classroom libraries books about children who are different from the white middle-class and also incorporate a wide variety of books that represent lifestyle and cultural beliefs of as international electronic journal of elementary education vol.5, issue 2, 143-156, 2013 154 many children as possible. the value of such literature for children include understanding, respect and tolerance for different cultural groups, understanding of universal rights and fundamental freedoms, and preparation for our children to live and prosper in a diverse society (norton, 2011). one goal in promoting diversity is to get students to look beyond their own local communities for learning. children need books that represent our entire world to develop positive attitudes and respect for all individuals. the data from table 3 indicates that the percent race of protagonists is balanced. the researchers identified the racial composition of only protagonists so that data was recorded consistently. at times this was challenging. an example is winnie the pooh. the researchers did not record a race for the protagonist, pooh, since he is not human. but, christopher robin is a supporting character and is white. future studies could investigate the race for supporting characters and how they contribute to the representation and perception of racial identities. an increase in the total number of white characters may lead to an increased perception of “whiteness” among the selections. but, when considering the protagonists statistical analysis indicates a balanced selection. therefore, we answered research question five by stating the k-2 irl of fiction books did contain selections that had balanced representation in the racial composition of protagonists. the gender bias was a small percentage, but still concerning. in the four books on the list, women were consistently performing household chores and seen only wearing dresses. for example, amelia bedelia by peggy parish was constantly illustrated wearing an apron over her dress and a bonnet. however, because amelia bedelia was published in 1963 when women in the united states were not seen wearing pants in picture books, this would make sense. so, in at least two of the books the gender bias was because of the age of the selection, not depicting women only as “domestic engineers.” however, books like amelia bedelia can help teachers and parents address gender issues by creating a teachable moment. gender awareness or fair treatment of the sexes can be discussed while the book is read. teachers and parents can overtly address the inaccurate and unfair stereotypes to both boys and girls. we need to teach children that some attitudes and viewpoints often found in earlier books are no longer acceptable (russell, 2012). therefore, we answered research question six by stating the k-2 irl of fiction books did not contain gender bias. as there were no observations for racial bias, one can reasonably infer that this specific section of the irl contains selections where characters were portrayed appropriately and respectfully. therefore, we answered research question 7 by asserting that the k-2 irl of fiction books did not contain racial bias. conclusions and limitations the overall conclusion is that the k-2 irl fiction books are out of date and do not contain enough quality selections. nor does the list contain a balance of selected texts written by female authors. furthermore, more selections with female protagonist are needed to balance the list. it was also encouraging to see that the list contained minimal gender bias and no racial bias. while the study provides a window into the k-2 irl there are some limitations that need to be recognized. some titles on the list were published before the dates that the awards chosen for this study were instituted. therefore, many of the titles were not candidates for the awards. other awards for excellence might yield different conclusions. additionally, some may view timeliness differently based on their focus and beliefs about technology, relevancy of the “classics”, and the importance of visual literacy in children’s books. as described earlier, supporting characters contribute to over presence of race and gender and should also be considered in interpreting these results. also, when analyzing books for bias there is a degree questioning the validity of a state reading list / berridge & thomas 155 of interpretation that exists between readers. one reader may not view bias in dialogue, dress, or role of a character, but another may. recommendations this study provides a model that others may use when asking the question, “how well does their sponsored list represent quality selections?” the k-2 irl review, or similarly conducted reviews of sponsored reading lists, are useful for classroom teachers, curriculum developers, and other professionals as they adopt new quality children’s literature to help teach their national or state standards. similar reviews can be used to assess the suitability of sponsored lists to address the learning and application of developing literacy skills in nonprint forms such as media and digital literacies. though adoptions may come from a sponsored list, quality may not be ensured. we encourage professionals to review and share their results with educators and community leaders as they work at creating a reading environment that is rich and academically rewarding for all students. they may wish to include different awardwinning titles that better represent their aspirations and literacy standards. the researchers assert that a well-developed and continually updated list can help guide parents and educators to appropriate grade-level literature that is current, culturally diverse, and gender balanced. ultimately, incorporating children’s literature with primary students that is relevant and engaging will help them become successful not only in school, but also in our global society. . . . gina berridge received her ph.d. in educational leadership, administration and foundations from indiana state university in 2006. she is an assistant professor at the university of southern indiana in the pott college of science, engineering and education. her research interests focus on literacy strategies and programs in the elementary classroom and teaching in general. jeff thomas received his ph. d. in science education from the university of southern mississippi. he is an associate professor at the university of southern indiana in the pott college of science, engineering and education. he works primarily with preservice science methods courses, field practicum experiences, and facilitating student teacher experiences. references indiana department of education (2008). indiana reading list. <https://learningconnection.doe.in.gov /standards/printlibrary.aspx> (last accessed april 23, 2012). lukens, r. j., smith, j. j., & coffel, c. m. (2013). a critical handbook of children’s literature (9th ed.). upper saddle river, nj: pearson. lynch-brown, c., & tomlinson, c. m. (2005). essentials of children’s literature (5th ed.). boston, ma: allen and bacon. lynch-brown, c., & tomlinson, c. m. & short, k. g. (2011). essentials of children’s literature (7th ed.). boston, ma: pearson. international electronic journal of elementary education vol.5, issue 2, 143-156, 2013 156 narahara, m. (1998). gender bias in children’s picture books: a look at teachers’ choices of literature. research report. long beach, ca: university of california. morgan, h. (2009). gender, racial, and ethnic misrepresentation in children’s books: a comparative look. childhood education, 85(3), 187-190. retrieved from http://search.proquest.com/docview/210394277?accountid=14752 norton, d. e. (2007). through the eyes of a child (7th ed.). upper saddle river, nj: pearson. norton, d. e. (2011). through the eyes of a child (8th ed.). upper saddle river, nj: pearson. russell, d. (2012). literature for children. boston, ma: pearson. sociologists for women in society (2011, may 3). gender bias uncovered in children's books with male characters, including male animals, leading the fictional pack. sciencedaily. retrieved november 11, 2011, from http://www.sciencedaily.com/releases/2011/05/110503151607.htm#. temple, c., martinez, m., & yokota, j. (2006). children’s books in children’s hands (3th ed.). boston, ma: pearson. tunnel, m. o., & jacobs, j. s., (2008). children’s literature, briefly (4th ed.). boston, ma: pearson. tunnel, m. o., jacobs, j. s., young, t. a., & bryan, g. w. (2012). children’s literature, briefly (5th ed.). boston, ma: pearson. u.s. department of education, (2008). schools and staffing survey (sass), public school and public school teacher data files, national center for education statistics: washington, d.c. united states census bureau (2010). < http://www.census.gov/>. (last accessed april 23, 2012). international electronic journal of elementary education, june 2016, 8(4), 629-642. issn:1307-9298 copyright © iejee www.iejee.com examining pre-service elementary school teacher beliefs and instructional practices in mathematics class yoppy wahyu purnomo a didi suryadi b sutawanir darwis c a university of muhammadiyah prof. dr. hamka, indonesia b indonesia university of education, indonesia c bandung institute of technology, indonesia received: 2 may 2016 / revised: 29 may 2016 / accepted: 30 may 2016 abstract the gap between theory and practice has become a critical issue in the effort of improving the learning of mathematics. beliefs may have been one of the contributing factors to the widening of the gap between theory and practice. therefore, examining the relationship between beliefs and practices in mathematics is crucial to gain an overview of the preparation of potential teacher candidates and the development of teacher education in the future. this study aims to examine the relationship between beliefs held by pre-service elementary school teachers and the instructional practices in mathematics class. this study employs the case study that focuses on one of the preservice elementary school teachers who is undertaking practical field experience in the 2015/2016 academic year. the findings of this study indicate that the instructional practices do not necessarily reflect the beliefs that are held. on the other hand, beliefs about the nature of mathematics influence more dominant than the other beliefs against instructional practices. keywords: teacher beliefs, pre-service elementary school teachers, instructional practices, mathematics class introduction basically, the school mathematics-based research has provided a valuable contribution to what is currently the best process for the acquisition of knowledge and skills (de corte, 2003). the results of either quantitative or qualitative research or the combination of both leads to suggestions for the points that have to be considered in developing the teaching of mathematics in schools. however, the phenomenon of the gap between theory (including the results and recommendations of research) and practice in schools is the critical issue that needs to be raised to the "surface" (geiger & goos, 2006; malone, 2000; smith, 2000). the gap is an inconsistency between what is expected from the results of the research and decision making in practice at school.  corresponding author: yoppy wahyu purnomo, university of muhammadiyah prof. dr. hamka, faculty of teacher training and education, jalan tanah merdeka, pasar rebo, jakarta, indonesia, phone: +6285-293223063, e-mail: yoppy.w.purnomo@uhamka.ac.id http://www.iejee.com/ international electronic journal of elementary education vol.8, issue 4, 629-642, june 2016 630 the teachers’ belief is one of the potential factors contributing to the gap between theory and practice. the plausible reason for this is that belief becomes one of the variables that leads a person in making decisions (pajares, 1992; thompson, 1992). in addition, the instructional practices conducted by the teacher come from the decisions about something that is believed to be true, which comes from the knowledge that they have gained. in other words, the focus on building knowledge and beliefs is relevant to the stages of developing teachers’ professionalism (e.g., fennema, carpenter, & franke, 1992; tatto et al., 2008; vacc & bright, 1999). empirically, on one hand, some researchers have found that teachers beliefs concerning the study of mathematics were consistent with their practices and behaviors in mathematics classes (e.g., golafshani, 2005; stipek, givvin, salmon, & macgyvers, 2001; zakaria & maat, 2012). on the other hand, some researchers have found that there were inconsistencies between the beliefs and practices in mathematics classes (barkatsas & malone, 2005; raymond, 1997). barkatsas and malone, through the case studies against a veteran teacher (i.e., ann), found that ann holds beliefs that are not always consistent with the instructional practices. the inconsistency is primarily caused by the class situation, experience, and social norms. the findings as disclosed by raymond (1997) that in addition to the class situation, experience, and social norms, inconsistencies between the beliefs and instructional practices are influenced by internal factors (e.g., the teacher’s personality) and external factors (e.g., the environment). in indonesia, the exploration of teachers' beliefs and practices is quite lacking, and it is the expectation of this study is able to give an overview and contribute suggestions to policymakers and subsequent researchers about how to build a strong foundation for developing the teacher education program. some researchers agree that the golden period to build the professional teacher began when candidates were educated in college (purnomo, 2015; siegel & wissehr, 2011; volante & fazio, 2007). based on these reasons, this study focuses on examining the beliefs and instructional practices of pre-service teachers and the relationship of the related factors between the two variables. dealing with the definition of beliefs belief is an arbitrary construct designed in such a way that it is hard to give a simple definition. there are many variations of the concept of belief that is used in mathematics education research, so some researchers often formulate their own definition of beliefs that may even be contrary to others (furinghetti & pehkonen, 2002; thompson, 1992). some researchers looked at the beliefs in the cognitive domains (e.g., thompson, 1992; törner, 2002), other researchers put it in the effective domain (e.g., mcleod, 1992), and several others looked at the beliefs in both domains (e.g., goldin, epstein, schorr, & warner, 2011; leder & forgasz, 2002). correspondingly, there are researchers who argued that the beliefs represent parts of the knowledge (e.g., furinghetti & pehkonen, 2002), some think that beliefs contribute to the attitude (e.g., mcleod, 1992), and the other state them as mere conceptions (e.g., thompson, 1992). referring to an online dictionary (www.merriam-webster.com), beliefs may be defined as the conviction of the truth of some statement or the fact of some being or phenomenon, especially when based on the examination of evidence. aligned with these definitions, some researchers consider that the characteristics of these beliefs refer to the degree of the person's conviction (furinghetti & pehkonen, 2002; thompson, 1992). in other words, beliefs associated with the person's psychological strength may alter the degree of conviction. however, this raises further questions about what the conviction itself is. on one hand, schoenfeld (1992) argues that beliefs can be defined as understanding and feeling of the individual which shapes the way that the individual conceptualizes and beliefs and instructional practices in mathematics class / purnomo, suryadi & darwis 631 affects their behavior. furinghetti and pehkonnen (2002) argue that the definition proposed by schoenfeld is more akin to "how beliefs function". thompson (1992) defines beliefs as part of one's conception, namely a person's mental structure which includes knowledge, belief, understanding, preferences, and views. thompson uses the terms beliefs and conceptions interchangeably because the differences were probably not very important to him. on the other hand, ponte and his colleagues (ponte, 1994; ponte & chapman, 2006) argued that the belief in stating that something is right or wrong, is not based around empirical evidence, and thus holds a proportional nature and does not require internal consistency. meanwhile, a conception is a cognitive construction that can be seen as the organizing framework of the underlying concept. pajares (1992) distinguishes the term belief (conception) and knowledge, in which belief is based on evaluation and decision while knowledge is based on objective fact. likewise, griffin and ohlsson (2001) stated that knowledge and belief refer to qualitatively different aspects of mental representation, in which knowledge refers to the representation of the proposition, and belief refer to the representation of the truth regarding the value of said proposition. griffin and ohlsson define knowledge as understanding or awareness of the ideas or propositions ("i understand the claim that humans evolved from monkeys"). after the proposition is known, one can accept it as true ("i believe the claim that ..."), reject it as false ("i do not believe the claim that ..."), or by reserving judgement ("i do not have an opinion about the claim that ... "). furinghetti and pehkonen (2002) analyzed the relationship between belief and knowledge by dividing knowledge into two aspects: objective knowledge (i.e., knowledge accepted by the community) and subjective knowledge (i.e., knowledge created by an individual and does not have to be evaluated by others , see also ernest, 1991, 1998). furinghetti and pehkonnen concluded from their analysis that beliefs refer to the individual's subjective knowledge, and when expressed as a sentence, may (or may not) be logically correct. thus, the beliefs may not be 100% logically correct, while knowledge holds a 100% probability of being correct. similarly, leatham (2006) made an analogy of the relationship between beliefs and knowledge to describe that of everything we believe, there are some things that we ''just believe '' and other things that we ''more than believe '' because we ''know'' them. the things that we ''more than believe'' are reffered to as knowledge and the things that we ''just believe '' are called beliefs. thus, beliefs and knowledge can be seen as complementary subsets of the set of things that we believe. abstract constructs on the definition of beliefs allow someone to give a conclusion based upon the individual perception. these constructs may occur because in order to lead the goal of research and being capable of describing each variable to focus on, the researcher should be able to give a decision and clarify the definition of the legal basis for the research focus. therefore, this study assumes that beliefs can be in the cognitive domain if we emphasize the relationship between beliefs and knowledge. on the other hand, if the beliefs are seen from the reaction to a particular situation means we consider the beliefs associated with the affective part of individual (furinghetti & pehkonen, 2002). the definition of beliefs refers to the subjective knowledge of the individual (furinghetti & pehkonen, 2002; op't eynde, de corte, & verschaffel, 2002) based on the experiences (raymond, 1997; thompson, 1992) and expressed in [attitude] proposisional (goldin, 2002; griffin & ohlsson, 2001; pajares, 1992), views, and perception (thompson, 1992) to a value of truth (goldin, 2002; griffin & ohlsson, 2001). thus, the term conception, [subjective] knowledge, belief, perception in this research is used interchangeably. international electronic journal of elementary education vol.8, issue 4, 629-642, june 2016 632 method participant this study uses case studies to explore the beliefs and instructional practices of one of the pre-service elementary school teachers, who shall be referred to as sarijem (pseudonym) and the relationship and potential factors associated in between these two variables. sarijem is a 7th-semester student, 21-years-old and ethnically javanese. she is from a hometown of entrants who followed her parents working in jakarta. sarijem was born in east java and has graduated from elementary school and junior high school in her native region and graduated high school in pati, central java. in addition, all the schools were public schools. this study was conducted at the time when participants joined the program field experience in the academic year of 2015/2016 at one elementary school in east jakarta. sarijem has not had any previous teaching experience, so this study is intended to contribute to the development of teacher training courses, especially in elementary schools. data collection data were collected through a questionnaire, videotaped classroom observations and an interview. the questionnaire with open-ended questions focused on obtaining data on the beliefs about the nature of mathematics, and the beliefs about mathematics teaching and learning. beliefs about the nature of mathematics are a viewpoint, perception or conception of someone’s overall perception of mathematics as a discipline (beswick, 2012; ernest, 1989; perkkilä, 2003; thompson, 1992). whereas, beliefs about teaching and learning is subjective knowledge or the implicit viewpoint of teachers on various types of teaching, the meaning of teaching and learning, the role of teachers and students in learning, how students learn mathematics, and class activities related to teaching mathematics (chan, 2004; ernest, 1989; thompson, 1992). videotaped classroom observations are used to obtain the behavioural and instructional practices in the classroom. in addition, beliefs, practices and related factors are explored with the semistructured interview. the questionnaire was given to sarijem before she performed a series of classroom teachings. the questionnaire contains open-ended questions in the following order (adapted from adam, 2012) . 1. how do you explain what mathematics is to someone who has never heard of mathematics before? 2. what do you think mathematicians do when they work with mathematics? 3. in your opinion, what is the best way for children to learn mathematics? 4. what methods do you have that can have an effect on the way students learn mathematics? 5. in maths class, what do you think about the role of (a) teachers and (b) students? 6. in your opinion, what is the most effective way of teaching mathematics? the six 35-minute teaching sessions were videotaped in order to obtain data on instructional practices in the classroom. sarijem taught the fourth grade with materials related to the greatest common factors and least common multiples. after the series of teaching activities were completed, interviews were conducted to elaborate sarijem’s beliefs that she had stated in her answers to the questionnaire. the interview was carried out using similar questions to those asked on the questionnaire and was recorded using the audio-recording application on a hand phone. furthermore, the beliefs and instructional practices in mathematics class / purnomo, suryadi & darwis 633 interview was also conducted to elaborate the instructional practices conducted by sarijem and the factors associated with beliefs and practices. some questions in addition to those contained in the questionnaire are shown as follows. 1. how do you teach mathematics? 2. why did you introduce the mathematical ideas in this manner? 3. how did you help students understand the new ideas? 4. what encouraged you to teach in this manner? 5. what limits you in being able to teach mathematics in the way you want/plan? data analysis the questionnaire responses were analyzed with an interesting pattern and elaborated through the interview. related audio recordings and the transcribed interview was analyzed to describe the beliefs held by sarijem. the pattern of responses related to the questionnaire and the interview are categorized into traditional beliefs (absolutist), primarily traditional, primarily constructivist, and constructivist (fallibilist) for the dimensions on the nature of mathematics and the teaching and learning of mathematics. the recorded video was played and viewed together with the participant along with the interview to explore the teaching practices and limitations that the participant faced throughout the teaching sessions. this was done to confirm the answers to the questions stated in the questionnaire and the interview beforehand. the video footage and interview has been transcribed, then read and verified again by the participant to ensure the accuracy and thoroughness. results and discussion this section discusses sarijem’s beliefs by splitting them into two separate parts, i.e. beliefs about the nature of mathematics and beliefs about teaching and learning mathematics. the analysis of the interconnectedness of beliefs and practices as well as potential factors that influence them was addressed afterwards. sarijem’s beliefs about the nature of mathematics based on the pattern of responses to the "open-ended questions" on the questionnaire and the interview session, beliefs about the nature of mathematics held by sarijem leads to the combination of traditional and constructivist view, with primarily traditional mathematics. sarijem states that mathematics is an exact science that is used by humans as a means of counting everyday problems. when asked more about mathematics, sarijem stated that mathematics is a science that is applicable and relevant to a person's experiences and everyday problems. ernest (1991) states that the constructivist view of looking at mathematics as part of a blend with the human culture, so that it cannot be separated with the knowledge of physical and other sciences. in other words, mathematics comes from and for social purposes. the researcher: how do you explain what mathematics is like to someone who has never heard of mathematics before? sarijem: ... mathematics is a science that is used by humans to help calculate ...include the symbols and rules to do the calculations.... the researcher: besides that, what else do you think about maths? sarijem: mathematics is also used in other studies and is relevant to the problems of everyday life. international electronic journal of elementary education vol.8, issue 4, 629-642, june 2016 634 the researcher: earlier, you said that mathematics includes a collection of symbols and rules. according to you, who makes the rules? is it the mathematicians or does everyone have a chance to make or add to them? sarijem: mathematicians, sir, because mathematicians have a thought process that is coherent and logical ... if we're not coherent and logical thinking. the researcher: ok. do you think that the mathematical rules that already exist could be wrong? sarijem: i think not, sir, because the rules have been established through a series of experiments and published afterwards. sarijem stated that the rules of mathematics are accepted to be true. the mathematical rules and facts can't be doubted and hold an undeniable truth. furthermore, sarijem also stated that the role of mathematicians is crucial in the development of mathematics. the mathematical objects are discovered by mathematicians through a series of experiments and published after perfection. sarijem’s views about maths are in line with the views held by an absolutist. absolutists see mathematics as absolute truth, accepted by everyone, and does not rely on the human knowledge and context (ernest, 1991; hersh, 1997). the views absolutists can be associated with the platonic, formalist, and logicist views stating that mathematics is as absolute as a divine gift, a formal language without error or contradiction, does not depend on human knowledge, waiting to be discovered and has existed before the birth of mankind, a set of rules and procedures that are rigid and picture mathematics as essential calculations (ernest, 1991; hersh, 1997; sriraman, 2004; white-fredette, 2010). in other words, mathematical objects are taken for granted to be applied by the user. sarijem’s beliefs about teaching and learning mathematics typically, sarijem adopts beliefs about mathematics teaching and learning that is primarily constructivist-oriented. sarijem stated that the best way to learn mathematics is by connecting mathematics with daily life. the following is sarijem’s statement about the best way to learn mathematics. figure 1. sarijem’s statement about the best way to learn mathematics translate: 3. in your opinion, how can children best learn mathematics? sarijem’s answer: in my opinion, it is best to associate mathematics with reality or everyday life. by linking mathematics to daily life children will easily be able to understand the concepts that are being delivered. because the concept is presented with something that can be imagined by children and occurs in the environment around them. beliefs and instructional practices in mathematics class / purnomo, suryadi & darwis 635 in the interview and discussion sessions, sarijem stated that "...by associating mathematics with daily life experiences, students can easily find and grasp the concept if they can imagine it." moreover, sarijem also stated that "... it is difficult for the students, especially in elementary school, if mathematics is not associated with anything they can imagine." sarijem’s beliefs about the best way of teaching and learning mathematics are consistent with her beliefs that mathematics cannot be separated from the human context and everyday life. in the questionnaire, sarijem stated that the teacher's role in learning is as a facilitator in guiding students to discover new knowledge. meanwhile, the student's role is as a person seeking information or knowledge. sarijem's statement was in line with the constructivist view which proposes that the conception and understanding of learners come from the construction of meaning where learners are involved in the process of building the individual interpretation of their experiences (applefield, huber, & moallem, 2000; kundi & nawaz, 2010; ari, tunçer, & demir, 2016). the following is sarijem's statement about the role of teachers and students in mathematics class. figure 2. sarijem’s statement about the role of teachers and students during the discussion sessions and interviews, the researchers identified what is believed by sarijem about the role of teachers and students is very close with her knowledge of the theory and learning perspective. however, sarijem’s incomplete understanding about the meaning of "teacher as a facilitator" and "students as active constructors" causes what is known as a fallacy of beliefs. when asked about how to facilitate student learning, sarijem expressed a way to help students who are having difficulty and assisted them in finding solutions to problems, while actively connecting with the activity of thinking. the researcher: at the time of facilitating the students, they were expected to participate actively in the learning process. what are the conditions or situations in which the students can be described as active? sarijem: when teachers ask open-ended questions or probing questions so that students can think of the problems encountered. the researcher: so, do you think active in this respect is when students think about the problems through the questions that you asked? sarijem: yes sir, (with accompanying assertion) ... when students are thinking, that means they’re being active, sir. international electronic journal of elementary education vol.8, issue 4, 629-642, june 2016 636 i found that during the interview session, sarijem’s beliefs about teaching have been including her beliefs about learning explicitly. sarijem adopts beliefs about learning in the perspective of a teacher, so when thinking about the best way to learn mathematics, it is always expressed with how to make students learn the best (a related approach to teaching the teacher to make students learn the best). thus, the beliefs held by sarijem about learning mathematics can be said to be a subset of beliefs about teaching mathematics. the plausible reasons for this is because the beliefs about teaching mathematics are subjective or the implicit viewpoint of teachers on various types of teaching, the meaning of teaching and learning, the role of teachers and students in learning, how students learn mathematics and class activities related to teaching mathematics (chan, 2004; ernest, 1989; thompson, 1992). in other words, beliefs about teaching mathematics include beliefs about learning mathematics. reflecting on instructional practices from sarijem’s beliefs and some factors that limit it under certain circumstances, the instructional practices conducted by sarijem reflect on what she believes. sarijem believes that mathematics is a science that is very relevant to the context, the experience and the daily life of human beings, in line with the beliefs about teaching and learning expressed in a way of how to relate mathematics to the context, experience and the student’s daily life. this reflects the beliefs held by sarijem by always trying to initiate the material that she attributed to the context and experience of students. sarijem also tried to mediate between mathematics and the "context" with the use of props and media. sarijem believes that her instructional practices are constructivist orientated because she considers that it integrates the "context" of students into mathematics learning in the classroom. she also felt that the media and props are indicators that she has applied constructivist-orientated practices. these practices are the reflection her beliefs that mathematics is relevant to the context of the students. however, referring to the opinion of lakatos (in ernest, 1991, 1998) and hersh (1997), mathematics is a human activity, in which mathematical objects are in the nature of human knowledge and are a product of human findings. "students are active" based on this view can be defined as the process of reinvention. students build their own knowledge of mathematics as facilitated by their teachers. this is in contrast to what happens in sarijem’s class, the use of media and props are more dominated by teachers as a model. most students act as spectators of what is practiced by the teacher. this is inconsistent with what has been stated that the teacher acts as a facilitator while students are active constructors of knowledge. the instructional practices are more likely to reflect the beliefs held by sarijem that mathematical objects can only be discovered by mathematicians, the only people capable of specifically discovering and formalizing forms of mathematics. mathematics are accepted as true, so that students are asked to be "users", which record what is seen and heard and then just apply it. i identified that sarijem’s knowledge of the philosophy of mathematics and learning theories less profoundly form the fallacy beliefs whose practices refer to what she believed was true. when explaining the solutions to solving problems related the greatest common factor (gcf) and least common multiple (lcm), sarijem uses factor trees for prime factorization. sarijem explained that the rules of using the factor trees start by dividing the number interested with the smallest prime number. in the interview, sarijem explains that such a rule is easily memorized, making it likely students will also find it easy to learn. sarijem states that the rule is fixed, so in order to solve mathematical problems, users must be use rules that already exist. the following is an excerpt of the interview when discussing the process of explaining the solution gcf (24, 32) beliefs and instructional practices in mathematics class / purnomo, suryadi & darwis 637 the researcher: why did you divide by two, why not four or eight? sarijem: actually, it’s not always two, sir. according to my memory, in order to find the gcf and lcm by prime factorization, we should start from the smallest prime number first, sir. so, when it cannot be divided by 2, then it can be continued by dividing by 3 and so on, instead of four or eight, because they are not prime numbers. textbooks, too, start from the smallest prime number anyway, sir. the researcher: so, should we not be numbers other than prime numbers? why is that? sarijem: no sir, because that is the rule, sir. during my own school experience, rules were like that. the researcher: ok. now, i would like to ask why we are looking for the gcf and lcm by finding prime factors? why not factor of other numbers? sarijem: i do not know sir, that's the rule of the finding the gcf and lcm, sir (laughing). sarijem's practices, which emphasized the rigid rules and procedures when explaining the prime factorization, are consistent with her beliefs that mathematics is an accumulation of facts, rules and procedures and then using said knowledge to solve the problem that is accepted as true. what she believes contradicts the constructivist view, which states that mathematics is not discovered, but constructed by humans so that the rules and procedures of mathematics can be created by anyone. thus, mathematical problems can be solved by various methods or approaches that can be undertaken by students. students can be guided to find a canonical sequence of 24 and 32 in their own way. some examples of how using mental strategies can be illustrated as follows (purnomo, 2014). figure 3. the mental strategies factorization i identified that in addition to the previous school experiences, the lack of mathematical knowledge for teaching also became contributing factors which lead to practices that affect sarijem’s approach towards instrumental teaching. sarijem did not yet realize that looking for lcm and gcf can be done using prime factorization epistemologically. i use the term “epistemological content knowledge” as one of the forms “mathematical knowledge for teaching” in this case. epistemological content knowledge requires teachers to reflect epistemologically on mathematical structure as the foundation in establishing and communicating mathematical arguments. for example, the teacher must interpret and able to connect between the prime factorization and the fundamental theorem of arithmetic. i also identified that the practice is carried out by sarijem were more performanceorientated, which is focused on the students’ ability to work on the problems quickly and correctly. the practice also still focused on the textbook as the main reference to teaching and did not dare to abandon the habit of using the rules contained in the book. sarijem international electronic journal of elementary education vol.8, issue 4, 629-642, june 2016 638 admitted that while in school, she was also taught to do so by her teacher, so it feels odd to not follow the rules out of habit. this instrumental teaching resulted in students learning leads to a superficial learning dominated by memorizing facts and rules. consequently, students often experience errors, do not interpret what was done, and lack the sensitivity and ability of a problem solver (purnomo, kowiyah, alyani, & assiti, 2014; yang & wu, 2010). focusing on the interaction between students and teachers, the interaction was dominated by one-on-one dialogue between sarijem and her students. the dialogue was conducted by sarijem using probing questions. high frequencies shown that sarijem would come near to students to seek information through a series of questions that are guiding towards the discussion. sarijem believed that was one way of making the students actively engage in the learning process. sarijem uses questions as a way to assess the experience and knowledge of her students. in other words, the richer the feedback obtained from the students, teachers will increasingly understand the weakness and can continue to improve it. this strategy as suggested by some researchers uses effective questions, and is one of the strategies for integrating assessment into learning (black, harrison, lee, marshall, & wiliam, 2003, 2004; lee, 2006; purnomo, 2015; sumantri & satriani, 2016). however, the peers involvement to provide feedback get less opportunity. the opportunity was not also provided in most of sarijem’s practices in the classroom, that the interaction between students and peers received less emphasis. this fact is not consistent with what she declared that students who engage in collaborative learning always benefit in their learning. in interviews and discussion sessions, sarijem stated that the opportunity to do collaborative learning is difficult because of time pressure and the material must be prepared by the mid-term deadline. furthermore, sarijem also realized that the practice that was planned was constrained by the learning environment, especially the behavior of students in the classroom. sarijem realized that she could not organize the class because of the number of students, which exceeded the quota. for these reasons also, sarijem had to limit the process of discussion between the teacher and the student. conclusions this research examines the beliefs and practices of pre-service elementary school teacher early in her experience teaching mathematics. the findings of this study indicate that (1) beliefs about the nature of mathematics held by sarijem are more traditional than her beliefs about teaching and learning mathematics, (2) the practices that are done by sarijem are primarily more traditional than constructivist, and (3) the instructional practices do not always reflect on the beliefs that sarijem holds. the complex relationship of related beliefs and practices were also encountered by some previous researchers (e.g., barkatsas & malone, 2005; raymond, 1997; shield, 1999). some identified factors restricting the practice of sarijem in mathematics class so that it is not always consistent with her beliefs, including the previous school experience, social norms, mathematical knowledge for teaching, the attitude that dares not to act out of habit, time constraints, high-stakes testing, curriculum, student behavior and the learning environment. further, inconsistencies between beliefs held by sarijem were influenced by the weak knowledge of the philosophy of mathematics and the learning theory. the findings of this study indicate the important role of the philosophy of mathematics, as proven instructional practices done by sarijem more reflected her beliefs about the nature of mathematics. this can also be found in previous studies (e.g., perkkilä, 2003), which reveals that beliefs about the nature of mathematics contribute more strongly than other beliefs. chassapis (2007) proposed some arguments about the philosophy of mathematics beliefs and instructional practices in mathematics class / purnomo, suryadi & darwis 639 that hold a key position in the professional knowledge of mathematics teachers. the first argument confirmed the direct relationship between the philosophy of mathematics and the basic features of mathematics education. the second argument is that the ideas, views, conceptions, or beliefs about mathematics teachers, teaching, and learning, reflected or related to the philosophy of mathematics implicitly. the third argument stated the undoubted assumption that the philosophy of mathematics is directly related to understanding the contents in mathematics as knowledge to be taught. it can be used as input to the curriculum of the elementary school teacher education program to integrate the philosophy of mathematics into the philosophies that are commonly used in the current curriculum. continuous reform towards higher education curriculum in indonesia generally and education studies program for elementary school teachers in particular have become a necessity. it is important that in addition to the knowledge of content, mathematical content knowledge for teaching also needs to be stressed. it is none other than the education studies program that is designed for the curriculum of elementary school teachers, who place more emphasis on aspects of mathematical content knowledge and is more oriented to high-level cognitive abilities, but the integration of mathematical knowledge for teaching is weak. the importance of practical experience for pre-service teachers also needs to be addressed by policy makers at the higher education level, by integrating field experience on an ongoing basis and not just at the end of their coursework. it is useful to provide an opportunity for pre-service teachers related to the self-reflection of instructional practices in the classroom as the way to develop knowledge and beliefs or maybe refine them in the future. this can be done in several ways, such as (1) involving pre-service teachers in selfstudy to either himself or the teacher's example as the subject, (2) making the program provide an opportunity for pre-service teacher ' observations in schools, and (3) involving pre-service teachers in research activities on an ongoing basis. • • • acknowledgements the authors would like to thank the indonesian directorate general of higher education have supported and funded this research in the scheme of "fundamental research grant" with the contract number: 113 / k3 / km / 2015. international electronic journal of elementary education vol.8, issue 4, 629-642, june 2016 640 references applefield, j. m., huber, r., & moallem, m. (2000). constructivism in theory and practice: toward a better understanding. the high school journal, 84(2), 35-53. arı, e., tunçer, b. k., & demir, m. k. (2016). primary school teachers' views on constructive classroom management. international electronic journal of elementary education, 8(3), 363-378. barkatsas, a. t., & malone, j. (2005). a typology of mathematics teachers’ beliefs about teaching and learning mathematics and instructional practices. mathematics education research journal, 17(2), 69-90. beswick, k. (2012). teachers' beliefs about school mathematics and mathematicians' mathematics and their relationship to practice. educational studies in mathematics, 79(1), 127-147. doi: 10.1007/s10649-011-9333-2 black, p., harrison, c., lee, c., marshall, b., & wiliam, d. (2003). assessment for learning: putting it into practice. maidenhead, u.k.: open university press. black, p., harrison, c., lee, c., marshall, b., & wiliam, d. (2004). working inside the black box: assessment for learning in the classroom. phi delta kappan, 86(1), 9-21. chan, k.-w. (2004). preservice teachers' epistemological beliefs and conceptions about teaching and learning: cultural implications for research in teacher education. australian journal of teacher education, 29(1), 1. chassapis, d. (2007). integrating the philosophy of mathematics in teacher training courses: a greek case as an example. in f. karen & j. p. v. bendegem (eds.), philosophical dimensions in mathematics education (vol. 42, pp. 61-79). new york: springer. de corte, e. (2003). intervention research: a tool for bridging the theory-practice gap in mathematics education. paper presented at the proceedings of the international conference, the mathematics education into the 21st century project, brno czech republic. ernest, p. (1989). the impact of beliefs on the teaching of mathematics. in p. ernest (ed.), mathematics teaching: the state of the art (pp. 249–253). new york: falmer. ernest, p. (1991). the philosophy of mathematics education. london: routledge falmer. fennema, e., carpenter, t. p., & franke, m. l. (1992). cognitively guided instruction. ncrmse research review: the teaching and learning of mathematics, (eric document reproduction service no. ed 372929, 1, 2). national center for research in mathematical sciences education madison, wi. furinghetti, f., & pehkonen, e. (2002). rethinking characterizations of beliefs. in g. c. leder, e. pehkonen & g. törner (eds.), beliefs: a hidden variable in mathematics education? (vol. 31, pp. 39-57). dordrecht, the netherlands: kluwer academic publishers. geiger, v., & goos, m. (2006). living in the gap: a tale of two different types of researchers. in p. grootenboer, r. zevenbergen & m. chinappan (eds.), identities, cultures and learning spaces, proceedings at the 29th annual conference of the mathematics education research group of australasia (pp. 254-261). sydney: merga. goldin, g. a., epstein, y. m., schorr, r. y., & warner, l. b. (2011). beliefs and engagement structures: behind the affective dimension of mathematical learning. zdm, 43(4), 547-560. doi: 10.1007/s11858-011-0348-z hersh, r. (1997). what is mathematics, really? oxford, new york: oxford university press. kundi, g. m., & nawaz, a. (2010). from objectivism to social constructivism: the impacts of information and communication technologies (icts) on higher education. journal of science and technology education research, 1(2), 30-36. leder, g. c., & forgasz, h. j. (2002). measuring mathematical beliefs and their impact on the learning of mathematics. in g. c. leder, e. pehkonen & g. törner (eds.), beliefs: a hidden variable beliefs and instructional practices in mathematics class / purnomo, suryadi & darwis 641 in mathematics education (vol. 31, pp. 95-114). dordrecht, the netherlands: kluwer academic publishers. lee, c. (2006). language for learning mathematics: assessment for learning in practice. berkshire, england: open university press. malone, j. (2000). bridging the gap: a challenge for the dual community. in j. bana & a. chapman (eds.), mathematics education beyond 2000, proceedings of the 23rd annual conference of the mathematics education research group of australasia (pp. 27-36). perth, wa: merga. mcleod, d. b. (1992). research on affect in mathematics education: a reconceptualization. in d. a. grows (ed.), handbook of research on mathematics teaching and learning (pp. 575-596). new york: macmillan. pajares, m. f. (1992). teachers’ beliefs and educational research: cleaning up a messy construct. review of educational research, 62(3), 307-332. perkkilä, p. (2003). primary school teachers’ mathematics beliefs and teaching practices. in m. a. mariotti (ed.), proceedings of the third conference of the european society for research in mathematics education (pp. 1-8). bellaria, italy. ponte, j. p. d. (1994). knowledge, beliefs, and conceptions in mathematics teaching and learning. paper presented at the proceedings of the fifth international conference on systematic cooperation between theory and practice in mathematics education. ponte, j. p. d., & chapman, o. (2006). mathematics teachers’ knowledge and practices. in a. gutiérrez & p. boero (eds.), handbook of research on the psychology of mathematics education: past,present, and future (pp. 461-494). rotterdam: sense publisher. purnomo, y. w. (2014). serial matematika untuk pgsd: bilangan cacah dan bulat. bandung: alfabeta. purnomo, y. w. (2015). pengembangan desain pembelajaran berbasis penilaian dalam pembelajaran matematika. cakrawala pendidikan, 34(2), 182-191. purnomo, y. w., kowiyah, alyani, f., & assiti, s. s. (2014). assessing number sense performance of indonesian elementary school students. international education studies, 7(8), 74-84. doi: 10.5539/ies.v7n8p74 raymond, a. m. (1997). inconsistency between a beginning elementary school teacher's mathematics beliefs and teaching practice. journal for research in mathematics education, 28(5), 550-576. schoenfeld, a. h. (1992). learning to think mathematically: problem solving, metacognition, and sense making in mathematics. in d. a. grouws (ed.), handbook of research on mathematics teaching and learning (pp. 334-370). new york: macmillan. shield, m. (1999). the conflict between teachers’ beliefs and classroom practices. in j. m. truran & k. m. truran (eds.), making the difference: proceedings of the 22nd annual conference of the mathematics education research group of australasia (pp. 439-445). adelaide: merga. smith, t. (2000). bridging the research-practice gap: developing a pedagogical framework that promotes mathematical thinking and understanding. mathematics teacher education and development, 2(1), 4-16. sriraman, b. (2004). the influence of platonism on mathematics research and theological beliefs. theology and science, 2(1), 131-147. sumantri, m. s., & satriani, r. (2016). the effect of formative testing and self-directed learning on mathematics learning outcomes. international electronic journal of elementary education, 8(3), 507-524. tatto, m. t., schwille, j., senk, s., ingvarson, l., peck, r., & rowley, g. (2008). teacher education study in mathematics (teds-m), conceptual framework. teacher education and development international study center, college of education, michigan state university, east lansing, mi. international electronic journal of elementary education vol.8, issue 4, 629-642, june 2016 642 thompson, a. g. (1992). teachers' beliefs and conceptions: a synthesis of the research. in d. a. grouws (ed.), handbook of research on mathematics teaching and learning (pp. 127-146). new york: macmillan publishing co, inc. törner, g. (2002). mathematical beliefs—a search for a common ground: some theoretical considerations on structuring beliefs, some research questions, and some phenomenological observations. in g. c. leder, e. pehkonen & g. törner (eds.), beliefs: a hidden variable in mathematics education? (vol. 31, pp. 73-94). dordrecht, the netherlands: kluwer academic publishers. vacc, n. n., & bright, g. w. (1999). elementary preservice teachers' changing beliefs and instructional use of children's mathematical thinking. journal for research in mathematics education, 30(1), 89-110. white-fredette, k. (2010). why not philosophy? problematizing the philosophy of mathematics in a time of curriculum reform. the mathematics educator, 19(2). yang, d.-c., & wu, w.-r. (2010). the study of number sense: realistic activities integrated into third-grade math classes in taiwan. the journal of educational research, 103(6), 379-392. microsoft word 4_kostelecka_3_1-2 approved by autor international electronic journal of elementary education vol. 3, issue 1, october, 2010. issn:1307-9298 copyright © iejee www.iejee.com home education in the post-communist countries: case study of the czech republic yvona kostelecká ∗∗∗∗ faculty of education, charles university in prague, czech republic abstract the paper analyzes the emergence of home education in european post-communist countries after 1989. the case of the czech republic representing the development and characteristic features of home education in the whole region is studied in detail. additional information about homeschooling in other post-communist countries are provided wherever they are available in order to provide a more comprehensive picture of the issue. the driving forces and history of home education after 1989 are described. current homeschooling legislation is analyzed with special attention paid to the processes of the legal enrolment of individuals into home education, supervision and assessment of educational results. the article concludes that despite the existence of country-specific characteristics, many features of home education in post-communist countries are similar. these generally include the rather strict regulation of home education and the high importance of schools as both gate-keeping and supervising institutions. keywords: home education, homeschooling legislation, czech republic, post-communist countries introduction the educational system is traditionally strongly influenced by the political system of the country and by its prevailing culture. that is the reason why educational systems in the czech republic and many other post-communist european countries still have some specific features that distinguish them in a number of ways from the countries of “old europe”. these specific ∗ correspondence: yvona kostelecká, faculty of education, charles university, hloubětínská 26 praha 9 198 00 czech republic. phone: +420 221 900 727. e-mail: yvona.kostelecka@pedf.cuni.cz international electronic journal of elementary education vol.3, issue 1, october,2010 30 features also include dealing with different forms of education including home education. while literature about home education in north america, western europe or australia is relatively plentiful (beck, 2002, 2008; dalahooke, 1986; meyer, 1999; petrie, 1995, 2001; priesnitz, 2003; ray, 1994, 1997; rothermel, 2002; thomas, 1999), information about homeschooling in the post-communist countries is scarce. it is partly due to the simple fact that in most of the post-communist countries home education represents a quite new phenomenon. educational researchers did not have enough time for its proper evaluation. in many post-communist countries the number of homeschooling families remain extremely limited, which left home education unnoticed even by local specialists for alternative education. this text primarily concentrates on the situation in the czech republic where much of both of the quantitative and qualitative information about home education is available. the sources of such information include the analysis of the legislation, several empirical studies about home education that had been conducted so far, the interviews with the promoters of home education and parents of homeschooled children, and the participant observations of the author of the article in the evaluation of home education and the evaluation of the learning outcomes of home educated children. the development of home education in the czech republic will serve as a model case, representing the development of home education in the postcommunist countries. detailed information about home education in the czech republic will be supplemented by information about home education in other post-communist countries wherever such information is available. in such cases, the main sources of information are the web pages of local homeschooling associations. the establishment of home education in the czech republic after 1989 the move towards the legalization of home education in the czech republic was, similarly to the other post-communist countries, initiated by a change of political regime after the fall of the iron curtain at the end of the 1980s. these historical events led to drastic changes put into motion by legislation that was aimed at introducing the principles of democracy, decentralization, diversity and effectiveness (adámková, 2007). soon after the regime change new education acts were adopted in the post-communist countries, changing local school systems to fit the needs of newly created democratic societies. the school systems that served as a tool for the unification of individuals, their values and attitudes during the communist era were transformed in a way that would enable more individualized and variable outcomes in education. home education in the post-communist countries / kostelecká 31 soon after the “velvet revolution” in 1990, the czech parliament adopted a new amendment of the education act1, which officially abolished the “unified school system”. this opened the door for more variety in the forms of education available. the establishment of private and denominational schools was allowed, which de facto ended the state monopoly on the providing of educational services. this amendment, however, did not mention home education at all, as there was effectively no demand for it at the time of its adoption. the situation in other postcommunist countries differs somewhat in this respect. in poland, the legal possibility to educate children at home was introduced by the education act2 that was adopted by the first post-communist parliament in 1991. in russia as well, the legal possibility for home education to exist was already established in the 1992 law on education3 (fladmoe & karpov, 2002). in estonia, the 1992 education act allowed compulsory schooling to be carried out at home (leis, 2005). in hungary, the 1993 education act4 introduced the status of a “private student” that could be educated individually, out of school. in contrast with poland, russia, estonia and hungary home education was practically an unknown entity to both the experts and the public in the czech republic until about 1996 and in romania until about 20005. the idea of home education also came rather late to slovakia, where an educational act legalizing home education was adopted in 20086. in spite of the fact that reforms of the czech educational systems were generally directed towards more decentralization, strengthening the role of municipalities in primary education, and increasing the educational, administrative and economic autonomy of schools (marvanová, 2001), advocates of home education were afraid7 that home education would be considered to be a marginal issue by the relevant politicians. in turbulent times of social and economic transformation, politician sought to reach consensus in the “key” issues of the educational reforms, but the legalization of home education definitely did not belong to such issues. many politicians, even those who personally favoured home education, avoided openly supporting the idea or even being personally connected to the issue, because the prevailing opinion of both the experts and the general public was not known and they did not want to be accused of supporting extremists’ views8. 1 the amendment of education act no. 29/1984 coll. adopted as act no. 171/1990 coll. (zákon ze dne 3. května 1990, kterým se mění a doplňuje zákon č. 29/1984 sb., o soustavě základních a středních škol (školský zákon) 2 educational act no. 95/1991 (dz.u. 1991 nr 95 poz. 425 ustawa z dnia 7 września 1991 r. o systemie oświaty) 3 educational act no. 3266-1 (закон об образовании от 10.07. 1992 n 3266-1) 4 act no. lxxix of 1993 on public education (1993. évi lxxix. törvény a közoktatásról). 5 web pages of asociańia home schooling românia (http://www.homeschooling.ro/ro/about.html) 6 education act no. 245/2008 coll. (245/2008 z. z.zákon o výchove a vzdelávaní (školský zákon) a o zmene a doplnení niektorých zákonov z 22. mája 2008) 7 interview of author with petr plaňanský, later member of the board of the association for home education, july 12th, 1998. 8 interview of author with jiří tůma, later the president of the association for home education, july 10th, 1998. international electronic journal of elementary education vol.3, issue 1, october,2010 32 the effort to gain the support of the public and from at least some politicians encouraged supporters of home education to engage in a higher level of public activity. supporters of home education started to form various formal and informal groups9 that worked to raise public awareness of home education and build a positive image of home education in czech society. the fact that groups of active promoters of home education in the czech republic included several respected experts from among lectures at universities to teachers and directors of elementary schools proved to have a very positive impact on the promotion of the case in its initial phase10. in addition to prestige, these people also had the cultural and social capital necessary for successful public relations activities. the increased activity of advocates for home education in the czech republic bore fruit surprisingly quickly. after short but intensive lobbying the promoters of home education were able to gain the favour of some politicians and upper level officials within the ministry of education. the unsolved questions remained, however, of how to make home education legal in a situation in which no real chance for the adoption of a new education act, or the amendment of the existing law, could be realistically expected any time soon. in spite of the problem, the ministry of education created a solution that made home education “legal” very quickly and in a quite unconventional way. the ministry used its executive power and enabled “experimental examination of home education as the different form of the elementary schooling” (kolář, 2000). the experiment was designed to last for five years11. starting september 1st 1998, home education was made a legal alternative to school attendance. however, it was enabled only as an educational experiment and only for children in the first five grades of elementary school. the declared aims of the experimental examination were to find a functional model of home education, to make an assessment of its outcomes, and to examine all potential risks that might eventually be connected with this type of education. home education during the period of its experimental examination the ministry of education first had to define a target group for home education and to determine the “guarantor” that would oversee and make 9 the association for home education (asociace pro domácí vzdělávání) was established several years later, in 2002. it serves as the umbrella organization unifying different groups of home educators in the czech republic and serves as a platform for the official negotiation of home educators with the public administration. similar associations came into existence in other post-communist countries. the polish association for home education (stowarzyszenie edukacji domowej) was established in 2003. the estonian centre for home education (mtu eesti koduõppe keskus) has been active in estonia since 2005. the romanian homeschool association (asociańia home schooling românia) was established in romania in 2002 while the homeschooling friends associastion (společnosť priateľov domácej školy na slovensku) is active in slovakia, the homeschooling association in bulgaria (училище в дома) and the association for home education (šeimos mokyklu namuose draugija) in lithuania. 10 interview of the author with jiří tůma, later the president of the association for home education, july 10th, 1998. 11 the experiment in fact lasted several more years than was formerly expected, until the adoption of the new education act. home education in the post-communist countries / kostelecká 33 an assessment of the experiment in practice. the ministry decided that the target group would be defined solely by age only children of early primary school age (from 1st to 5th grade) were allowed to participate in the experiment. in this respect, the czech solution was rather unusual. only slovakia12 and estonia (until 2008) (rand & leitmaa, 2007) similarly restricted home education to children in the first years of compulsory schooling. the czech ministry also chose selected elementary schools as guarantors of the practical realization of the home education experiment (kolář, 2000). in contrast to the previous one, this solution was very typical and was adopted in many other post-communist countries: poland13, estonia (leis, 2005), slovakia14, slovenia15, hungary16 and romania17. but the role of schools here is even more prominent. they are not only designed as institutions that are to supervise the actual home education: they also received the authority to decide whether a particular family can homeschool their own children. parents who are interested in home education must apply to schools for permission to educate their children at home, as in for example poland18, estonia (leis, 2005) and slovakia19. in some states, like slovakia20, such permission can be obtained only from a local school in the school district where applicants live. in poland, after 1991 only state schools could issue such permission. the first private school was granted such authority no sooner than september 2004 (dueholm, 2006). in the first year of the experiment (school year 1998/1999), the czech ministry of education granted authority to issue permission for homeschoolers only to two schools. both supervising schools were denominational schools. during the course of the experiment two more schools (public schools) were chosen to be supervising schools. many powers were delegated to the directors of participating schools. directors made decisions about the enrolment of children into the experiment, as well as made ones concerning the form of evaluations of educational outcomes of home educated children. the state, represented by the ministry of 12 education act no. 245/2008 coll. (zákon o výchove a vzdelávaní (školský zákon) a o zmene a doplnení niektorých zákonov z 22. mája 2008) 13 educational act no. 95/1991 (dz.u. 1991 nr 95 poz. 425 ustawa z dnia 7 września 1991 r. o systemie oświaty) 14 education act no. 245/2008 coll. (zákon o výchove a vzdelávaní (školský zákon) a o zmene a doplnení niektorých zákonov z 22. mája 2008) 15 elementary school act no. 3535/2006 [zakon o osnovní šoli – zosn–upb3 (uradni list rs, št. 81/06 z dne 14. julija 2006)] 16 act no. lxxix of 1993 on public education (1993. évi lxxix. törvńy a közoktatásról) 17 reply of secretary of state of ministy of education and research gabriela pastor on interpelation of mp adrian-sirojea mihei from march 10, 2008. available online at webpages of romanian parlament (http://www.cdep.ro/pls/steno/steno.stenograma?ids=6451&idm=9,01&idl=1) 18 educational act no. 95/1991 (dz.u. 1991 nr 95 poz. 425 ustawa z dnia 7 września 1991 r. o systemie oświaty) 19 education act no. 245/2008 coll. (zákon o výchově a vzdelávaní (školský zákon) a o zmene a doplnení niektorých zákonov z 22. mája 2008) 20 education act no. 245/2008 coll. (zákon o výchově a vzdelávaní (školský zákon) a o zmene a doplnení niektorých zákonov z 22. mája 2008) international electronic journal of elementary education vol.3, issue 1, october,2010 34 education, did not set any special constraints for the practical realization of home education aside from the general requirement for the home educating parents to meet the educational standards valid for elementary schools, the duty for supervising schools to evaluate outcomes of home education twice every school year and for them to provide the ministry with a written report evaluating the course of the experiment at the end of every school year (kostelecká, 2003). schools in some other post-communist countries received a level of authority very similar to this (dueholm, 2006; leis 2005). the first families enrolled in the home education experiment the basic information about the families that were allowed to homeschool their children within the framework of the experiment are available from the official registers kept by the supervising schools and from evaluating reports that participating schools had to submit to the ministry of education (zpráva… 2000, 2001, 2002, výroční zpráva, 2002). some specific information about participants of the experiments was obtained by several empirical studies that used surveys and interviews with families (kostelecká, 2003; marvánová, 2001; mifková, 2009). it was possible to identify two different types of families from the available data that differ in the motives which led them to participate in the experiment. the first type consisted of families that decided to participate in the experiment “under the pressure of specific circumstances”. the second type consisted of families who can be labelled as “devoted home educators”. the former type is represented by families with children who had already had negative experiences associated to attendance at school (too much stress in school, unsuitable approach by teachers to children, problems with classmates, problems in school due to frequent absences, various health problems, inability of school to meet specific requirement of children, too time consuming when commuting to a suitable school, etc.). parents of such children came to the conclusion that their available school was not able to fulfil their expectations. some of them were afraid that continuing to attend the school could have a serious negative impact on the further development of their children. parents of such children could hardly be considered home education enthusiasts. they usually understood home education as an escape from their school problems, as one of a number of possible ways to solve their problems. if they had an acceptable school for their children they would probably not consider home education at all. in some cases, members of this group considered the decision to participate in the home education experiment as the temporary solution necessary for a period before they found a suitable school for their kids. the motives of the devoted home educators were quite different. their children were not forced out of school due to any specific problems. in the majority of cases they never attended school. children that attended school had practically no problems there. the main motive for the participation in the home education experiment was an attempt by parents to preserve a high degree of influence over the education of their children. such parents home education in the post-communist countries / kostelecká 35 tended to stress the building and preservation of close relations between parents and children within the family. in this regard, they often mentioned their desire to maintain more influence over the socialization of their kids, the need for more space for the transmission of family values, individualized learning, the possibility to provide children with more time to reach social maturity before they leave their family and join their school class, or to provide children with more time for their individual interests and hobbies as reasons for participating in the experiment. some parents of this group expressed dissatisfaction with the values that regular schools espouse, or complained that children spent an inadequate amount of time outside of home due to long commuting times. the collected information also suggests that devoted home educators were overrepresented among the first wave of participants21 in the experiment (kostelecká, 2003). such families deviated from the average czech family in several aspects the number of children in families was substantially higher and many parents declared themselves to be devoted christians. both fathers and mothers of the first homeschoolers were had a higher level of education than their counterparts of the same age. in spite of the high level of education, the majority of mothers stayed at home. compared to average czech families that typically have two incomes, the home educating families had a slightly lower average income in spite of the high level of education of the parents. it is clear that home educating families who were a part of the first wave of homeschoolers were typically representative of the traditional family. while husbands spent much of their time outside of the family due to earning money to support the family, their wives took care of the household and were mainly responsible for the education of the children. later, the number of children who had previous negative experiences while attending school and ones with specific learning problems increased among homeschoolers. the directors of supervising schools observed that such families have somewhat different needs than that of the former group. parents, who had opted for home education as a way to avoid their children’s problems at school, tended to contact schools much more frequently to seek help or professional advice (kostelecká, 2003). it is possible to distinguish rather distinct groups of homeschoolers in many post-communist countries. but these may not necessarily be in the group of homeschoolers avoiding school under pressure of specific circumstances or the group of devoted home educators. more often than not, 21 a questionnaire including 58 questions (the majority of which were multiple choice questions with some open-ended questions) was sent by post to all 62 families that participated in the first year of the experiment in early 1999. contacts to home educaing families were obtained from two supervising schools. after two reminders (one written and the other by a telephone call), 35 questionnaires altogether were collected. questions in the questionnaire were answered exclusively by parents of home educated children. information obtained by the questionnaire was subsequently supplemented by interviews with parents of 12 home educated children who filled in the questionnaire. interviews were conducted in 2003. international electronic journal of elementary education vol.3, issue 1, october,2010 36 it is possible to distinguish home education for children with specific learning needs and home education for common children. sometimes, educational laws explicitly make such a distinction. the russian law22 distinguishes between home education and family education. home education is intended for children with special learning needs. only parents whose children suffer from illnesses specified by legislation can obtain permission to do home education for their children. an individual curriculum is prepared for the children of successful applicants. parents can choose between whether their children obtain individualized instructions in school or that they themselves teach their children at home, with the help and support of teachers. on the other hand, family education “is designed for children whose parents do not want children to attend school for any reason. such parents could either teach their children themselves or they could hire a private teacher. in both cases they have to follow standard school curricula” (fladmoe & karpov, 2002). estonian law also makes a distinction, treating homeschooling for medical reasons differently than home education that is practiced because parents wish to do so. in the former case, home education is possible only if the nearest school is not able to guarantee the inclusion of a child and when a doctor considers home education to be necessary for the child. the child who receives permission for home education for medical reasons is then assigned to a teacher who works out an individual curriculum plan and is responsible for educating the child (leis, 2005). in slovenia as well, law23 explicitly distinguishes between the home education of children with special learning needs and the home education of other children. similar situation can be observed in slovakia where law24 specifies individual education for children whose health does not allow them to participate in regular schooling. parents of such children must submit application accompanied by the opinion of the paediatrician. in specific post-communist countries (e.g. bulgaria) home education is allowed only to children with special physical or mental needs, but homeschooling for most other families is still prohibited. in bulgaria, families who homeschool their children clandestinely could be subject to fines (porumbachanov, 2007). practical implementation of the experiment in individual supervising schools the practical organization of the home education experiment in the czech republic was substantially different under the supervision of different schools (zpráva, 2000, 2001, 2002). the director of a supervising school had the authority to determine specific forms of organization of home education. the enrolment of children into the experiment was usually organized in several steps. in the first step, applicants were asked to write an official application and to fill in a questionnaire. the second step was an interview 22 educational act no. 3266-1 (закон об образовании от 10.07. 1992 n 3266-1) 23 elementary school act no. 3535/2006 [zakon o osnovní šoli – zosn–upb3 (uradni list rs, št. 81/06 z dne 14. julija 2006)] 24 education act no. 245/2008 coll. (zákon o výchově a vzdelávaní (školský zákon) a o zmene a doplnení niektorých zákonov z 22. mája 2008) home education in the post-communist countries / kostelecká 37 of both parents and children with the school director. the aim of the interview was to find out the motives for home education and assure that the parents are able to provide adequate education for their children. schools were interested in the education of the parents, the functionality of a family, and the material and space conditions for home education. some schools wanted to be assured that the children would have enough contact with other children outside of the family. some schools required a written opinion from a physician. in individual cases schools also required a written opinion from an accredited educational and psychological counsellor. only a minimum of applications were discarded. most applicants were successfully enrolled in the home education experiment, and signed a written agreement with a respective supervising school. sixty-two children were enrolled into the experiment in its first year. the number of children enrolled gradually increased and reached 307 in the fifth year of the experiment25 (nováková, brant & tupý, 2002). each participating school had its own ideas of how home education should ideally be. one of the two denominational schools adopted an approach in which parents would have the main responsibility for the home education of their children. as a consequence, the school tended to interfere with home education in families less than that of the other participating schools. this school considered itself primarily to be a place where parents doing home education can get help and advice and where the educational outcomes of home education are to be evaluated. the school offered consultations with experienced teachers and participation in activities organized by school. the cooperation between the school and the parents during the school year was based exclusively on a voluntary basis. the inexistence of any time consuming requirements and demands from a supervising school made more room for activities organized by the participating families themselves. parents and children started to cooperate amongst themselves, establishing various working groups for similarly aged children. some working groups worked on a long-term basis and were relatively stable (e.g. working groups specialized on the learning of foreign languages). other working groups were constituted only on a temporary basis, such as children working together to complete some specific project (environmentalist, historical, natural science experiments, etc., in accordance with the interests and needs of the children. some children participated only in one working group; other children worked in several working groups simultaneously (zpráva, 2000, 2001, 2002). 25 in bulgaria, for example, only 15 to 20 families educated their children at home in 2007 (porumbachanov 2007). in estonia, the number of home educated children reached 1008 in the 2005/2006 school year, but only 70 were reported as home educated by parental wish, while the other 938 were home educated for medical reasons (leis tiia 2005). in hungary, the number of home educated children reached 6830 in the 2006/2007 (kolasinska et al 2007). in slovakia, the numner of home educated children is not known (email correspondence of author with jana pajgerová, vicecharman of homeschooling friends associastion, june 20, 2010). international electronic journal of elementary education vol.3, issue 1, october,2010 38 the other supervising schools understood their role in the experiment differently. they wanted experienced teachers to guide parents doing home education to assure that children would be provided with a good education. the schools required the obligatory participation of families in regular consultations, pushed parents to keep detailed records about the course of education at home, and required regular evaluation of educational outcomes by testing in schools. the relatively tight control of schools over parents choosing home education were motivated by an idea that it is necessary to ensure the early diagnostics of possible problems and, thus, prevent any possible didactic mistakes made by parents. this way of organizing home education proved to be quite time consuming for the parents involved. according to galisova (1999), one school even required parents to keep special records separately for individual subjects to which a detailed time schedule of the learning process had to be recorded, including all activities, exercises and tests. parents were required to prepare in written form all of the lessons to be given and to evaluate regularly in writing the educational progress of the children in individual subjects. in addition, children were required to participate regularly (once or twice a week) in some lessons in class while parents with children were obligated to meet with consultants from amongst school teachers at least once a month. the task of the consultants was to check the progress of the children, to identify eventual problems and to propose a solution. in addition to obligatory requirements, home educated children were offered additional voluntary participation in regular classes in the school (nováková et al., 2002). children from higher grades formed a group and were taught some difficult topics by a teacher in the school. the aim was to help parents with the teaching of difficult subjects and to prepare the children for a smooth transition from home education to regular school attendance (výroční zpráva, 2002). the process of the evaluation of home education also differs substantially under the guidance of different schools. school were legally required by the ministry of education to evaluate the progress of home educated children twice a year, but the form of evaluation was at the discretion of their directors. one school opted for individualized evaluation that took the form of a discussion between teacher(s) and the child, who was always accompanied by parent(s). the discussion was aimed towards checking whether or not the child was educated adequately according to his or her age and individual abilities. great attention was paid to the materials that the child was required to bring. these materials included various textbooks, notebooks, review sheets, paintings, photos, and other materials, such as a student portfolio, that were for documenting the children’s progressive learning growth. parents were required to provide the evaluator with their written evaluation report of the child’s educational progress and written records in which the topics of study were documented on a weekly basis. all materials provided by the child and his or her parents served as the background information for the official evaluation report (gališová 1999; nováková, et al., 2002; zpráva, 2000, 2001, 2002). home education in the post-communist countries / kostelecká 39 the other schools evaluated the educational progress of home educated children with biannual examinations that consisted of written tests and the oral examination of children. examination covered various subjects mathematics, the czech language, reading, oral presentation, and knowledge from other obligatory subjects (history, geography, natural sciences, etc) (nováková et al., 2002). one of the schools took into account that the home educated children had individualized curricula. the school, therefore, sought to prepare individually tailored evaluation tests, which proved to be extremely difficult and time consuming for the evaluators. other schools used the same tests for all children of the same grade, which pushed the parents of home educated children to adjust their curricula to the requirements of the test. the final official evaluation report was based exclusively on the results of the test and oral examinations and was written by teachers/evaluators from schools (nováková et al., 2002; výroční zpráva, 2002; zpráva, 2000, 2001, 2002). although it is hardly possible to claim that home educated children were educated without proper control by the state during the experiment, such as close supervision by schools, frequent terms of evaluations, etc., the ministry of education suggested new conditions for the experiment in 2001 (podmínky, 2001). these new conditions were aimed towards tightening the control the state had over home educating families and assumed the reduction of powers of school directors. the plans of the ministry were opposed by participating schools and were also fiercely refused by home educating families and their associations. finally, after some negotiation with the involved parties, the ministry decided not to change the conditions of the experiment. this debate over the scope of the control the state has over home education, however, influenced the new education act that was being prepared. home education under the new education act six years after the home education experiment had started the new education act was adopted by the czech parliament (act no. 561/2004 coll). the act does not use the term “home education”, but instead introduces “individual education”26 as a different way of how to meet the requirement of compulsory school attendance. the directors of elementary schools have the authority to permit home education to children who are at the lower primary school age (1st to 5th grade). parents have to submit an application that includes the motives for individual education, school diplomas proving that the people who would be primarily educating the children (not necessarily the parents) have at least completed secondary education, a description of the material and space conditions for educating at home, the list of textbooks and educational material that are to be used during home 26 terms that differ from “home education” are also used in other post-communist countries. for example slovak law speaks about “individual education”, lithuanian about “self education” or “independent studies”, while russian about “family education”. international electronic journal of elementary education vol.3, issue 1, october,2010 40 education, a written opinion by an accredited educational and psychological counsellor, amounts other things. home education can only be permitted if applicants have serious reasons, provided that all other requirements of the law are fulfilled. the law, however, does not specify which reasons should be considered serious. the law gives authority to schools to verify information provided by the applicants but does not suggest how to do that. requirements of the law are rather detailed but they are not uncommon compared to some of the other post-communist countries. both lithuanian and slovak laws set similar requirements for applicants. in fact, slovak law27 is even stricter and explicitly defines who is responsible for the examination that determines whether all of the requirements have been fulfilled. the quality of education of homeschooled children can be examined by the state school inspection office. parents are obliged to allow in-site inspections at home. moreover, the educator of homeschooled children can only consist of certified elementary school teachers. the czech educational act requires that child must be examined twice during a school year. a similar frequency of examinations (one or twice a year) is required in poland28, estonia (leis, 2005), hungary29, slovenia30 and slovakia31. the use of the word “examination” instead of “evaluation” is a problem as it implies a rather quantitative method of evaluation by onetime tests. the strict requirement of the bi-annual testing of knowledge in a broad range of subjects to some extent contradicts one of the key reasons for home education – the possibility of adjusting curricula to the individualized needs and abilities of particular children. although the rules of participation in home education seem to be quite strict when the requirements set by law are considered, it is not so difficult to obtain permission for home education in practice. in the czech republic, parents have complete freedom to choose an elementary school for their children. therefore, parents interested in home education tend to submit an application to schools that are known as home education friendly. such schools offer very helpful conditions to potential applicants and are willing to supervise families from all regions within the czech republic. although the education act allows the home education of children in the first five grades of primary school only, a new opportunity was created for children at an older age (6th to 9th grade). since september 1st 2007, the home education of older children has taken place, this time also as an 27 education act no. 245/2008 coll. (zákon o výchově a vzdelávaní (školský zákon) a o zmene a doplnení niektorých zákonov z 22. mája 2008) 28 educational act no. 95/1991 (dz.u. 1991 nr 95 poz. 425 ustawa z dnia 7 września 1991 r. o systemie oświaty) 29 act no. lxxix of 1993 on public education (1993. évi lxxix. törvńy a közoktatásról) 30 elementary school act no. 3535/2006 [zakon o osnovni šoli 3535– zosn – upb3 (uradni list rs, št. 81/06 z dne 31. 7. 2006)] 31 education act no. 245/2008 coll. (zákon o výchově a vzdelávaní (školský zákon) a o zmene a doplnení niektorých zákonov z 22. mája 2008) home education in the post-communist countries / kostelecká 41 experimental examination under the supervision of several schools and designed by the ministry of education. conclusions some features of home education in the post-communist countries are similar while other features are country specific. in all countries, there was no room within the law for home education during the communist era; the first attempts to introduce home education into national educational systems appeared only after the fall of the iron curtain at the end of 1980s. in some countries, the legal possibility for home education was incorporated into the first post-communist laws concerning education. in other postcommunist countries it took several years more before home education was made legal, and yet in other countries the gradual process of the legalization of home education is not completely finished. in some post-communist countries home education is open basically to all children while in other post-communist countries laws only allow access to home education to selected groups of children – typically ones with special physical or mental needs, or to the youngest children below a specific age. in most post-communist countries laws concerning education require obligatory school attendance. the law assumes that education can and should be acquired through a school. the wording used in laws does not leave much space for the idea that education could be obtained anywhere outside of school. this poses an obvious problem for home education. in contrast to countries where laws concerning education demand obligatory education, the legalization of home education in the post-communist countries is more complicated. special provisions of laws that settle the apparent conflict between obligatory school attendance and the intended legality of home education are necessary. in most post-communist countries, children are not legally entitled to home education; nothing like the right to home education exists. to be allowed to homeschool their children, parents are obliged to submit an application and obtain permission, usually from schools. the successful applicants must meet a broad range of criteria that are set by the state, eventually by the school which is authorized to issue the permission. the decision about whether criteria were met by the applicant is done by the directors of schools. especially when criteria are stated vaguely by the law (e.g. parents have to have serious reasons for home education) or when criteria are set directly by the school, the decision is in fact exclusively in the hands of the school director. thus, paradoxically, parents who want to educate their children outside of school must ask the school for permission to do so. this puts parents into a very submissive position vis-à-vis school in countries where only district schools where the child resides may issue permission. in countries where any public or registered private school may issue permission, the situation is completely different for potential applicants. parents may choose the school to which they submit the international electronic journal of elementary education vol.3, issue 1, october,2010 42 application. if financial subsidies for schools are calculated on the basis of enrolled students, schools may even compete for homeschoolers. in most cases, legal home education in the post-communist countries is practiced in close interaction with supervising schools. there is both a positive and negative aspect to this. the negative aspect consists of the fact that some schools do not properly take into account specific features of home education and tend to push home educating parents into using methods that are suitable in school classes rather than in individualized education at home. the influence of schools is particularly visible in the way in which progress in education is evaluated. schools tend to prefer an assessment of progress of children by school-style written and oral examinations. parents, who know in fact more about the real academic progress of their children thus have only an advisory role in the evaluation. the school directors and teachers tend to treat parents doing home education the same ways as professionals treat amateurs and tend to guide and counsel them. the strong position of schools also leads to situations in which rules for homeschooling families might substantially vary within one country despite the same legislation. the positive feature of the close relations between homeschooling parents and supervising schools consists of the fact that schools provide some services for the homeschoolers. these families might quite often use some school facilities (libraries, gyms, computers, natural science labs, etc.). children might also be allowed to participate in selected curricular and extracurricular activities at the school. the educational outcomes of home education are evaluated in a similar way in most post-communist states – children are evaluated frequently (usually twice a year) and often through the form of written and oral exams organized by supervising schools. in no post-communist country has home education become a mass phenomenon thus far. the numbers of home educated children vary between the tens and several thousands. in spite of the fact that families interested in home education are not particularly numerous, the legalization of home education in the postcommunist countries seems to be quite important. in countries where the legalization of home education has not yet been completed, parents who are interested in home education try to keep their children out of the school system and to avoid any contact with the authorities (porumbachanov, 2007). in such situations the children’s education may not be recognized by the local secondary or tertiary schools or by the local employers. this poses a greater risk for children and for the whole of society than the legalization of home education and its operation under the clear rules and supervision of proper authorities does. • • • home education in the post-communist countries / kostelecká 43 yvona kostelecká, ph.d., assistant lecturer at faculty of education, charles university in prague. her research focus is: home education, integration of foreigners into czech school system, and international migration of researchers and skilled workers. references adámková, p. (2007). reforma veřejné správy na úseku školství a problematika financování školství v obecných otázkách. in dny veřejného práva, sborník abstraktů příspěvků z mezinárodní conference (pp. 333 342). brno: masarykova univerzita. beck, ch. (2002). home education – new political tension? the case of northern europe. paper presented at comparative education society of europe conference. london, july 2002. beck, ch. (2008). home education and social integration. critical social studies, 10(2), 5969. blok, h. (2004). performance in home schooling: an argument against compulsory schooling in the netherlands. international review of education, 50(1), 39 – 52. dalahooke, m. m. (1986). home educated children’s social/emotional adjustment and academic achievement: a comparative study. doctoral dissertation. los angeles: california school of professional psychology. dueholm, n. (2006). history of homeschooling movement in poland [www page]. url http://www.hslda.org/hs/international/poland/200612191.asp fladmoe, h., karpov, e. (2002). home education and family education in russia [www page]. url http://folk.uio.no/cbeck/ home%20education% 20and%20family %20education%20in%20russia.htm gališová, a. (1999). počátky domácí školy v české republice. absolventská práce. svatý jan pod skalou: vyšší odborná škola pedagogická. kolasińska, e., kuźmicka j., smoczyńska a. (2007). edukacja domowa. warszawa: polskie biuro eurydice. kostelecká, y. (2003). domácí vzdělávání. doctoral dissertation. prague: faculty of education, charles university. kolář, p. (2000). systém domácího vzdělávání v usa a ve vybraných zemích evropy. informační studie no. 5.155. praha: parlamentní institute. leis, t. (2005). home education in estonia [www page]. url http:// folk.uio.no /cbeck/ estonia.htm marvanová, g. (2001). experiment domácího vzdělávání v čr. thesis. brno: faculty of education, masaryk university. meyer, l. (1999). home-schooling in holland [www page]. url http://www.alternativelearning.org/ale/home-liz.html mifková, b. (2009). problematika vztahu rodiny a školy a důvody vedoucí k domácímu vzdělávání. thesis. brno: faculty of education, masaryk university. morkes, f. (2004). devětkrát o českém školství. praha: pedagogické muzeum j.a. komenského v praze. nováková, j., brant, j., tupý, j. (2002). domácí vzdělávání v české republice ve školním roce 2001/2002. průběžná zpráva o stavu pokusného ověřování domácího vzdělávání. praha: vúp. international electronic journal of elementary education vol.3, issue 1, october,2010 44 parzniewski, d. (2001). pionerzy edukacji domowej [www page]. url http://strony.wp.pl/biale/art/pionierzy.htm petrie, a. (1995). home educators and the law within europe, international review of education, 41(3-4), 285-296. petrie, a. (2001). home education in europe and the implementation of changes to the law, international review of education, 47(5), 477-500. podmínky pokusného ověřování domácího vzdělávání v čr (2001). směrnice ministerstva školství, mládeže a tělovýchovy č.j.: 19 530/2001-22 vydaná v praze dne 14. června 2001. porumbachanov, p. (2007). the homeschooling in bulgaria. speech on the 2. kolloquium zu home education in rothenfels/main, april 27.-29. 2007 priesnitz, w. (2003). home based learning in canada. an overview of laws [www page]. url http://www.life.ca/hs/hsreport.html rand, e., leitmaa, d. (2007). üle tuhande lapse õpib kooli asemel vanemaga kadus, eesti päevaleht [www page]. url http://www.epl.ee/uudised/408885 rakestraw, j. f. (1987). an analysis of home schooling for elementary school-age children in alabama. doctoral dissertation. tuscaloosa: university of alabama. rakestraw, j. f. (1988). home schooling in alabama, home school researcher, 4(4), 1-6. ray, b. (1990). a nationwide study of home education: family characteristics, student achievement, and policy matters. paeonian springs, va: home school legal defence association. ray, b. (1994). a nationwide study of home education in canada: family characteristics, student achievement, and other topics. salem, or: national home education research institute. ray, b. (1997). strengths of their own home schoolers across america: aca-demic achievement, family character-istics, and longitudinal traits. salem, or: national home education research institute. rothermel, p. (2002). home-education: rationales, practices and outcomes [www page]. url http:// www. jspr. btinternet.co.uk/ research/ researcchpaper/ bera workingpaper. htm thomas, a. (1999). research in england and tasmania [www page]. url http://www.tnet.com.au/-piffle/herlin/research/athomas1.htm tomczyk, p. (2001). nauczanie bez dzwonka. rzeczpospolita online. no. 191, 17. tůma, j. (2001). k domácímu vzdělávání … učitelské noviny, 104 (37), p. 19. výroční zpráva o domácím vzdělávání 2001/2002. (2002). praha, zš letohradská. zpráva o průběhu pokusného ověřování odlišné organizační formy základního vzdělávání – domácího vzdělávání, školní rok 1999/2000. (2000). ostrava: zš ostrava-výškovice. zpráva o průběhu pokusného ověřování domácího vzdělávání za školní rok 2000/2001 v bratrské škole. (2001a). praha: bratrská škola – církevní základní škola. zpráva o průběhu pokusného ověřování odlišné organizační formy základního vzdělávání – domácího vzdělávání, školní rok 2000/2001. (2001b). ostrava: zš ostrava-výškovice. zpráva o průběhu pokusného ověřování odlišné organizační formy základního vzdělávání – domácího vzdělávání, školní rok 2001/2002. (2002). ostrava: zš ostrava-výškovice. international electronic journal of elementary education, 2014, 6(3), 395-414. 395 student-described engagement with text: insights are discovered from fourth graders timothy g. weih  university of northern iowa, usa received: 01.04.2014 / revised: 25.06.2014 / accepted: 28.06.2014 abstract this article reports on a research study investigating student-described engagement with selfselected text in a classroom where a core reading program (in the context of this study meaning instruction based primarily on manuals and commercial textbooks) comprised the majority of their literacy instruction. fourth grade students were invited to characterize their responses to their self-selected reading within focus group discussions. data instruments included audio taped focus group discussions, student photographs, observational field notes, and students’ literature. implementing the constant comparative method for data analysis, outcomes were determined and implications for classroom practice suggested. keywords: elementary reading methods, critical literacy, intrinsic motivations for reading, children’s literature, united states. introduction the misuse of children’s literature nurturing meaningful engagement between students and what they are reading by evoking students’ personal and emotional connections to the text is a phenomenon that research has shown to enhance meaning making (cochran-smith, 1984; barone, 2011; pantaleo, 2004; short, 1992; sipe, 1998). however, classroom practices can sometimes create stumbling blocks that hinder students’ deeper understandings (rosenblatt, 1982). the problem lies within several areas one of which is the lack of teacher education programs requiring a course or courses in the study of children’s literature (hoewisch, 2000). cooper (2007) admonished that educators acquire strong instructional background knowledge about children’s literature and how to use literature for supporting children’s psychosocial development. another problem area is the practice of implementing comprehension strategy instruction using children’s literature that narrowly focuses on understanding story elements, i.e., settings, characters, problems, and events, rather than textual analysis (calkins, 2000; collins, 2004; daniels, 2002; fountas & pinnell, 1996; harvey & goudvis, 2000; keene & zimmerman, 1997). this shift was most likely fueled by the  timothy g. weih, university of northern iowa, 1227 west 27th street cedar falls, iowa, 50614, usa, phone: 1+319-273-7466, e.mail: timothy.weih@uni.edu international electronic journal of elementary education vol.6, issue 3, 395-414, 2014 396 mandates for students to show adequate progress in high-stakes reading tests (guthrie, in press). a further area of concern is the use of core reading programs (i.e., manuals and commercial textbooks) which tend to lump reading instructional strategies together without regard to matching text type or genre to the appropriate strategy. for example, informational texts (e.g., expository texts), rather than narrative texts, lend themselves well for teaching such strategies as k-w-l, concept mapping, t-notes, qar, sq3r, learning logs, main idea and detail charts. these strategies are primarily used to help students gain and learn information from text that they can take away from the text to use primarily on a test. in comparison, stories, or narrative material, are more suited to invite students to make personal and emotional connections to text, thereby enhancing deeper understandings of these genres (nathanson, 2006). textbook reading an additional road block that stands in the way of students forming deeper understandings of text is that the majority of school reading is based upon traditional textbooks (alvermann & moore, 1991) which consists primarily of standard forms of traditional language that limit the possibility for multiple discourses among students (new london group, 1996). both new and highly experienced teachers are frequently required to maintain a level of standardization to a prescribed reading program, rather than develop curriculum that aligns with what research has suggested as being pedagogically sound practice for teaching students reading and language (peasealvarez, samway, & cifka-herrera, 2010; spencer, 2009). some teachers have even referred to specific reading program manuals as “the bible” (see compton-lilly, 2011). this pressure to adhere to highly specified reading programs have shown to be fueled by high-stakes accountability polices (achinstein & ogawa, 2006; colburn, 2001, 2005; stillman, 2009; stillman & anderson, 2011). in a response to this pressure, teachers have reduced their reading curriculum to skills and strategies (crosland & gutierriz, 2003; oullette, dagonstino, & carifio, 1999; valli & buese, 2007). this has placed teachers into an imposed contradiction to teach in ways that are in opposition to what they know are best literacy practices (dooley & assaf, 2009; valli & chambliss, 2007). consequences of skills-based reading instruction a serious consequence could follow if students’ primary experiences with reading are for skills instruction alone, they may ultimately see reading as a chore and something only connected to school work. dyson (2003, 2008) and compton-lilly (2007) reported how today’s expectations for literacy instruction narrows the opportunities for students to develop a rich repertoire of literary understanding. dyson (2003), following a study of students’ literacy growth over time, argued that students do not learn literacy in a series of stages with sequentially learned skills that are the same for all students. however, literacy instruction today has seen a shift from student-centered, differentiated curriculum and instruction to mandated core programs, i.e., manuals and commercial textbooks, that treat all students as if they are all the same (handsfield & jimenez, 2008). in these situations, teachers are encouraging students to respond to their reading through sub-skill development (cooper, 2007) as frequently seen on worksheets that lead them to the same convergent way of thinking about a text. motivation to read past research has demonstrated that students’ success in reading is linked to their intrinsic motivations to read (gottfried, 1990; guthrie & wigfield, 2000). wigfield and guthrie (1997) described students’ motivations to read in terms of their personally held values, beliefs, needs, and goals. consequently, when classroom reading instruction is student-described engagement with text: insights are discovered from fourth graders / weih 397 based upon what students are interested in reading, the greater potential exists for students to expand their efforts towards reading (pitcher et al., 2007). another area that past research has shown to inspire students towards reading is allowing them some choice in what they read (kurkjian & livingston, 2005; livingston & kurkjian, 2003; worthy, 1996; worthy, turner, & moorman, 1998). worthy (1996) argued that if students are not reading text that they enjoy, they may develop an aversion to reading that could last the rest of their lives. strommen and mates (2004) also echoed this view by arguing that a serious problem exists when students experience the majority of classroom reading assignments as unconnected to their interests and needs that they soon will become nonreaders. in related research, pacheco (2010) furthered this problem by claiming that current classroom practice in reading instruction has seen a reduction in allowing students choices in what they read based upon assumptions that students do not always choose literature worthy for learning. additional research has suggested that what students choose to read on their own frequently is not viewed school-worthy enough to count as literature (comber & simpson, 2001; janks, 2000; lewison, leland, & harste, 2008; vasquez, 2001). unique reading experiences cognizant of the current landscape of reading instruction in today’s classrooms, the aim of this study was to discover how students were engaging with self-selected text in a classroom where the teacher was expected to teach reading from a core reading program and to prepare students to take standardized reading tests. with this in mind, in responding to literature discussion prompts focused on eliciting students’ personal responses, would students, for instance, mimic their core reading instruction and respond in similar ways, most often focusing on the rote literary comprehension aspects of a text, or would they express a more critical, personal interpretation of text based upon their perspectives, opinions, and beliefs? the theoretical framework for this study is grounded within reader response criticism. this school of literary theory focuses on the reader and his or her personal and unique experience of a literary work. the reader is viewed as an active participant who creates meaning from the text through interpretation. rosenblatt (1969, 1982), a prominent theorist in reader response criticism, held that readers experience a text through the melding of the emotional and intellectual dimensions of reading, and that readers need to have an engaged relationship with a text, or a “lived-through” experience before being able to critically analyze a text, and this involves emotional and intellectual aspects of the experience. she argued that these reader responses are not mutually exclusive from each other, but instead, work in concert with each other depending on the reader’s purpose or stance for reading (1982). this engagement or transaction (rosenblatt, 1978) between the text and the reader is necessary for positioning the reader to experience the joy of reading, and thereby increasing the likelihood that he will become a life-long reader. moreover, rosenblatt (1976) proposed that there exists a transactional event between the reader and the text that takes into account the reader’s social and cultural context. based upon this assumption, when students demonstrate understanding of text, they do so through the lens of their personal perspectives, opinions, and beliefs. pantaleo (2004) contributed to this understanding by examining students’ textual connections through small-group discussions and found supporting evidence that students’ discussions about literature reflected their personal understandings of the text as they expressed interpretations in relationship to their own life experiences. students read to gain something from the experience. within rosenblatt’s (1978) reader-response theory, a reader positions himself or herself in a particular stance international electronic journal of elementary education vol.6, issue 3, 395-414, 2014 398 towards a text. if the focus is to take knowledge away from the text, then an efferent stance is assumed, whereby the reader will think about the informational concepts and details to be retained for personal use. when readers are focused on a story or “poem,” then they will more likely be thinking about the feelings, emotions, and perspectives conveyed within the text and use these as bridges to make connections to their own humanness. rosenblatt held that the relationships between texts, narrative and expository; and stances, aesthetic and efferent; represent dominat tendencies in readers, but not necessarily rigid divisions. in turn, readers can and do form informational attitudes towards narrative texts and emotional attitudes towards expository texts. when students’ needs are met through reading, this is something that they not only want to repeat, but also to share with each other. it is through these student exchanges that students learn to understand themselves and those around them (rosenblatt, 1982). teachers can capitalize on students’ natural emotions by reading aloud to them selections that build excitement, intrigue, mystery, and that are laced with juvenile humor. students frequently express their emotions through non-verbal ways such as wide-eyes, gaping mouths, and tense bodies. when students experience reading in this manner, the activity becomes something that they want to repeat. when teachers want students to experience the emotion, feelings, since of social justice, or human connectedness through reading, then the appropriate texts lie within the narrative genres of fiction (stoodt-hill & amspaugh-corson, 2009). story related activities that would influence students to connect emotionally to the text might include dramatization (see paley, 1981, 1990), visual representations of the story which could take the form of graphic arts or multimedia formats (see bedard & fuhrken, 2011), and peer discussions, poetic interpretations, and community service projects (see adams, 2007). through these types of responses, students are engaged in multimodal approaches to learning (de koning & van der schoot, 2013; gardner, 2006), transmediation (mills, 2011; siegle, 2006), and aesthetic responses to texts (greene, 2001; johnson, 2008; lohfink & loya, 2010). studies on motivation and effective teaching strongly suggests that children’s literature plays a key role in engaging students (pressley, dolezal, raphael, mohan, roehrig, & bogner, 2003). intrinsic motivation to read is tied to a student’s joy in reading that is done “for its own sake,” which is characterized by the student’s excitement and interest in the act of reading (guthrie & wigfield, 2000). it is through this excitement and motivation that literary relationships are born. the context in which students are engaged in reading instruction can have a powerful influence on the stance that students assume while reading. in other words, what students are being asked to do in their reading classroom will have an influence upon the reading connections that they form (sipe, 2008). open-ended response to reading sipe (2000), in a study examining the spontaneous responses of second graders to literature found that they used their comprehension and interpretation of story events as springboards into their own creative purposes. in a similar context, while investigating a group of second graders who were allowed and encouraged to respond to literature through creative expressions, adomat (2010) found that the students’ understandings of the literature was enhanced. taking a different, but related approach, soundy and drucker (2010) investigated how students used illustrations in their literature as models for their own creations. they found that the students were able to demonstrate their own literary understandings through self-created illustrations. student-described engagement with text: insights are discovered from fourth graders / weih 399 austin (2010) examined readers’ responses to a novel and found that her participants responded to how the author created the characters, setting, problem, events and theme. in addition, she claimed that a reader’s background knowledge and experiences relative to the content of the novel, had a strong impact upon the reader’s responses. in addition, mellor and patterson (2000) as well as smagorinsky and o’donnell-allen (1998), suggested that students develop reader responses in relationship to how authors have situated characters both socially and culturally within a narrative. martin, smolen, oswald, and milam (2012) analyzed the oral and written responses of eighteen third grade students while they read literature that contained themes of social justice. they concluded from their data that the students responded to their readings with a heightened sense of respect, caring, and understanding towards other people in need. in addition, moller (2012), indicated that while students were discussing a novel in which the author brings to the forefront concerns about the social and political treatment of a certain group of people, the students became increasingly empathetic in their understanding that at first, was lacking. heath (2013) provided evidence based upon her three decades of research on the learning lives of working-class families that students seek out reading opportunities that develop and broaden their individual special interests. she asserted that it is crucial for educators to pay attention to what interests students and to support them in their choices of learning projects. additionally, in a study involving 384 students, pitcher, et al. (2007), discovered that students were most motivated to read literature that connected to a topic that held special interest to them. moreover, they found that students’ reading interests and needs usually did not match the assigned reading in their classrooms. in her earlier research, heath (1986) argued that developing a child’s imagination through reading was the necessary component for comprehension and textual interpretation, and therefore, teaching to the imagination through children’s literature should be the goal of reading instruction. vygotsky (1978) pointed out that imaginative thinking is the precursor of abstract thought. he held that children’s imaginative play leads to the creation of a zone of proximal development that enhances problem-solving skills beyond what the child currently is capable. cooper (2007) further confirmed that imaginative play and imaginative literature are connected through their mutual support for helping children develop problem solving skills and in addition, enhancing their intellectual, social, and emotional development. researchers such as coles (1989) and greene (1995) admonished the necessity of imaginative literature for the promotion of self-awareness, creation of new knowledge, and the awareness of social aims. frequently, when students’ imaginations are sparked, their responses are manifested through out-of-school, self-generated, and spontaneous multimodal activities (siegel, 2006) that spring from literature, but are not teacher assigned. for example, students role-play character action (sipe, 2000), illustrate visualized content (see de koning & van der schoot, 2013), create videos (see schillinger, 2011), develop storyboards (see bogard & mcmackin, 2012), or record songs (cardany, 2012). we, as educators, have the responsibility to ignite spontaneous and intrinsic reader responses within students; however, when teachers adhere to a core literacy curriculum, which is typically standardized (one-size-fits-all) and scripted, they are in effect assuming that all students are the same, or “generic” (luke, 1995/1996), and there is scant opportunity for students to experience authentic lived-through engagement (rosenblatt, 1969, 1982) with a text that would cause them to create the literary relationships necessary to foster a life time of reading desire and enjoyment. international electronic journal of elementary education vol.6, issue 3, 395-414, 2014 400 these are the concerns that led to this present study, and insights were discovered from the primary source, the students. method participants the goal of the study was to describe and interpret social phenomena in a natural setting (schwandt, 1994). therefore, the study utilized the constructivist paradigm of being descriptive, qualitative, and naturalistic (guba & lincoln, 1994). atkinson and hammersley (1994) explained that qualitative researchers are not understood simply as objective cameras for recording data, but rather interpret the data through the perspectives of their own subjectivities and intellectual backgrounds, and become part of the context that they are studying. glaser and strauss (1967) explained that researchers who act from the constructivist research paradigm do not approach data with rigid expectations, but rather build grounded theory based on the conceptual relationships they construct from the data. in addition, tracy (2013) described qualitative research as immersing oneself into the context being studied in order to make sense of it. the school the study was developed over the course of five months in a fourth grade classroom located in an urban community made up of two adjoining cities with a population of about 100, 000 located in the midwestern united states. the school itself has a population of about 400 students kindergarten-grade six, with 89% european american, 5.3% african american, 3.7% hispanic, and 3.9% asian american. the teacher and researcher the classroom teacher, connor (pseudonyms used throughout), was in his eighth year of teaching, all spent in the fourth grade. both his undergraduate and master’s program emphasized the constructivist approach that involved children in actively constructing their own meaning through social collaboration within the classroom setting. as the researcher, i added my own background to the social context of the classroom. my teaching experience included 14 years as an elementary classroom teacher. although i am presently a university professor and researcher, the common ground between myself and connor as classroom teachers enabled us to communicate fluidly with understanding. the students the classroom students were a heterogeneous group of nine boys and 11 girls. there was one african american girl and one asian american boy, and the rest were of anglo european ethnicity. in this class all students were either reading at grade level or above. the classroom context the classroom was organized with student desks clustered in small groups, tables, book shelves with novels and reference materials, and a computer area with four desktop computers. during connor’s literacy block, students were engaged individually and silently in a version of the daily five (boushey & moser, 2006). connor’s version of the daily five involved students in reading alone, listening to books on tape, working on writing, and working on a computer with a spelling program. while students were working by themselves, connor called small groups to come to a kidney shaped table in front of the classroom for reading instruction from the manual guided reading: good first teaching for all children (fountas & pinnell, 1996) that he used for his core student-described engagement with text: insights are discovered from fourth graders / weih 401 reading program. adhering to this core program, connor delivered instruction, according to the manual, in “story” elements including characters, plots, and themes; however, included within the directions are for the students to look for important ideas and concepts, which are elements more typically associated with expository text. most of the books that connor used followed the story grammar format, and the book list in the manual mainly lists narrative titles. for homework, students took their group assigned books home and wrote summaries in a journal. at home, students also kept a reading log in where they recorded the number of pages read and the amount of time spent reading self-selected books. in the middle of the literacy block, students engaged in sustained silent reading (ssr) and ate a snack, after which the daily five resumed along with the guided reading instruction. data collection observational field notes. i kept a researcher’s journal (maykut & morehouse, 1994) in which i wrote various types of notes throughout the study. i wrote descriptive notes about the context of the setting including the appearance of the classroom and the activities of the students and teacher (bogden & biklen, 2007). merriam (1998) asserts that observations represent a first-hand encounter with the phenomenon of interest and when combined with interviewing, allow for a more complete interpretation. i recorded reflective notes regarding my ideas for the procedures for the study that would best fit into the social context of the classroom (richardson, 1994). i wrote summaries of conversations that i had with connor that later served as member checks (guba & lincoln, 1989). while the students were involved in the focus interviews, i wrote analytic reflections that represented my initial or gut reactions (tracy, 2013) regarding my interpretations of the responses that the students were conveying to me. audio recordings. the students were audio taped during the focus group interviews in order to obtain the best possible record of their exact words (patton, 1990). the students spoke in their own words in response to the interview prompts and were not following any type of a predetermined script or survey. photographs. connor provided photographs of all the students. the photographs provided rich, descriptive data that i used during data analysis to bring back memories of the students that served to further my understanding and interpretation of their spoken words on the audio tapes (bogden & bilken, 2007). literature. the literature that the students discussed during the focus group interviews was documented and became part of the analysis process. students were not required to bring literature, and they were not limited in what they decided to discuss. phases of data collection. the study was constructed in a series of phases over five months. during the first phase, the goals were to gather data specifics about the school, teacher, and students. during the second phase, data were gathered relating to the classroom’s culture, routines, and organization. it was during this phase that plans were made between connor and me for implementing the focus group interviews. the last phases of the data construction were the focus group interviews. focus group interviews. in qualitative research, interviews can be thought of as a conversation with a purpose (lincoln & guba, 1985). interviews can range on a scale from highly structured where the interview questions are carefully formulated beforehand, or an unstructured format, where the interviewer asks and actively listens for the sake of understanding what the interviewees are experiencing and communicating in their own words (maykut & morehouse, 1994). the type of interviews chosen for this study were semi-structured, focus group interviews. merriam (1998) explains that the semi-structured interview is halfway between the structured and international electronic journal of elementary education vol.6, issue 3, 395-414, 2014 402 unstructured interview with some of the same open-ended questions asked of all the participants in pursuance of the phenomenon under study, but allowing for questions and answers to naturally occur. similarly, tracy (2013) described focus group interviews as guided group discussions that are characterized by interactive dialogue between the interviewer and the participants. this above description illuminates the focus group interviews for this present study. the focus interviews were recorded in two ways, notes taken during the interview and audio recordings. the students were randomly assigned to five groups for the interviews. the aim of the study was to discover from the participants how they were responding to the literature in their lives given the current emphasis on testing and focusing reading instruction on preparing students to take these tests. for the purpose of learning from the students, the following open-ended, focus interview discussion prompts were created and are presented below along with the intended rational for each one. 1) tell me the title of a book, magazine, or something else that you are now reading or have recently read that you really liked. the aim of this prompt was to discover the genres and formats of literature the students enjoyed along with giving them the approval of discussing alternate literature that may or may not be connected to classroom practice. in addition, the prompt was intended to be easy for the students to answer, therefore acting as a segue into an open-ended discussion framed by the subsequent prompts. 2) tell me what you liked about the literature. the intention of this prompt was to discover from the students their intrinsic motivation for reading the literature they chose to discuss. 3) tell me about activities, either in school or out-of-school, that you did, that were related somehow to the literature that you read. this could also be an activity that you have not done yet, but would like to do someday. this discussion prompt was created to discover from the students how they were responding to the literature they were reading. 4) what about a book, magazine, or other reading material draws you to want to read it? the purpose of this prompt was to find out from the students if they had a specific genre or set of text characteristics that appealed to them when selecting literature for their personal enjoyment. data analysis preparation of the data analysis. in pursuance of making the data collected during the duration of the study readable, workable, and to provide for trustworthiness, the taped focus group discussions were transcribed, the researcher’s journal was typed, and the photographs of the students were printed. the data were coded as to types and sources (maykut & morehouse, 1994). unitizing. the next phase of data analysis was to identify the units of meaning contained within the data. lincoln and guba (1985) described a unit of meaning as the smallest piece of information about something that can be understood without any additional details other than the knowledge of the broader context from which it came. unitizing is a component embedded into the constant comparative method of qualitative data analysis (glaser & strauss, 1967; lincoln & guba, 1985). maykut and morehouse (1994) explained that the constant comparative method, identifying and categorizing specific units of information and comparing the units to previous student-described engagement with text: insights are discovered from fourth graders / weih 403 information, provides the researcher with a clear direction for engaging in analysis of data that is both challenging and illuminating. lawrence-lightfoot and davis (1997) furthered defined this analysis practice as examining all of the data for repetitive refrains. discovery. the units of meaning gleaned from the data were further refined during the discovery phase of analysis. taylor and bogden (1984) described this process as looking for recurring words, phrases, and concepts across the units of meaning. the discovery list for this study was constructed and reconstructed three different times in taking multiple soundings (gilligan, brown, & rogers, 1989) while searching for subtle meanings and complex perspectives expressed within the data. rather than listening to a story, a story was being discovered from the data (lawrence-lightfoot & davis, 1997). inductive category coding. during this process of data analysis, the prominent ideas were selected from the discovery list, and each of these became a provisional category. this process continued with grouping the units of meaning into related categories using the look/feel alike criteria advanced by lincoln and guba (1985). rules of inclusion. once the units of meaning were grouped into categories, they were examined to determine the overall message contained within them. this information, or rule of inclusion, then became a propositional statement of fact grounded in the data (taylor & bogden, 1984), and used to either include or exclude subsequent units of meaning for each category. once these were determined, the units of meaning were coded as to their rule-based category. examining relationships and patterns across categories. further syntheses of the data involved examining relationships between categories and studying the propositions for those that stood-alone or formed salient relationships and patterns (maykut & morehouse, 1994). during this final analysis in applying the constant comparative process, it became evident that some of the categories shared ideas that were related. these ideas were grounded in the data and stood as evidence of what i was learning from my participants in the study, and is what led to the determination of the study outcomes, which are presented and discussed in the next section results and discussion: defining the students’ text engagements five outcomes emerged from the data analysis process and revealed the nature of the students’ responses with the literature they chose to discuss. each outcome is discussed below along with example evidence grounding the outcome within the data and past research. outcome 1: students respond to text through the enjoyment of an event in their reading this outcome was evidenced in the data by those students who expressed literary understandings of events described in stories wherein the characters are involved in solving problems that sometimes leads to unique and humor-filled situations. this outcome from the present study expands upon previous research from sipe (2000) suggesting that students respond to stories through interpreting and evaluating events. in the data excerpts below, fed recounts a funny event in his book where some characters are pranked with a bucket of water being thrown on them. billy bob details a recurring event in his book wherein whenever the main characters encounter bad luck, they blame it on a family curse. rob is impacted by an event in his book where two characters receive broken bones as consequences of their ill-conceived actions. selected examples from the data that stood as evidence that appeared to be representative of this outcome included the following: international electronic journal of elementary education vol.6, issue 3, 395-414, 2014 404 fred: i am reading diary of a wimpy kid [kinney, 2007]. it is funny, because on halloween, whenever teenagers walk by, his dad will throw a bucket of water on them, and i like that because it is funny…i like to read funny books. billy bob: i am reading holes [sachar, 2000]. i like that part where he gets sent to jail, because, well actually, because they keep on blaming their great, great, great grandfather, they say no good pig steeling great, great, great grandfather, i think…um, because every time they blame him and um, when he got sent to court, he was at the wrong place at the wrong time, and that’s the curse from his great, great, great grandpa. rob: i am reading the invention of hugo cabret [selznick, 2007]. i like the part where the girl and the boy were going to break into the apartment, but the boy slammed his hand in the door and broke it, and the girl dropped a box on her leg, and she broke her leg. data that represented this outcome was reflective of the students’ reading instruction in the classroom with their core reading program which emphasized recalling events from the plots of stories. during the discussion, the students’ body language and inquisitive expressions aimed at me gave me the impression that they thought i had a correct answer in mind, and they were charged with finding out what it was. this too, was indicative of their classroom reading instruction wherein they were asked to relay details of the stories back to their teacher. even though the discussion prompts were intended to be open-ended, students seemed to be constrained, possibly due to being on unfamiliar ground. this outcome was indicated in past research findings suggesting that reading instruction based on core programs, narrowly focuses on story elements rather than on textual analysis (calkins, 2000; collins, 2004; daniels, 2002; fountas & pinnell, 1996; harvey & goudvis, 2000; keene & zimmerman, 1997). outcome 2: students respond to their reading through self-creative endeavors patterns in the data demonstrated that students responded to their reading through creative expressions that sprang from the text. this outcome builds upon previous research by adomat (2010) confirming that students use literature as a platform for expressing their understandings through a variety of creative modalities. additionally, sipe (2000) also concluded that students engage and respond to literature by using their comprehension of the text as a springboard into their own creative expressions. moreover, soundy and drucker (2010) suggested that students use the illustrations in literature as models for their own creations, thereby demonstrating their literary meaning making and understanding. in the data excerpts below, fred gives an account of how he responded to his book by creating a three dimensional representation of himself reading the book. frankie felt inspired to mold a clay dragon like the one depicted in his book. kinsey details a bookmark that she made after reading her book, but then comes up with an idea influenced by her book to create her own, unique version of the story. rob conveys that he drew a picture of himself reading his book. isabella also made a drawing after reading her book wherein she represented the front cover. some data examples that were categorized into this outcome are given below: fred: i made the book out of a paper and a self-portrait of myself with the book in my hands and it looks like i am reading it. frankie: i want to make a clay dragon like they did in the book [ice fire, d’lacey, 2006]. i like adventure stories. kinsey: i like james and the giant peach [dahl, 1961]. i have this thinkmark [http://www.thinkmark.org/], and i put this page number down, and i write what was interesting on that page, and like something that either i don’t like, i like, or doesn’t make student-described engagement with text: insights are discovered from fourth graders / weih 405 sense or something….maybe just like write, in writing workshop time, maybe write like um, maybe kind a like a book that has talking animals like…like maybe i could make a big leaf that they imagined that could never get so big. rob: after i read the book, i made a picture of myself, and i drew a picture of the book and put the book in my hands. isabella: when i read the book [the lost treasure of the emerald eye, stilton 2004], i always like to write a response after, and um, sometimes i like to draw the cover. students really struggled with thinking about having done or doing anything creative in relation to their book that they chose to share. after clarifying with each other and again looking to me for the answers, they came up with something to talk about. however, the self-portrait that fred and rob discuss was actually a school assignment. the thinkmark and “write a response” that kinsey and isabella share, were also school assignments. this outcome also supports previous research findings showing that today’s literacy instruction narrows the opportunities for students to develop creative expressions that represent their engagement and understanding (compton-lilly, 2007; dyson, 2003, 2008). in contrast, however, frankie reflects back to his book and comments that he wants to make a clay dragon. this unit of meaning was very exciting to discover, and also suggests that he was willing to explore with a more intrinsic expression of his inner text relationship in an unconventional modality, at least, not experienced in his classroom instruction. outcome 3: students respond to their reading through self-created goal setting related to special interests evidence grounded in the data suggested that students’ literary responses were connected to their personal goals that they set for themselves that were related to their individual special interests. this evidence from the present study expands upon past research from heath (2013) suggesting that students seek out reading opportunities that develop and enrich their special interests. equally important, this outcome builds upon past research from pitcher et al. (2007) indicating that students like to read literature that connects to their personal hobbies and activities in their search for more information. as evidenced from the data excerpts below, skyler felt stirred from her reading to go camping, and elizabeth shared that she enjoys digging, something that happened in her reading, and that she had already engaged in. a couple of examples from the data representing this outcome are as follows: skyler: i am reading the book the river [paulsen, 1993], and i like my book because it has a lot of nature in it…i like the books that are informational…the information that maybe you could use for writing or a subject…the activity that i would like to do is go camping. elizabeth: the activity that i already did was digging [reading holes (sachar, 2000). the data that formed this outcome was scarce and lacking, but supports previous research pointing out that current practices in reading instruction has seen a reduction in allowing students choices in what they read based on the assumptions that students do not always choose literature worthy for learning or even reading material that would count as literature (comber & simpson, 2001; janks, 2000; lewison, leland, & harste, 2008; vasquez, 2001). this serious problem is evidenced in the data for this present study, and echoes the concern that if students are not reading text that represents their international electronic journal of elementary education vol.6, issue 3, 395-414, 2014 406 interests and goals, they could develop an aversion to reading (worthy, 1996) and stand the risk of becoming nonreaders (strommen & mates, 2004). outcome 4: students respond to their reading through the enjoyment of the author’s writing creativity through the refinement of the data, there emerged the outcome that the students responded to authors’ creativities in developing characters, settings, problems, humor, and including, making nonfiction interesting. this outcome supports previous research from austin (2010) suggesting that students develop text-related responses to how authors portray characters, develop problems, content, and theme. additionally, this outcome connects to the research of mellor and paterson (2000) and smagorinsky and o’donnell-allen (1998) who further provided evidence that students engage with text and develop literary responses connected to how they perceive characters acting in association with how authors have created the narrative. as suggested from the selected data excerpts below, fred, isabella, billy bob, elizabeth, and rob, all appreciated how the authors of their literature created the experiences of humor. isabella discussed liking literature characters and mystery problems that the reader can work through to solve. frankie recounts how the author of his book used a lot of interesting, descriptive words that were unique. elizabeth conveyed her interest in how the author of her book created intriguing nicknames for the characters. rob commented that he looks for a theme of humor in the literature he reads, and if he likes the first book, he looks for more by the same author. kinsey related that she connects with books that have exciting plots. included below are selected examples from the data that represented the categories that formed this outcome: fred: i am reading diary of a wimpy kid [kinney, 2007]… it is funny. isabella: i read the book the lost treasure of the emerald eye [stilton, 2004]. i like about the book because it has a problem in it and i like how it is very funny and stuff… it’s like i write the genre, ah the connections, and the things that i like, and the characters and stuff…i always like the books that have mysteries and big problems that you have to solve and stuff, and i like funny books and stuff. billy bob: i like sports magazines with jokes in them. frankie: i am reading ice fire [d’lacey, 2006]. i like it because it is very descriptive…like they use sayings and sort of things that you really don’t hear, like it describes people, and it described this witch and said her teeth looked like curled up snails…i like adventure stories. elizabeth: i am reading holes [sachar, 2000] and i like the book because i like all their nicknames, and they all have nicknames…i look for the books that are funny and stuff like that. rob: i like to read stuff that has humor, or if sometimes if i’ve read the first book and there is a series, i will look for the whole series. kinsey: i like james and the giant peach [dahl, 1961]… it is interesting and um it just has like a lot of talking animals and stuff that come to life…the fact that a peach grows bigger than a house and they go inside of it….i look for stuff that is interesting, ‘cause i don’t like reading books that don’t have very much interesting things in them, because then you never come to an interesting thing, and it gets a little boring in that book… like i tried reading, i forgot the title but it was like a brat’s girl book [multiple authors, 20032006], and all it was talking about was the brats girls and like um a concert that they were going to, and it never had like interesting things or like a mystery or something that they had to do or something, or like into it, what they did and stuff. student-described engagement with text: insights are discovered from fourth graders / weih 407 the data that formed this outcome was more plentiful as compared to the units that formed the other outcomes. reasons for this are provided from past research confirming that today’s reading instruction is grounded in sub-skill development, i.e., grammar, main-ideas, details, vocabulary, making connections, and literary elements (crosland & gutierriz, 2003; oullette, dagonstino, & carifio, 1999; valli & buese, 2007). the concern with this practice is that it can impede students from fully engaging with text and forming the deeper, emotional relationships necessary to insure enhanced meaning making (cochran-smith, 1984; barone, 2011; pantaleo, 2004; short, 1992; sipe, 1998). outcome 5: students respond to their reading through self-expressed caring for others this outcome was evidenced in only one unit of meaning that was generated from one student. it is included as an outcome because of the value it represents. this outcome expands upon past research from martin, smolen, oswald, and milam (2012); and also including the research from moller (2012) suggesting that students develop a heightened sense of social awareness and concern for others based upon their reading of literature in which the authors have brought these concerns to their understandings. billy bob was the only student in the study that indicated this response to his reading when he said that he was inspired to donate to charity. the data excerpt with his comment is included below. billy bob: i would like to donate to charity like they did [holes (sachar, 2000]. there was only one unit of meaning that formed this outcome and given today’s focus on raising social awareness and empathy with students, should have been many more. the lack of data units for this outcome stands to confirm a serious problem inherent in a one-size-fits all core reading program that leads students to the same convergent way of thinking about a text (cooper, 2007). instead of exploring a more critical, personal interpretation of the text based upon individual perspectives, opinions, and beliefs, students are being prepared to take high-stakes reading tests (guthrie, 2002). limitations and future directions the participants of this study comprised students from one fourth grade classroom situated in one elementary school. future investigations could examine a different grade level, across grade levels, or across schools. the participants in this study were predominately european american. it would be beneficial for future studies to include a more diverse population with regards to ethnicity. the setting for this study was a medium sized city located in the midwestern united states, it would be of interest for future research to examine a small, rural community not connected or in close proximity to a larger metropolitan area, and located in a different region of the united states or a different country. conclusions this present study is significant because it advances multiple theories regarding the adverse effects of core curriculum policies surrounding the teaching of reading and language by supporting these propositions, many of which have been expressed in opinion papers, with student-generated evidence. additionally, in comparison to previous qualitative studies that began with examining data based upon established categories, the findings of this present study emerged from a discovery process grounded in the data analysis and not from predetermined categories. finally, this international electronic journal of elementary education vol.6, issue 3, 395-414, 2014 408 present study differs from many past studies in that the students’ literature-based responses were generated from their self-selected literature and not from pre-selected literature determined by the researcher or classroom teacher. given these significances, the findings of this study were the results of an authentic discovery process (lawrence-lightfoot & davis, 1997). fortunately, many of the mandated state and federal curriculum policies are beginning to expire and there have been many studies that have suggested their ineffectiveness (gamse, bloom, kemple, & jacob, 2008; spencer, 2009; peasealvarez, samway, & cifka-herrera, 2010). teachers are becoming freer to center their curriculum on student needs and cultural significances. but how can teachers guide their students in forming deeper, long lasting relationships built upon emotional and intellectual connections to what they read? reader response will need to be nurtured within students. teachers can begin by allowing and providing for students to make their own personal choices in what they read. this will not be easy, as students that have long been schooled with generic, one size-fits-all reading curriculum, may have developed an aversion to reading or they may have a poor self-image of themselves as a reader. interviews and surveys can be given to students to discover their reading interests, and then teachers and librarians can work together to provide the literature that students are most excited about. once students have the literature in hand that they most desire, then teachers can work to elicit reader response from them by first modeling their own response during a highly engaging read aloud. this may need to happen through several books or events. eventually, students will begin to think beyond the literal, text-structured questions typically asked during guided reading lessons, and think more deeply about how they are feeling inside about what they are hearing or reading. it is helpful for teachers to stop at key parts during a read aloud event and illicit this emotional response from students to draw out their thoughts and criticisms, allowing and encouraging students to freely express themselves without being judged right or wrong by teachers or their peers. this may happen slowly at first, but once students become comfortable with realizing that the teacher does not already know the answer and that they are not being put into a contrived position of trying to figure out what it is-watch out! the mold will be broken of the teacher with a short question and the student with a short answer, and students will start having meaningful discussions that will flow between each other as they make meaning together, and the language arts block will become more student-centered, than program-centered. some teachers may not feel comfortable at first, especially if they are used to following a scripted curriculum where the path is heavily trodden, student and teacher senses have been dulled, and the chance of discovering new meaning together is not promoted. but when teachers join students in making meaning together from a shared book, both will come away from the experience with new and exciting perspectives about themselves and the world in which they live. • • • timothy g. weih, is an associate professor of education at the university of northern iowa, cedar falls, usa. he teaches literacy methods courses and his research interests focus on the natural relationships children and youth develop with the literacy, both new and old, they choose for their leisure enjoyment. student-described engagement with text: insights are discovered from fourth graders / weih 409 references achinstein, b., & ogawa, r. (2006). (in)fidelity: what the resistance of new teachers reveals about professional principles and prescriptive educational policies. harvard educational review, 76, 30-63. adams, c. (2007, january/february). lessons on character. instructor, 116(5), 28-30. adomat (2010). dramatic interpretations: performance responses of young children to picturebook read-alouds. children’s literature in education, 41, 207-221. alvermann, d. e., & moore, d. w. (1991). secondary school reading, in r. barr, m. l. kamil, p. b. mosenthal, & p. d. pearson (eds.), handbook of reading research (vol.2, pp. 951-983). new york: longman. atkinson, p., & hammersley, m. (1994). ethnography and participant observation. in n. k. denzin & y. s. lincoln (eds.), handbook of qualitative research (pp. 248-261). thousand oaks, ca: sage. austin, p. (2010). sing a different song: local readers’ responses to paul volponi’s hurrican song: a novel of new orleans, journal of children’s literature, 36(2), 51-56. barone, d. (2011). making meaning: individual and group response within a book club structure. journal of early childhood literacy, 13(1), 3-25. bedard, c., & fuhrken, c. (2011) writing for the big screen: literacy experiences in a moviemaking project. language arts, 89(2), 113-124. bogard, j., & mcmackin, m. c. (2012). combining traditional and new literacies in a 21st century writing workshop. the reading teacher, 65(5), 313-323. bogdan, r. c., & biklen, s. k. (2007). qualitative research for education: an introduction to theories and methods (5th ed.) boston, ma: allyn & bacon. boushey, g., & moser, j. (2006). the daily five: fostering literacy independence in the elementary grades. portland, me: stenhouse. calkins, l. m. (2000). the art of teaching reading. upper saddle river, nj: allyn & bacon. cardany, a. b. (2012, october). exploring music dynamics through children’s literature, general music today, 26(1), 38-49. coburn, c. e. (2001). collective sensemaking about reading: how teachers mediated reading policy in their professional communities. educational evaluation and policy analysis, 23, 145-170. coburn, c. e. (2005). shaping teacher sensemaking: school leaders and enactment of reading policy. educational policy, 19, 476-509. cochran-smith, m. (1984). the making of a reader. norwood, nj: ablex. coles, r. (1989). the call of stories: teaching and the moral imagination. new york: houghton mifflin. international electronic journal of elementary education vol.6, issue 3, 395-414, 2014 410 collins, r. (2004). growing readers: units of study in the primary classroom. portland, me: stenhouse. comber, b., & simpson, a. (eds.). (2001). negotiating critical literacies in classrooms. mahwah, nj: lawrence earlbaum. compton-lilly, c. (2007). re-reading families: the literate lives of urban children-four years later. new york: teachers college press. compton-lilly, c. (2011). by the book and behind-the-glass: teacher self-regulation in one reading intervention. language arts, 88(6), 429-438. cooper, p. m. (2007). teaching young children self-regulation through children’s books. early childhood education journal, 35, 315-322. crosland, k., & gutierrez, k. (2003). standardizing teaching, standardizing teachers: educational reform and the deprofessionalization of teachers in an english-only era. educators for urban minorities, 2(2), 24-40. d’lacey, c. (2006). icefire. new york, ny: scholastic. dahl, r. (1961). james and the giant peach. new york, ny: puffin books. daniels, h. (2002). literature circles: voice and choice in book clubs and reading groups. portland, me: stenhouse. de koning, b. b., & van der schoot, m. (2013, june). becoming part of the story! refueling the interest in visualization strategies for reading comprehension. educational psychology review, 25(2), 261-287. dooley, c. m., & assaf, l. c. (2009). contexts matter: two teachers’ language arts instruction in this high-stakes era. journal of literacy research, 41, 354-391. dyson, a. h. (2003). the brothers and sisters learn to write: popular literacies in childhood and school culture. new york: teachers college press. dyson, a. h. (2008). the pine cone wars: studying writing in a community of children. language arts, 85(4), 305-315. fountas, i. c., & pinnell, g. s. (1996). guided reading: good first teaching for all children. portsmouth, nh: heinemann. gamse, b. c., bloom, h. s., kemple, j. j. & jacob, r. t. (2008). reading first impact study: interim report (ncee) 2008-4016). washington, dc: national center for education evaluation and regional assistance, institute of education sciences, u.s. department of education. gardner, h. (2006). multiple intelligences: new horizons. new york, ny: basic books. gilligan, c., brown, l. m., & rogers, a. g. (1989). psych embedded: a place for body, relationships, and culture in personal theory. in a. rubin, r. zuker, r. emmons, & frank (eds.), studying persons and lives (pp. 86-147). new york: springer. glaser, b., & strauss, a. (1967). the discovery of grounded theory. chicago, ill: aldinie. green, m. (1995). releasing the imagination: essays on education, the arts, and social change. new york: jossey-bass. green, m. (2001). variations on a blue a guitar. new york, ny: teachers college press. student-described engagement with text: insights are discovered from fourth graders / weih 411 guba, e. g., & lincoln, y. s. (1994). competing paradigms in qualitative research. in n. k. denzin & y. s. lincoln (eds.), handbook of qualitative research. (pp. 105-117). thousand oaks, ca: sage. gottfired, a. e. (1990). academic instrinsic motivation in young elementary school children. journal of educational psychology, 82, 525-538. guthrie, j. t. (2002). preparing students for high-stakes test taking in reading. in a. e. farstrup, & s. samuels (eds.), what research has to say about reading instruction (3rd ed.) (pp. 370391). newark, nj: international reading association. guthrie, j.t., & wigfield, a. (2000). motivation and engagement in reading. in kamil, m. l., mosenthal, p.b., pearson, p.d., & barr, r. (eds.), handbook of reading research: volume iii. (pp. 403-422). mahwah, nj: erlbaum. handsfield, l. j., & jimenez, r. t. (2008). revisiting cognitive strategy instruction in culturally and linguistically diverse classrooms: cautions and possibilities. language arts, 85(6), 450458. harvey, s., & goudvis, a. (2000). strategies that work: teaching comprehension to enhance understanding. portland, me: stenhouse. heath, s. b. (1986). separating “things of the imagination” from life: learning to read and write. in w. h. teale, & e. sulzby (eds.), emerging literacy: writing and reading (pp. 156-172). norwood, nj: ablex. heath, s. b. (2013). “so, what’s it about? your book, i mean?” journal of adolescent & adult literacy, 56(4), 266-270. hoewisch, a. k. (2000). children’s literature in teacher-preparation programs. review of research in education, 19, 269-297. janks, h. (2000). domination, access, diversity, and design: a synthesis for critical literacy education. educational review, 52, 175-186. johnson, m. h. (2008). developing verbal and visual literacy through experiences in the visual arts: 25 tips for teachers. young children, 63(1), 74-79. keene, e., & zimmerman, k. (1997). mosaic of thought. portsmouth, nh: heinemann. kinney, j. (2007). diary of a wimpy kid. new york: amulet books. kurkjian, c., & livingston, n. (2005). the right book for the right child for the right situation. the reading teacher, 58(8), 786-795. lawrence-lightfoot, s., & davis, j. h. (1997). the art and science of portraiture. san francisco, ca: sage publications. lewison, m., leland, c., & haste, j. (2008). creating critical classrooms: k-8 reading and writing with an edge. mahwah, nj: lawrence earlbaum. lil’bratz books. multiple author series. (2003-2006). new york: grosset & dunlap. lincoln, y. s., & guba, e. g. (1985). naturalistic inquiry. newbury park, ca: sage. livingston, n., & kurkjian, c. (2003). timeless and treasured books. the reading teacher, 57(1), 96-103. international electronic journal of elementary education vol.6, issue 3, 395-414, 2014 412 lohfink, g., & loya, j. (2010). the nature of mexican american third graders’ engagement with culturally relevant picture books. bilingual research journal, 33(3), 346-363. luke, a. (1995-1996). text and discourse in education: an introduction to critical discourse analysis. review of research in education, 21, 3-48. martin, l. a., smolen, l. a., oswald, r. a., & milam, j. l. (2012). preparing students for global citizenship in the twenty-first century: integrating social justice through global literature. maykut, p., & morehouse, r. (1994). beginning qualitative research: a philosophic and practical guide. washington, dc: falmer press. mellor b., & patterson, a. (2000). critical practice: teaching “shakespeare.” journal of adolescent & adult literacy, 43(6), 508-517. merriam, s. b. (1998). qualitative research and case study applications in education (rev. and expanded from case study research in education). san francisco, ca: jossey-bass. mills, k. (2011). “i’m making it different to the book”: transmediation in young children’s multimodal and digital texts. australasian journal of early childhood, 36(3), 56-65. moller, k. j. (2012). developing understandings of social justice: critical thinking in action in a literature discussion group. journal of children’s literature, 38(2), 22-36. nathanson, s. (2006). harnessing the power of story: using narrative reading and writing across the content areas. reading horizons, 47(1), 1-26. new london group. (1996). a pedagogy of multiliteracies: designing social futures. harvard educational review, 66, 60-92. oullette, g., dagostino, l., & carifio, j. (1999). the effects of exposure to children’s literature through read aloud and an inferencing strategy on low reading ability fifth graders’ sense of story structure and reading comprehension. reading improvement, 36, 73-89. pacheco, m. (2010). english-language learners’ reading achievement: dialectical relationships between policy and practices in meaning-making opportunities. reading research quarterly, 41, 75-93. paley, v. g. (1981). wally’s stories: conversations in the kindergarten. cambridge: harvard university press. paley, v. g. (1990). the boy who would be a helicopter: uses of storytelling in the classroom. cambridge: harvard university press. pantaleo, s. (2004). exploring grade 1 students’ textual connections. journal of research in childhood education, 18(3), 211-225. patton, m. q. (1990). qualitative evaluation and research methods (2nd ed.). newbury park, ca: sage. paulsen, g. (1993). the river. new york: random house. pease-alvarez, l., samway, k. d., & cifka-herrera, c. (2010). working within the system: teachers of english learners negotiating a literacy instruction mandate. language policy, 9(4), 313-334, retrieved from http://escholarship.org/uc/item/3010t3kq;jsessionid=f132683920ee7b56dc3b3cf95388df bf#page-1. student-described engagement with text: insights are discovered from fourth graders / weih 413 pitcher, s., lettie, k., albright, c., walker, n., seunarinesingh, k., moggie, s., headly, k., ridgeway, v., peck, s., hunt, r., & dunston, p. (2007). assessing adolescents’ motivation to read. journal of adolescent and adult literacy. 50(5), 378-396. pressley, m., dolezal, s. e., raphael, l. m., mohan, l., roehrig, a. d., & bogner, l. (2003). motivating primary grade students. new york: guilford. richardson, l. (1994). writing: a method of inquiry. in n. k. denzin & lincoln (eds.), handbook of qualitative research (pp. 516-529). thousand oaks, ca: sage. rosenblatt, l. m. (1969). pattern and process-a polemic. english journal, 58(7), 1005-1012. rosenblatt, l. m. (1976). literature as exploration (3rd ed.). new york: noble & noble. rosenblatt, l. m. (1978). the reader, the text, the poem: the transactional theory of the literary work. carbondale, il: southern illinois university press. rosenblatt, l. m. (1982). the literary transaction: evocation and response. theory into practice, 21(4), 268-277. sachar, l. (2000). holes. new york: yearling books. schillinger, t. (2011). blurring boundaries: two groups of girls collaborate on a wiki. journal of adolescent & adult literacy, 54(6), 403-413. schwandt, t. a. (1994). constructivist, interpretist approaches to human inquiry. in n. k. denzin & y. s. lincoln (eds.), handbook of qualitative research (pp. 118-137). thousand, oaks, ca: sage. selznick, b. (2007). the invention of hugo cabret. new york: scholastic. short, k. (1992). researching intertextuality within collaborative classroom learning environments. linguistics and education, 4, 313-333. siegel, m. (2006). rereading the signs: multimodal transformations in the field of literacy education. language arts, 84, 65-77. sipe, l. r. (1998). individual literacy response styles of first and second graders. in t. shanahan and f. v. rodriquez-brown (eds.), forty-seventh yearbook of the national reading conference (pp. 76-89). chicago, il: national reading conference. sipe, l. r. (2000). the construction of literary understanding by first and second graders in oral response to picture storybook read-alouds. reading research quarterly, 35(2), 252-275. sipe, l. r. (2008). individual literary response styles of first and second graders. in t. shanahan and f. v. rodriquez-brown (eds.), forty-seventh yearbook of the national reading conference (pp. 76-89). chicago, il: national reading conference. smagorinsky, p., & o’donnell-allen, c. (1998). the depth and dynamics of context: tracing the sources and channels of engagement and disengagement in students’ response to literature. journal of literacy research, 30(4), 515-559. soundy, c. s., & drucker, m. f. (2010). picture partners: a co-creative journey into visual literacy. early childhood education journal, 37, 447-460. spencer, t. g. (2009). complicating what it means to “struggle”: one young child’s experience with mandated literacy initiative. contemporary issues in early childhood, 10, 218-231. international electronic journal of elementary education vol.6, issue 3, 395-414, 2014 414 stillman, j. (2009). taking back the standards: equity-minded teachers’ responses to accountability-related instructional constraints. new educator, 5(2), 135-160. stillman, j., & anderson, l. (2011). to follow, reject, or flip the script: managing instructional tension in an era of high-stakes accountability. language arts, 89(1), 22-37. stilton, g. (2004). lost treasure of the emerald eye. new york: scholastic. stoodt-hill, b. d., & amspaugh, l. b. (2009). children’s literature: discovery for a lifetime. new york: allyn & bacon. strommen, l. t., & mates, b. f. (2004). learning to love reading: interviews with older children and teens. journal of adolescent & adult literacy, 48, 188-200. taylor, s. t., & bogdan, r. (1984). introduction to qualitative research methods: the search for meanings (2nd ed.), new york: wiley. tracy, s. j. (2013). qualitative research methods: collecting evidence, crafting analysis, communicating impact. malden, ma: wiley-blackwell. valli, l., & buese, d. (2007). the changing roles of teachers in an era of high-stakes accountability. american educational research journal, 44, 519-558. valli, l., & chambliss, m. (2007). creating classroom cultures: one teacher, two lessons, and a high-stakes test. anthropology and education quarterly, 38, 57-75. vasquez, v. (2001). getting beyond “i like the book”: creating space for critical literacy in k6classrooms. urbana, il: ncte. vygotsky, l. s. (1978). mind in society: the development of higher psychological processes. cambridge: harvard university press. wigfield, a., & guthrie, j. t. (1997). relations of children’s motivation for reading to the amount and breadth of their reading. journal of educational psychology, 89(3), 420-432. worthy, j. (1996). a matter of interest: literature that hooks reluctant readers and keeps them reading. reading teacher, 50(3), 204-212. worthy, j., turner, m., & moorman, m. (1998). the precarious place of self-selected reading. language arts, 75(4), 296-304. 97 © 2022 published by kura education & publishing. this is an open access article under the cc byncnd license. (https://creativecommons. org/licenses/by/4.0/) copyright © www.iejee.com issn: 1307-9298 international electronic journal of elementary education december 2022, volume 15, issue 2, 97-107 engagement in at-risk students trajectories: perspective from the levels of approach to the student abraham bernárdez gómeza,*, eva maría gonzález bareab abstract introduction student engagement within education has been an area of increasing interest over recent years as one of the factors that needs to be taken into consideration when addressing one of the main issues in education school dropout. the objective of this paper is to provide oversight of how being engaged in the educational trajectories of young people who are at risk takes place. this will involve applying the theory of life course as the main driver of the research. a qualitative method is used which adopts a biographical and narrative approach via semi structured in-depth interviews. the main results have revealed that: 1) there are stages in which events that facilitate positive or negative engagement are more predominant; 2) generally, there are more negative critical events that result in a decline in the quality of student engagement; 3) there are several factors that stand out in each of the stages of the student trajectories. there are many issues within education that are rooted in history. some have extensive research behind them and therefore provide additional detail (garnica et al., 2019). this is relevant to the problem tackled within this study, that of the school dropout rate of vulnerable, also known as at risk, students (bernárdez-gómez et al., 2021; thureau, 2018; vandekinderen et al., 2018) and how these students can then return to their studies (cuconato et al., 2017; tomaszewskapękała et al., 2017; portela et al., 2022; ribaya, 2011). this text presents some of the results of a wider research project where the factors that do or do not benefit school engagement of young dropouts and returnees are examined. therefore, the aim of the research was to determine the various factors intervening in student engagement via these stages of young people’s educational trajectories when they have dropped out of school. this research examines the circumstances of those who have been prioritised for education administrations over the years and in spite of this still demonstrate their relevance. the obsolete europe 2020 strategy to the current keywords: engagement, educational trajectories, narrative inquiry, life history, life course theory received : 28 september 2022 revised : 5 december 2022 accepted : 19 december 2022 doi : 10.26822/iejee.2023.282 a,* corresponding author: abraham bernárdez gómez, department of didactics and school organization, faculty of education, university of murcia, spain. e-mail: abraham.bernardez@um.es orcid: https://orcid.org/0000-0003-1862-5554 b eva maría gonzález barea, department of didactics and school organization, faculty of education, university of murcia, spain. e-mail: evamgon@um.es orcid: https://orcid.org/0000-0001-7848-4736 98 december 2022, volume 15, issue 2, 97-107 sustainable development goals or the horizon europe framework programme (eu regulation, 2021) proposed goals which could be aimed at enhancing educational quality and well-being in schools. consequently, those organisations are referred to by us as oecd (viac & fraser, 2020), and we can say that wellbeing in educational centres therefore has a direct link to educational quality provided by the school system. therefore, it should be demonstrated how the challenge can no longer be used to retain valuable students, but instead to determine friendly spaces within the centres where studies can be performed, and engagement maintained throughout the educational trajectories. educational trajectorias and life course theory studying educational trajectories is based on studying the life courses of the individuals (hutchison, 2019; monarca, 2017). this is a popular theory used within the field of social sciences and has increased in relevance to research within the social sciences (tomaszewskapekala et al., 2017; mena, fernández & enguita, 2010). development of the paradigm on which life course theory (lct) has been developed is multifactorial in nature, as is the dropout problem. the various factors that can be found within their past research are consistent (bernárdez-gómez, 2022; gottfried & hutt, 2019). therefore, using the life course theory, events are presented that have a significant effect on the trajectory of students. these events are the factors which have been found by authors including salvàmut, oliver trobat & comas forgas (2014) or nichol, defosset, perry & kuo (2016), who have referred to various spheres of proximity to the individual: micro, meso and macrosocial. simultaneously there are different events that can occur which have a different intensity. first, life events present throughout the whole trajectory but yet have no particular significance for the subjects (kang, 2019; tarabini, 2018). conversely, there are critical events, which are powerful enough to stimulate a transition within the trajectory that lead to a change in their direction. the various events that happen within an individual trajectory have a significant effect on the resultant implications (crosnoe & benner, 2016; vicente & gabari, 2019). one frequently occurring definition of the idea of engagement is that student attachment exhibits within their studies and affects the intensity with which they are dedicated to a specific educational task (emery et al., 2020). in this instance, when various events lead to deterioration in student engagement, this ignites a process of disengagement (yusof et al., 2018). at this point the situation is being referred to in which a student begins a gradual withdrawal from the educational experience provided by the school (gebel & heineck, 2019). therefore, the various data that have been extracted from the research will demonstrate the types of engagement that more frequently occur within the trajectories of dropout students. the stages whereby students present more positive or negative engagement are also presented. finally, events are identified that produce quality of engagement either in a positive or negative direction. engagement in students´trajectories student engagement, as a construct reflecting the link with school student tasks has been much examined over recent years (cooley et al., 2021). these studies have begun to question how the teaching and learning process has developed based on a school’s history, academic life, and characteristics that result from students (boyaci, 2019; mayhew et al., 2016; teuscher & marakova, 2018). one of the main researchers in this area is astin, who defined student involvement as the amount of energy that a student invests in the educational experience (astin, 1984, 1993). this energy, which is modulated by the learning process, places us in terms of quality (zabalza & zabalza cerdeiriña, 2022) within a process that can be understood as a continuous phenomenon passed through by the individual. the concept of non-involvement or low quality of it within students manifests within the literature as an abstract issue within any field (bernárdez-gómez, 2022). therefore, it is a heterogeneous phenomenon for which there is very little agreement of the mobilisation, hanging, internal or virtual absenteeism, or lack of belonging or affiliation (fernández enguita, 2011). the intersection of all of these aspects is the final project of moving away from common concepts between students and schools and the removal of the feeling of belonging (tomaszewska-pekala et al., 2017; mena, fernández & enguita, 2010). this causes us to reflect on the distinction between schools and students and the potential dichotomy between students who acted in a way that was initiated towards the institution, and those who present with problems. consequently, it is an issue that affects all students, and which has several levels of intensity, different means, and different results. these results have traditionally resulted from the situations of the students within a risk context. trajectories of vulnerable students risk at school affects students who, because they are vulnerable, "suffer learning difficulties with some severity in the educational systems, centres and classrooms we have" (escudero & gonzález, 2013, p. 13). students at risk who experience difficulties in their education are not uncommon and the phenomenon is often seen as an epidemic in the education system (karacabey & boyaci, 2018; rogers, 2021). this makes it ripe for further investigation. 99 engagement in at-risk students trajectories: perspective from the levels of approach to the student / gómez & barea from a wider perspective, the subjects who find themselves in a situation of risk are those who "due to certain personal characteristics and perhaps a set of them, as well as social, community and family, have high probabilities of reaching undesirable results by being exposed to the influence of situations and contexts of risk" (escudero & gonzález, 2013, p. 20). it is important to note from this definition how the subjects, in spite of their singularities, are subject to a large influence by the realities in which they are framed, and stay as the ones that will promote, or not, situations of risk (barros, souza neto, silva, & guedes, 2019). similarly, these students are those who go on to have the highest likelihood of absenteeism, abandonment, and school failure (jurado & tejada, 2019; rubio, 2017) as well as undesirable results and insufficient skills to integrate into patterns of society, family, and working life (gonzález & san fabián, 2018). this results from a danger of the individual not developing their potential because of the risk situations which can go onto affect their educational career. method design considering this study’s objectives and the previous research that has been conducted on this issue (deterding & waters, 2021), this study was conducted using a qualitative approach. in particular, the design had a biographical-narrative nature. by developing this type of methodology, we were able to establish the various events that occur during the student trajectories (brandenburg, 2021) with the aim of gathering and understanding the various relationships that can develop over the course of a life (rodríguezdorans & jacobs, 2020). this methodology therefore tries to make sense of and build meaning from individual events that, via the researcher, are then evoked in the individuals (portela et al., 2019, 2022). this involves using a reflective and introspective process of the individual and contrasting the different events that have occurred within their life or in regard to an aspect of their life. data collection the data collection method was used to conduct research on life histories using semi-structured indepth interviews. this has been determined as one of the best approaches to carry out biographicalnarrative research (rodríguez-dorans & jacobs, 2020) as it allows the researcher to be immersed within the issue and be flexible enough to develop the interviews according to the different needs as they arise (deterding & waters, 2021). the individual aggregate method was used to validate the interviews (traverso, 2019) by producing an initial draught and gaining feedback from field experts. the interviews were structured to have three sections, one was a series of initial questions so that the researcher could verify that the interviewee was suitable for this study and to establish the personal and socio-humanitarian context from which they come. second, a central block was dedicated to investigating each stage of the student’s career: the stage prior to leaving school, the stage at which they were away from the school, and the stage of reincorporation. lastly, a block of questions were aimed to investigate the aspects that were unclear and assess their trajectory as well as their prospects. sample the sample was chosen on a non-probabilistic purposive basis. there were three criteria used for selecting the individuals: a) young people that had been away from school and, subsequently; b) they were at the moment of their participation involved in studying some of the measures aimed to foster their reincorporation into the educational system; c) the size of the sample. this final factor was the most relevant, as twice as many people were chosen as recommended by hernández-sampieri, fernández y baptista (2018) for this research, from three to five individuals. lastly, the sample consisted of 10 people from programmes which were deemed to be encouraging people back into training. in particular, four comprised of basic professional training, four of professional training programmes, and two of entrance exams. the characteristics of these types of reinstatement programmes were specifically for “students at risk of educational exclusion and/or which feature personal characteristics or schooling background which result in a negative appraisal of the school framework” (bernárdez et al., 2021, p. 257). thus, it is guaranteed that one of the candidate requirements is that they have been through a situation of distance and reincorporation into the school. data analysis qualitative data analysis was performed using two complementary techniques: content analysis (friese, 2020) and barton & lazarsfeld's qualitative data analysis model (in taylor et al., 2015). using these two techniques sequentially was done with the support of the analysis software atlas.ti. first, a content analysis was performed, whereby: 1) the information was reduced, and the data was prepared for coding; 2) the data was structured through categorisation and; 3) relationships between the different categories were extracted. second, the process was complemented by the model of analysis as described by barton & 100 december 2022, volume 15, issue 2, 97-107 lazarsfeld (1961): 4) systematising the relationships via code concurrences; 5) making matrix formulations using semantic networks and; 6) conducting a theorysupporting analysis. the benefits of using qualitative analysis by barton and lazarsfeld (1961) was pointed out by glasser & strauss (1967) who described it in analysis that could be conducted using this procedure on both a simple and more complex level. using this procedure has meant there is greater relevance to using this process via support from data analysis software such as atlas.ti (friese, 2020). software helps with monitoring the analytic steps by using tools that help with each stage, as in previous research (bernárdez, gonzález & rodírguez, 2022; bernárdez, portela, nieto & rodríguez, 2022), and which are described below. the categories used in the analysis can be seen in table 1. within which we can highlight the categories that are related to student involvement as perceived by them, the event type, the stage in their educational trajectories in which the event takes place, and the type of event depending on the approach of the student. results and discussion trajectories of the students were analysed and reconstructed, and each has demonstrated the uniqueness of an individual to whom the story belongs. in terms of this paper, we aim to focus on how engagement is located within these life stories and also how engagement is manifested within the quality of the stories and within each of the trajectory stages. conversely, the main events in the student trajectories and their relationship with engagement quality will also be reflected. engagement in students' trajectories first, when we focus on how engagement is reflected in each of the events of the subjects, we can assess that there is a likelihood of an event tending towards a negative nature (figure 1). in both of their critical events there is a greater importance for the subject and in their life events, everyday events, as well as a greater connection with events that lead to a provocation of negative engagement. yet, if we are only to focus on life events there can be no difference between the events that lead to the student distancing themselves or the level of engagement with their studies. cooccurrence coefficients1 of 0.57 and 0.47 are presented, which have a small difference in this aspect. conversely, there is an observed substantial difference between positive and negative critical events, with negative events occurring most frequently. this reveals that in the trajectory of the individuals, events tend to push them towards leaving school. this is the result of the table 1. codes and groups of codes used in the analysis. level of student approach engagement type stages in educational trajectory macrosocial -education and training system -dominant social values -links between training and employment -positive engagement -negative engagement previous primary stage stage away mesosocial -community -family -peer group -educational centre life course theory previous secondary stage reinstatement stage -life events -critical events microsocial -individual characteristics -interpersonal relationships distancing stage timeless 101 engagement in at-risk students trajectories: perspective from the levels of approach to the student / gómez & barea events being decisive in the direction that they take students in terms of their career, with seven points of difference in their co-occurrence coefficient and their quintuplicate number of events. figure 1. co-occurrence between types of events and different qualities of engagement moreover, if we only focus on identifying at which point the events are concentrated according to their positive or negative engagement, we are then only able to highlight two differentiated aspects (figure 2). the events which have the highest negative engagement occur in the stages prior to the student leaving their studies. specifically, there is an outstanding co-occurrence coefficient (0.48) in the stage before the dropout happens. this could be over the result of progression between the stages before the dropout occurs, as the quantity of negative events multiplies, while the quantity of positive events is unchanged. this accords with the events that develop a positive engagement, the majority of which happened within the re-entry stage, with a significantly higher co-occurrence coefficient than in the other stages. figure 2. co-occurrence between different engagement qualities and stages of the trajectories. the factors that emerge according to the quality of engagement as previously mentioned, there are some stages in student trajectories that differ from others because of engagement quality in terms of events that occur within them. thus, the presence of different factors within the stages is heterogenous. in every stage of the different subjects there is event diversity that refers to a number of different factors affecting the school trajectory, however, there are many factors that predominate over others within every stage. these factors are illustrated in figure 3. figure 3. semantic network2 of the relationship between quality of engagement, factors and stages of the trajectory. the semantic network that emerges from these cooccurrences, which we have found between the factors in the various stages, are reflected in table 4. the network is created in terms of different events occurring in each stage and whether the events lead to a positive or negative engagement. table 4. cooccurrences between types of events present in the trajectories and each of the stages they pass through. previous primary stage previous secondary stage stage away reinstatement stage no. quote cooc no. quote cooc no. quote cooc no. quote cooc individual characteristics 11 0.03 49 0.11 26 0.11 51 0.17 interpersonal relationships 27 0.09 64 0.17 10 0.05 7 0.02 community 19 0.08 26 0.08 8 0.06 6 0.03 family 70 0.20 65 0.14 31 0.11 50 0.14 peer group 59 0.16 139 0.34 25 0.08 18 0.04 teachers 54 0.16 79 0.19 20 0.08 44 0.13 educational centre 99 0.25 157 0.34 17 0.04 57 0.13 education and training system 5 0.02 23 0.06 11 0.07 37 0.15 dominant social values 17 0.06 31 0.07 25 0.13 54 0.20 links between training and employment 4 0.02 8 0.02 22 0.16 39 0.17 note. extracted from data analysis in atlas.ti. 102 december 2022, volume 15, issue 2, 97-107 mesosocial factors, influence of the family and its relationship with the center and peers. in the initial stage which occurred before the students left their primary or secondary school, there was a decrease in engagement quality which resulted from factors which were close to the experience of the students. these school related events, as well as those related to family and peer groups, stand out particularly within primary education although they do not have a high co-occurrence coefficient, nor did they have an especially high number of citations. as seen from the student quotations, the relationship with each group is especially relevant. in fact, there is a presence that defines the trajectory of the students who are at risk of exclusion. it helped me because i used to say to dad, i don't understand this, can you help me, but maybe he didn't come to help me and that, i don't know... it bothers me because from childhood to adolescence, he doesn't ask me: hey, what's wrong with you? (d3:35)3 i remember one of them very fondly, one in particular who was my tutor in class. [...] the man must have retired years ago, he was older, he was a good guy, he was a constant source of laughter (d1:41) when addressing each factor individually in terms of the dimensions of the meso social level, we find that a difficulty occurs specifically in terms of the analysis and the breadth of factors that comprise each dimension. beginning with those that demonstrate less relevance in terms of student trajectory and those referring to the community, it can be seen that there is anecdotal evidence indicating a low relevance for students in terms of this factor (ribaya, 2011). this may also show that there is a limited influence in the stage before the distance when the subjects have stated that the environment in which they are developing their studies and which highlights how harmful this can be to their studies as a result of the environment being too distant from the school culture (salvà-mut et al., 2014). if we adhere to the order that determines the relevance of events according to the quantity of appearances that appear in the coding, then the family is presented as a dimension that takes its significance from an influence on how the life stories of students are configured. thus, throughout every stage there is a continual presence of events which are related to it, whereby attending to circumstances lends a trivial cohesion towards the family nucleus (garnica et al., 2019). this includes the excessive responsibility of students with roles to which they do not belong, at least because of age, or little involvement from the family (tarabini, 2018) in their children’s education and, by extension, in what happens in school. conversely, the group of equals is also relevant primarily in the stage before abandonment, which is a stage associated with student friendship and which is significantly influential (cooley et al., 2021; salvà-mut et al., 2014). the primary events showing a student’s stories are linked to those which derive from students and that have a lower educational level and problematic behaviours than those that distance themselves from school and their education. lastly, at the mesosocial level, the school is the primary dimension generating the events influencing student trajectories. it is therefore important for schools in terms of student educational trajectories that factors comprising this dimension are relevant to the lives of students within a psychosocial context. microsocial factors, interpersonal relationships and individual characteristics. at the secondary education stage there is a similar phenomenon, with micro social factors having an increasing relevance in terms of individual characteristics and relationships. upon reaching adolescence they have already begun a stage in that development where the events closest to them have not increased in importance. how they handle their personal characteristics and the influences on them from their closest circle of friends will be decisive in how they develop their trajectory and when the most negative events happen. mathematics has always been very difficult for me, i see a number and i don't know what to do with it. (d2:48) there are teachers who have given me many opportunities, of course, who have helped me [...] and they are good people who have been very good to me (d5:54) conversely, the stage when they are away is not representative of a specifically significant stage for the student. although they do start to have a sense of how macrosocial factors influence their trajectory and in a more significant way. in this way, via the quote we are seeing an example of how dominant social values influence student trajectories when they have moved away from education. i was ashamed to be asked or that we were among us, talking in the group of friends at home and, maybe, we would talk about it: at seven euros an hour for six days 7×6... and i would stay, oh my god, don't ask me. (d9:44) among the various factors at the microsocial level are examples of event diversity that also affect student trajectories. if individual characteristics are considered (monarca, 2017), then issues of security and self-esteem as well as behavioural issues become externalised, including a delinquency or aggression that develop internally as well as depressive states (kang, 2019). according to salvà-mut et al. (2014), 103 engagement in at-risk students trajectories: perspective from the levels of approach to the student / gómez & barea these difficulties influence how a school provides feedback to students when a significant number of them are only referring to the negative experiences experienced during their school time (boyaci, 2019; teuscher and makarova, 2018) because of issues such as low participation or not having a sense of belonging, or a lack of academic skills and learning ability. there may also be issues of educational support needs (yusof et al., 2018). yet those events that are related to the interpersonal relationships of students also have a greater significance as well as being a hindrance to them, because they are deeply connected to events linked to their own individual characteristics. this is because whilst there is an absence of valuable and relevant relationships in both older students and their teachers, there will always be a lack of reference for generating positive experiences in their core where they feel that sense of belonging that is often lacking. macrosocial factors, dominant social values and the employment training system. lastly, and more positively, student engagement is presented as being an improvement during their re-entry phase. events that are related to the dominant social values as well as the relationship between training and employment or specific training lead to a significant increase in engagement. at this stage students see the importance of training because of interventions that are not within their control. this results in them assuming that they are unable to intervene and that it is necessary that they should just align themselves with the issue. they didn't think so much about you, it's like university, you know? ... simply: i'll give it to you (professors giving the homework) if you like it, fine, if you don't like it, if you don't like it at all, get a life, that's it, i wash my hands of it like pilate. in vet they give you the opportunity to meet, to say: i like this, i'm going to do this. and above all, they are looking out for you. (d2:62) because i need a job, and i need to try to pick up everything else, to learn everything i haven't learnt. (d5:83) various events that can be found are reflected in the quotations from the macro social level and the factors affecting the stages of the educational trajectories within the previous levels. this is because many reflect on the actions that are conducted by a person or an individual level as a direct result of culture and society that they are involved in (fernández sierra, 2017; jimenez, 2008; salvà-mut et al., 2014). these are the primary sources from which macro social factors emerge. one example of this is dimension that is related to the educational system. although within life stories events of different programmes and measures are aimed at alleviating the issue discussed, there remain legislative issues towards what happens within schools. these events can make it impossible to devise any improvements or changes at a particular level and therefore these are barriers of a structural nature. another factor that has a basis in every level is that of dominant social values. we know that students take on a self-image based on preconceived ideological patterns (barros et al., 2019; karacabey and boyaci, 2018). this danger results in the assumption of a future eventuality that belongs to a specific ethnic group, that of growing up in a specific environment where there is little value placed on education or that the culture does not place a particular value on education. given this issue, there is a fear that this problem lies within a foundation that is both diffuse and global. conversely, although there is no strong link with other levels of all dimensions, there is a referral to the relationship between training and employment which manifests mainly in reintegrating students into the educational system. the main reason is that the perception is that subjects developed their training and positioning as a form of motivation that helps them to access the job market and presupposes improvements within the expectations they have of the labour market. a major incentive in training is that it is employment orientated, and the subjects therefore assume that they have taken the correct path. yet, this high level of involvement demonstrates that there was a singularity that takes an extensive amount of time to emerge and which leads us to question why, before leaving school, there is a lack of awareness of this specific need. conclusions the apex point of the students examined here is the end product of slowly distancing themselves from the common ideas of the school and moving away from a feeling of belonging and engagement with the school (emery et al., 2020). similarly, this distance is again lowered by the students being re-involved in their studies through various factors producing the same effect. this results in us reflecting on the separation between a student’s school and a potential dichotomy between these units in accordance with the institution and those who are presenting with problems. therefore, it is a phenomenon affecting every student and which has multiple levels of intensity and results (salvà-mut et al., 2014). likewise, it is also a process that is very expressive of the student’s boredom with their school and with the schools having both outdated and rigid (nichol et al., 2016; yusof et al., 2018) teaching methodologies which manifest as tedious and irrelevant for the student’s education. the various events occurring in each student’s school career are neither episodic nor are they disconnected from one another (christodoulou et al. 2018). plus, the aim of constructing their life history is an attempt towards understanding the continuities, events, and 104 december 2022, volume 15, issue 2, 97-107 experiences happening within a school’s trajectories and how they divert and establish the twists and turns in student lives and passage through school (gebel & heineck, 2019; hutchison, 2019). the emergent school trajectories have, more or less, been reflected in the various trajectory models used as a reference (cuconato et al., 2017; tomaszewskapękała et al., 2017). however, there are specific differences between the emerging patterns and those used as a reference which reveal that the trajectory models are more sharply focused on processes of absenteeism; dropping out of school does not fully chart the trajectory of someone returning to training. yet, blasbichler & vogt (2020) state that student perspectives have their own course life, even though their trajectory develops in a specific direction, as they are both dynamic and reversible. they do this by using their own resilience, whereby the situation in which they find themselves is corrected and after which they are unaffected. finally, and in line with the objective set, there are various manifestations of student engagement which are strongly tied to different career stages. this is because of the strong relationship between various factors and specific quality of engagement. there are also factors facilitating student involvement and other factors that facilitate their distance. similarly, it was also verified that, in each trajectory stage, there are factors in which its presence is predominant. funding this research has been financed by the predoctoral studies grants from the ministry of economy, industry and competitiveness of the spanish government (bes2017-081040). ethics this research has been approved by the research ethics commission of the university of murcia (id: 3226/2021) footnotes 1. co-occurrence coefficients are offered by the atlas.ti software from the relationship strength between two codes. 2. in the semantic network, in addition to the different relationships between codes, we will find the total number of citations of a code (letter g) and the relationships with other codes that have been established for that code (letter d). 3. for the citation of the material, the coding offered by the analysis software has been used, where the d indicates the interview number, and the next number indicates the citation within that document. references astin, a. w. (1984). student involvement: a developmental theory for higher education. journal of college student personnel, 25(4), 297308. astin, a. w. (1993). what matters in college? four critical years revisited. josseybass. barros, t. m., souza neto, p. a., silva, i. & guedes, l. a. (2019). predictive models for imbalanced data: a school dropout perspective. education sciences, 9, 275. https://doi.org/10.3390/ educsci9040275 barton, a. h. & lazarsfeld, p. f. (1961). sorne functions of qualitative analysis in social research. in s.m. lipset y n. j. smelser (eds.). sociology: the progress of a decade. prentice-hall, pp. 95-122. bernárdez-gómez, a. (2022). vulnerabilidad, exclusión y trayectorias educativas de jóvenes en riesgo [vulnerability, exclusion and educational trajectories of at-risk youths]. dykinson. https:// doi.org/10.2307/j.ctv2gz3s8d bernárdez-gómez, a., gonzález barea, e. m. & rodríguez entrena, m.j. (2021). educational reengagement programs and measures in the region of murcia. an approach to their understanding. international journal of sociology of education, 10(3), 246-270. http:// doi.org/10.17583/rise.7516 bernárdez-gómez, a., gonzález barea, e. m. & rodríguez entrena, m.j. (2022). narrative perspective of the school trajectory of at-risk students. international journal of learning, teaching and educational research, 21(11), 112128. https://doi.org/10.26803/ijlter.21.11.7 bernárdez gómez, a., portela pruaño, a., nieto cano, j.m., & rodríguez entrena, m.j. (2022). study of educational trajectories through biograms and semantic exploration, an approach to narrative studies. international journal of qualitative methods, 21, 1–14. https://doi. org/10.1177/16094069221141316 blasbichler, a. & vogt, m. (2020). between and beyond. the course of a life in the realms of history of education, general pedagogy and comparative studies. interview with edwin keiner. espacio, tiempo y educación, 7(2). 235247. https://doi.org/10.14516/ete.380boyaci, a. (2019). exploring the factors associated with the school dropout. international electronic journal of elementary education, 12(2),145-156. https:// doi.org/10.26822/iejee.2019257661 105 engagement in at-risk students trajectories: perspective from the levels of approach to the student / gómez & barea brandenburg, r. (2021). i started letting the teachers in’: what factors contribute to successful educational outcomes for disengaged young adults? pedagogy, culture y society, 29(3), 397413. https://doi.org/10.1080/14681366.2020.174710 7 christodoulou, m., bousia, c. & kiprianos, p. (2018). the life–course formation of teachers’ profession. how emotions affect vet teachers’ social identity. raserevista de sociología de la educación, 11(2), 280-296. https://doi. org/10.7203/rase.11.2.12300 cooley, m. e., mihalec-adkins, b. p. & womack, b. (2021). the relational context of school engagement and associations with youth behaviors. journal of emotionaland behavioraldisorders, 29(3), 148161.https://doi.org/10.1177/1063426620945686 crosnoe, r. & benner, a. d. (2016). educational pathways. in m. j. shanahan, j. t. mortimer, & m. kirkpatrick johnson (eds.). handbook of the life course: volume ii (pp. 179-200). springer. https://doi. org/10.1007/978-3-319-20880-0_8 cuconato, m., majdzinska, k., walther a. & warth, a. (2017). student´s decisionmaking strategies at transitions in education. in a. walther, m. parreira do amaral, m. cuconato & r. dale. governance of educational trajectories in europe. pathways, policy and practice (pp. 223-247). bloomsbury. https://doi.org/10.5040/9781474287203.ch-011 deterding, n. m., & waters, m. c. (2021). flexible coding of in-depth interviews: a twenty-firstcentury approach. sociological methods & research, 50(2), 708-739. https://doi. org/10.1177/0049124118799377 emery, l., spruyt, b., boone, s. & van avermaet, p. (2020). the educational trajectory of newly arrived migrant students: insights from a structural analysis of cultural capital. educational research, 62(1), 18-34. https://doi.org/10.1080/00 131881.2020.1713021 escudero, j. m. & gonzález, m. t. (2013). estudiantes en riesgo, centros en riesgo: dimensiones, interpretaciones e implicaciones prácticas. in j. m. escudero (coord.). estudiantes en riesgo, centros escolares de riesgo: respuestas educativas al alumnado en situaciones de vulnerabilidad [students at risk, schools at risk: educational responses to students in vulnerable situations] (pp. 13-46). diego marín. fernández enguita, m. (2011). del desapego al desenganche y de este al fracaso escolar [from detachment to disengagement and from this to school failure.]. cadernos de pesquisa, 41(144), 732-751.https://doi.org/10.1590/s010015742011000300005 fernández sierra, j. (2017). alumnado inmigrante en la eso: vulnerabilidad pedagógica del sistema educativo [immigrant students in eso: pedagogical vulnerability of the educational system.]. educación xx1, 20(1), 121-140, https:// doi.org/10.5944/educxx1.12855 friese, s. (2020). atlas.ti 9 user manual. atlas. ti scientific software development gmbh. https://doc.atlasti.com/manualwin.v9/atlas. ti_manualwin.v9.pdf garnica, m. del c. g., escalona, n. m., páez, a. q. & cordero, g. m. (2019). aproximación interdisciplinar a los retos actuales de protección de la infancia dentro y fuera de la familia [interdisciplinary approach to the current challenges of child protection within and outside the family.]. thomson reuters aranzadi. gebel, m. y heineck, g. (2019). research handbook on the sociology of education. rolf becker. glasser, b. g., & strauss, a. l. (1967). the development of grounded theory. alden. gonzález, m. t. & san fabián maroto, j. l. (2018). buenas prácticas en medidas y programas para jóvenes desenganchados de lo escolar [best practices in measures and programs for out-of-school youths]. reice, revista iberoamericana sobre calidad, eficacia y cambio en educación, 16(2), 4160. https://doi.org/10.15366/reice2018.16.2.003 gottfried, m. & hutt, e. (2019). absent from school: understanding and addressing student absenteeism. harvard education press. hernández-sampieri, r., fernández, c. & baptista, p. (2018). metodología de la investigación [research methodology] (6ª ed.). mcgraw hill education. hutchison, e. d. (2019). an update on the relevance of the life course. perspective for social work. families in society, 100(4), 351-366. https://doi. org/10.1177/1044389419873240 106 december 2022, volume 15, issue 2, 97-107 jurado, p. & tejada, j. (2019). disrupción y fracaso escolar. un estudio en el contexto de la educación secundaria obligatoria en cataluña [disruption and school failure. a study in the context of compulsory secondary education in catalonia.]. estudios sobre educación, 36, 135155. https://doi.org/10.15581/004.36.135-155 kang, d. (2019, abril, 4-5). structural association between teacher-studentrelationship and school engagement among korean elementarystudents [oral communication]. annual meeting of the american educational research association, toronto, canada karacabey, m. f. & boyaci, a. (2018). factors contributing to secondary school dropouts and the dropouts’ socioeconomic profiles: sanliurfa sample. educational administration: theory y practice, 24(2),247-293. https://doi.org/10.14527/ kuey.2018.007 mayhew, m., rockenbach, a., bowman, n., seifert, t., wolniak, g., pascarella, e., & terenzini, p. (2016). how college affects students (vol. 3): 21st century evidence that higher education works. jossey-bass. mena, l., fernández, m. & enguita, j. (2010). desenganchados de la educación: procesos, experiencias, motivaciones y estrategias del abandono y del fracaso escolar [disengaged from education: processes, experiences, motivations and strategies of school dropout and failure]. revista de educación, número extraordinario, 119-145. monarca, h. (2017). políticas públicas de apoyo educativo y configuración de trayectorias escolares [public policies for educational support and the configuration of school trajectories]. magis. revista internacional de investigación en educación, 10(20), 69-84. https://doi.org/10.11144/javeriana.m10-20.ppae nichol, l., defosset, a., perry, r. & kuo, t. (2016). youths' perspectives on the reasons underlying school truancy and opportunities to improve school attendance. the qualitative report, 21(2), 299-300. https://doi.org/10.46743/21603715/2016.2231 portela pruaño, a., nieto cano, j. m. & torres soto, a. (2019). la reincorporación formativa de jóvenes que abandonan tempranamente la educación: relevancia de su trayectoria previa [the educational reincorporation of young early school leavers: relevance of their previous trajectory]. revista española de pedagogía, 77(272), 103-121. https://doi.org/10.22550/rep771-2019-07 portela pruaño, a., rodríguez entrena, m. j., torres soto, a. & nieto cano, j. m. (2022) why vulnerable early school leavers return to and re-engage with education: push and pull reasons underlying their decision. intercultural education, 33(2), 156172. https://doi.org/10.1080/14675986.2021.201820 9 reglamento (ue) 2021/695 del parlamento europeo y del consejo, de 28 de abril de 2021, por el que se crea el programa marco de investigación e innovación «horizonte europa» y se establecen sus normas de participación y difusión, y se derogan los reglamentos (ue) n.o 1290/2013 y (ue) n.o 1291/2013 ribaya, f. j. (2011). la gestión del absentismo escolar [truancy management]. anuario jurídico yeconómico escurialense, 44, 579-596. rodríguez-dorans, e. & jacobs, p. (2020). making narrative portraits: a methodological approach to analysing qualitative data. international journal of social research methodology, 23(6), 611-623. https://doi.org/10.1080/13645579.2020.171 9609 rogers, m. (2021). contextualised, not neoliberalised professionalism in early childhood education and care: effects of prescribed notions of quality on educator confidence in australia: effects of prescribed notions of quality on educator confidence. international electronic journal of elementary education, 13(4), 549-564. https://doi.org/ 10.26822/iejee.2021.211 rubio, j. g. (2017). inclusión y exclusión oculta en la escolarización obligatoria española [inclusion and hidden exclusion in spanish compulsory schooling]. profesorado. revista de curriculum y formacióndel profesorado, 21(4), 119-138. salvà-mut, f., oliver trobat, m. f. & comas forgas, r. (2014). abandono escolar y desvinculación de la escuela: perspectiva del alumnado [school dropout and school disengagement: student perspective]. magis. revista internacional de investigación en educación, 6(13), 129-142. https://doi.org/10.11144/javeriana.m6-13.aede tarabini, a. (2018). la escuela no es para ti. el rol de los centros educativos en el abandono escolar [school is not for you. the role of schools in school dropout.]. octaedro. taylor, s. j., bogdan, r., & devault, m. (2015). introduction to qualitative research methods: a guidebook and resource. john wiley & sons. 107 engagement in at-risk students trajectories: perspective from the levels of approach to the student / gómez & barea teuscher, s. & makarova, e. (2018). students’ school engagement and their truant behavior: do relationships with classmates and teachers matter? journal of education and learning, 7(6), 124-137. https://doi.org/10.5539/jel.v7n6p124 thureau, g. (2018). la asociación española de escuelas de segunda oportunidad. la segunda oportunidad del sistema educativo [the spanish association of second chance schools. the second chance of the educational system]. educar(nos), 81, 11-13. tomaszewska-pękała, h., wrona, a. & marchlik, p. (2017). finding inspiring practices on how to prevent esl and school disengagement. lessons from the educational trajectories of youth at risk from nine eu countries [dataset]. publication 6. project resl. eu. financiado por la unión europea (320223). traverso macías, d. (2019). diseño y validación de un cuestionario sobre la conciencia, regulación y comprensión de las emociones basado en la competencia lingüística [design and validation of a questionnaire on emotion awareness, regulation and understanding based on linguistic competence.]. https://rodin.uca.es/ xmlui/handle/10498/21696 vandekinderen, c., roets, g., van keer, h. & roose, r. (2018). tackling social inequality and exclusion in education: from human capital to capabilities. international journal of inclusive education, 22(1), 1-20. https://doi.org/10.1080/13603116.2017.1 362044 viac, c. and p. fraser (2020), "teachers’ wellbeing: a framework for data collection and analysis", oecd education working papers, no. 213, oecd publishing, paris, https://doi. org/10.1787/c36fc9d3-en vicente, m.i., & gabari, m.i. (2019). relationships between the dimensions of resilience and burnout in primary school teachers. international electronic journal of elementary education, 12(2), 189-196. https://doi.org/10.26822/ iejee.2019257666 yusof, n., oei, t. p. s. & ang, r. p. (2018). voices of adolescents on school engagement. asiapacific education researcher, 27(1), 23-32. https://doi.org/10.1007/s40299-017-0362-1 zabalza, m. a. & zabalza cerdeiriña, m. a. (2022). coreografías didácticas en educación superior. una metáfora del mundo de la danza [didactic choreographies in higher education. a metaphor for the world of dance]. narcea. international electronic journal of elementary education, 2016, 8(3), 455-466 a general investigation of the in-service training of english language teachers at elementary schools in turkey ebru melek koç  inonu university, elt department, turkey received: 29 june,2015 / revised: 17 november,2015/ accepted: 19 november 2015 abstract this study presents a critical diagnosis of in-service teacher-training activities offered to englishlanguage teachers in turkey and aims to investigate whether those teachers are satisfied with the activities. thirty-two english-language teachers participated in this study. data were collected from 32 elementary-school teachers of english as a foreign language, using a general evaluation form prepared by the researcher. the results indicate that the teachers are not satisfied with their inservice teacher-training activities and that in-service training does not fulfil their needs. the study also proposes an in-service teacher training model in distance format. keywords: in-service teacher training, english language teachers, elementary school education. introduction ‘continuousness’ is the basic concept underlying ‘lifelong education.’ high-quality education is based on teacher quality, and ‘continuousness’ is one of the most important factors in teacher training. ‘continuous professional development—cpd’ consists of four similar parts: 1) pre-service or initial teacher education; 2) in-service teacher training (inset); 3) further education; and 4) vocational training/education, which is the underlying principle of ‘lifelong education’ (eca, 2006, p.7). the constant vocational training that takes place during teacher training is categorized as either pre-service or after-service. studies indicate that the pre-service teacher-training programs are inadequate to provide a sufficient set of skills (can, 2005; lucas & unwin, 2009). thus, teachers need inset to fill in the gaps from pre-service training and for continuous professional development, which keep teachers up-to-date throughout their careers with respect to the skills required in a contemporary knowledge-based society. ryan (1987) mentions that inset refers to any type of activities such as courses, and seminars related to the job. in this sense, any kind of teacher training activities such as short courses, seminars, workshops, certificate/diploma programs and postgraduate  ebru melek koç, inönü university, elt department, malatya, turkey. phone: +90 422 377 4176, e-mail: ebrumelekkoc@gmail.com international electronic journal of elementary education vol.8, issue 3, 455-466, 2016 456 programs, which result in professional development of teachers, are regarded as a part of inset. inset in turkey inset varies significantly from one country to another because each country has its own policies. in turkey, the first organized inset began in 1960 with the establishment of the ‘office of training teachers on the job’ in the ministry of education (me). this office became the ‘department of in-service training’ in 1975, but the centre was only able to provide training to a limited number of teachers because of financial difficulties and inadequate office space. the ministry of education has combined the approaches of the local and central administrations and since 1993, it has continued to provide a practical inservice teacher-training program for large groups of people. since 1993, central in-service trainings in turkey have been conducted by the ministry of education’s department of inservice teacher training, and local trainings have been conducted by the provincial directorates for national education (pdne). the training activities are arranged cooperatively by the me and the pdne and are conducted face-to-face, either centrally or locally. a current list of the inset courses (in turkish) is presented on the official page of me (edb.meb.gov.tr/net/_standart_program/index.php?dir=standart+programlar%2f). in-service training seminars are arranged by taking the location of the teachers into consideration. the duration of the courses can vary, but they generally last for approximately 25 to 120 hours. the courses are led by instructors at the me or pdne; instructors from nearby universities occasionally deliver the lectures. the content of each course is determined by the instructor. at the end of the activity, the participants are asked to fill an evaluation or feedback form. it is important to note that inset refers to any in-service teacher training activities organized by me and pdne. inset for english language teachers turkey, has been implementing educational reforms for years. recently, a radical change called 4+4+4 was introduced to the turkish education system. this new system, aims to divide the educational system into three main periods (primary/secondary/high school), increase the compulsory education period to the average established in eu and oecd countries and provide higher quality education. this new system has also been accompanied by new reforms in the teaching of foreign languages, such as starting foreign language learning at an early age, specifically, at the primary school level. this change has increased the importance of not only ‘teaching english as a foreign language’ but also the ‘quality of english-language teachers.’ to teach effectively, an english teacher must possess both adequate subject-matter knowledge and the required skills. however, elementary-school english teachers have been found to have difficulties teaching young learners (haznedar; 2003; hüttner, mehlmauer-larcher, reich & schiftner, 2012; i̇şpınar, 2005; khandehrou, 2011; lamie, 2002; nicolaids & mattheoudakis, 2008; salı, 2008). teaching english to elementary-school students requires special knowledge and pedagogical skills related to the new curriculum: language acquisition and development among young learners; teaching methods and techniques; the effective use of audiovisual materials; understanding individual differences and collaborative learning; and classroom management (akyel, 2003; güven, 2005; i̇şpınar, 2005; olivia, 1968; öztürk, 2006). the needs of english teachers who instruct young learners are different from those of teachers who instruct older learners (akyel, 2003; güven, 2005; özdemir, 1998). therefore, inset’s content for elementary-school english teachers should be unique (reilly & haworth, 2001). the literature also supports the idea that elementary-school teachers of english as a foreign language require training in a variety of subjects (al-mutava, 1997; in-service training of english language teachers / koç 457 chacon, 2005; eslami & fatihi, 2008; hazneder, 2002; krol et al., 2004; özbay, 2009; özdemir, 2007; polat, 2010; salı, 2008; sevinç, 2006; symeonidou & phtiaka, 2009). that notwithstanding, turkey’s inset programs primarily cater to general classroom teachers (çiftçi, 2008; demirtaş, 2008; maral, 2009; şahin, 1996), science teachers (kanlı, 2001; kaya et al., 2004; tekin & ayas, 2006), and preschool teachers (kıldan & temel, 2008; uşu & cömert, 2003). english teachers are offered very few opportunities to participate in inset. indeed, teachers’ subject areas are ignored when organizing inset programs. therefore, current inset practices are ineffective and fail to meet all teachers’ needs (coşkun, 2014; division of research & development in education (earged), 2006; önen et al., 2009). to develop their subject knowledge, gain the required skills, and become knowledgeable about current technological developments, elementary english-language teachers must be trained sufficiently to achieve effective foreign language teaching and learning in the classroom. little research has been conducted to evaluate the professional training programmes offered to elt teachers (cooper & keefe, 2001; owsten et al., 2008; young & lewis, 2008). this study aims to investigate the experiences of elementary-school english-language teachers’ experiences who have participated in inset activities offered by me and pdne. the underlying research question is: what are the perceptions of english language teachers at elementary school about inset activities offered by me and pdne? methodology participants the participants in the study are 32 elementary-school teachers of english as a foreign language who have participated in various inset activities organized by me and pdne. the teacher’s amount of teaching experience differs from two years to 22 years. 25 of them are female and 7 of them are male. data collection tools and procedures aminudin (2012) mentions five features of effective professional development: content focus, active learning, collective participation, duration and coherence. these features were taken as core concepts when constructing the items of the present questionnaire. besides, the items of the inset evaluation scale developed by tekin and yaman (2008) for science teachers were adapted. to obtain a general view of english-language teachers’ perceptions of inset activities, the researcher prepared an evaluation form consisting of three main parts. the first part included questions about the teachers’ demographic information, such as their gender and level of teaching experience. research has confirmed that there is not a significant difference between the impact of professional development in classroom practices and years of experience (robinson, 2011), and that there is no significant correlation between satisfaction with professional development and age of the teachers (hustler et al., 2003). therefore, further statistical analysis regarding these variables was not conducted. the second part included eleven statements related to the inset activity’s content, instructor, and evaluation. in the last part, the participants were asked to write a detailed account of their experience related to their participation in the inset activity. international electronic journal of elementary education vol.8, issue 3, 455-466, 2016 458 data analysis and data collection the questionnaire data were analysed descriptively using a statistics program. item analysis was applied, which means the frequencies and percentage scores of each item were calculated. the scores for ‘strongly agree’ and ‘agree’ were merged and defined as ‘agree.’ similarly, ‘strongly disagree’ and ‘disagree’ were merged and scored uniquely as ‘disagree.’ the evaluation and intent forms were attached to an email sent to state elementaryschool english-language teachers, soliciting them to volunteer to participate in this study. out of 102 teachers, thirty-two of them responded positively. results and discussion a very significant finding of this study is that more than half of the teachers (62.2%) indicated that the inset activities were not relevant to their needs. many studies have come to the same conclusion: teachers’ real needs have not been met by inset activities (çiftçi, 2008; çimer et al., 2010; gökdere & çepni, 2004; karagöz, 2006; önen et al, 2009; özer, 2001; öztürk & akar; 2005). content-focus is one of the most important features of an effective inset activity (aminusin, 2012; birman, desimone & garet, 2000). in the same line, content-specific inset activities are reported to be their most beneficial professional development experience by teachers in a variety of studies (aminudin, 2012; robinson, 2011) similarly, research by education, research and development (earged, 2006), which was conducted in 14 provinces and gathered data from 1067 teachers, revealed that inset is neither efficient nor effective at fulfilling teachers’ educational needs. one possible reason for this dissatisfaction is that in-service training activities have not taken into consideration teachers’ various ranks (gökdere & çepni, 2004). most of the teachers (65.1%) indicated that the content covered in the inset activities was clear and comprehensive. however, most reported that the inset activities were not motivating (67.9%) and did not allow for active participation (79.6%). these findings are supported by similar research, which has found that the methods applied during inservice training are neither efficient nor proper (çalgan, 2008; çimer et al., 2010; öztürk & akar, 2005), nor are they motivating (çimer et al., 2010). the reason that the inset activities are not motivating and interactive could be related to the lecturers’ teaching approaches and teaching abilities in the field. this assumption is supported by research that finds that inset instructors are not necessarily leading experts in the field (earged, 2006; çimer et al, 2010; özer, 2001). in this line, harland (2014) suggests that inset based on constructivist approaches is more effective. similarly, related literature reveals that the tools and materials used at inset training have not been updated to meet recent scientific and technological developments (taymaz et al., 1997) this could provide another reason that inset is not motivating. another finding of this study is related to the evaluation of inset activities. the findings of the present study revealed that only 39.3% of the teachers think that the evaluation process for inset activities is satisfactory. this could be due to the fact that out of the four evaluation steps, only the ‘reaction’ step is achieved through disturbing an evaluation form to the teacher participants, which is the cheapest way. on a related note, the teachers stated that they could not apply the knowledge from their inset activities in their classrooms. this finding is supported by kanlı (2001), who finds that teachers were not able to actualize the knowledge that they gained. similar studies have also found that the end-of-training assessments are inefficient and not scientific (earged, 2006; çimer et al., 2010; taymaz et al., 1997). taken together, these findings indicate that the in-service training of english language teachers / koç 459 ‘assessment’ period constitutes another deficiency of meb and pnde’s in-service trainings. the teachers’ success is not evaluated at the end of the training; instead, they are asked to evaluate the inset activity in which they have participated. according to kirk patrick (1959) there are four steps for evaluating a learning process. : reaction (how well did the learners like the learning process?), learning (the extent to which the learners gain knowledge and skills), behaviour (capability to perform the newly learned skills)and results (investigation of the noticeable results of the learning process in terms of reduced cost, improved quality, efficiency, etc). the purpose of an in-service teacher-training program is to enable teachers to develop their knowledge, apply this knowledge in the classroom, and achieve the projected behavioural changes. to effectively measure the program’s success, the teachers’ behaviours and knowledge levels both before and after the in-service teacher-training program should be compared. if the teachers’ knowledge level has increased, if they display the projected behaviours, and if they can apply the acquired knowledge in the classroom, then the in-service teacher-training program is successful. however, in the training activities conducted by the me and pdne, no assessments of the teachers are made either during or at the end of the course. in addition to their responses to the statements on the second part of the evaluation form, the teachers were asked to write about a personal experience in an inset activity on the third part of the form. one example of a response (from a female teacher) is stated below. she indicated that although she did not find the inset experience useful, it was interesting to meet different teachers around turkey and share ideas. she also suggested that the group size during training should be small. ‘a few years ago, i participated in an inset activity titled ‘english teaching methods and techniques.’ it was a weeklong seminar, and i found it partially useful. i do not think that the content was different from the training i received during my undergraduate education at the university. in that sense, the inset program did not add to my current knowledge. one thing i liked about the seminar was that i had the opportunity to meet other english-language teachers from other schools and other cities, and we were able to exchange ideas. however, the group was very crowded. i think it would be more effective if the group size was smaller.’ similarly, another teacher mentioned her negative inset experience in regards to the course instructor. ‘i have been teaching for 14 years. so far i have participated in three inset activities. however, i do not think that these either have added to my current subject knowledge or have helped to develop my teaching skills. one of the courses i participated in was ‘methods and techniques in practical speaking’ organized by pdne in balıkesir. it was a five-day seminar. the course instructor was a friend of mine. she participated in the same seminar organized by me in ankara, and she was tasked to be instructor of this seminar in balıkesir. it is a well-known fact that most of the english language teachers are not very good at practical english. so, how could such a course instructed by my friend can be effective? such courses should be instructed by native speakers of english. what is more, english language teachers should be provided with opportunities to participate in inset courses abroad.’ international electronic journal of elementary education vol.8, issue 3, 455-466, 2016 460 conclusion this study aimed to investigate english-language teachers’ perceptions of the inset activities offered by the me and the pdne, which have played a very important role because they are the only government agencies in turkey to offer in-service teachertraining programs. significantly, this study’s findings are consistent with the findings of similar studies in related areas and reveal that the in-service teacher-training programs run by the me and pdne are ineffective and do not fulfil the needs of elementary-school english-language teachers. in this line, the findings of the present research could be used for the continued development of in-service english language teacher training, and in turn for increasing the quality in foreign language teaching at primary schools in turkey. another importance of the findings of the present research for the continued development of language teaching in turkey is that the present study could provide a database for future research in turkey, which could provide turkish authorities and stakeholders such as ministry of education department of in-service teacher training and the provincial directorates for national education (pdne) with the information to understand the underlying reasons for such needs, and in turn create possible solutions for language teachers’ professional development. the present study is significant in that it has confirmed that the inset model used by me and pdne is not effective for professional development of english language teachers. taking the two facts into consideration; that inset plays a vital role in teachers’ professional development, and that the in-service training offered by the me is not effective, this study suggests an urgent call for developing new inset activities based on the needs of english language teachers in turkey and proposes a new inset model. the proposed new model is based upon the characteristics of what makes a high quality of inset programme and distance education. it will both provide a more effective training for english language teachers and also set a good example model for in-service training programs implemented in other subjects. the components of the newly proposed in-service teacher training model are: 1) needs analysis: identification of needs is the most important issue for the success of a programme (daloğlu, 2004; o’sullivan, 2001; ruba, 1985). so, needs analysis, which is the basic part of developing a programme, is very significant in that the results of it will provide inset programme developers with the necessary information to design a course specific to english language teachers’ needs. 2) distance learning mode: the main reason why inset should be offered by distance is that distance learning is flexible in terms of time and place, therefore enables a wide range of english language teachers to be trained. in turkey, many state universities have postgraduate programmes in elt, which are offered face-to-face. however, when teachers in rural areas are considered, face-to-face format does not seem to provide every teacher with the opportunity to be trained professionally. inset by distance education could minimize this limitation. 3) cost of free online inset: inset (offered by higher education institutes and accredited private language schools) in most countries require a course fee, which all teachers may not afford. in order to provide professional development opportunities for a wide range of english language teachers, online and cost of free in-service teacher training should be offered. in-service training of english language teachers / koç 461 4) collaboration between the council of higher education and the ministry of education (meb): designing an online programme has ‘educational’ and ‘organisational’ aspects. the educational aspect requires collaboration of a variety of subject experts in the field of english language teaching, educational technology, programme development, and distance education, which could be achieved under the responsibility of the council of higher education. the ministry of education could participate in collaborate by implementing the programme, and acting as a liaison between the teachers’ and the council of higher education. 5) active participation: according to ‘interactive learning’ and ‘social-constructivist’ theories, the more interaction occurs among the components of the programme (student-materials-instructor), the more efficient the learning will be. richness of the social environment of the individual is an important factor and it is very significant for the individual to learn effectively to be involved in an interaction with this environment. designing education contexts involving different types of interactive treatments, which allow students to communicate more among their peers and the course materials may lead to effective teaching, and thus increase the effectiveness of distance education (beldarrain, 2006; moore, 1989). in line with this approach, teachers should be provided with opportunities for maximum interaction, and therefore learn actively in an interactive environment onlinementor support: another important factor for effective learning is the concept of ‘scaffolding’ (vygotsky, 1978). it refers to learning with the help of someone who has more information and is more experienced. it is very important that the experts at the english language teacher training departments of the universities take the role ‘mentors’ in the proposed inset program. in the proposed model the mentors are responsible to provide online constant and configured consultancy service for the teachers attending the distant in-service training program by giving feedback on teachers’ lesson plans, and their teaching performances in the classroom, making explanations to the questions about the points of the content of the subject. besides giving feedback, the mentors are also responsible for continuous assessment of teachers via e-portfolios, which include sample lesson plans and video recording of teacher classroom performances. in this sense, it has been hypothesized that continual online mentor support during the in-service training could result in effective learning. 6) assessing the impact of inset: the purpose of the in-service teacher training is to develop the subject knowledge of the teachers and also to change their teaching behaviours positively. the proposed model accepts kirk patrick’s four-level evaluation model. for short term assessment (reaction), which is also considered as the general evaluation of the programme, interviews and evaluation questionnaires can be used to evaluate inset from a general aspect. the new model uses pre and post assessments to investigate the extent to which teachers gain new knowledge (learning). that is teachers’ subject knowledge and teaching skills should be assessed prior and then after the inset. to determine whether any changes in the teaching behaviours have occurred (behaviour), teachers are observed for an extended period of time after the inset activity. such assessment could give information about the ‘long-term impact’ of teachers, which is an aspect ignored by programme developers. this study also suggests that further research studies should be conducted through a collaboration between the me, pdne, and related departments of the universities to develop and implement new in-service teacher-training models. the application of more international electronic journal of elementary education vol.8, issue 3, 455-466, 2016 462 effective models for in-service english-language teacher training will both improve the quality of the teachers and increase the achievements of foreign language students. this study investigated the perceptions of english-language teachers at the elementaryschool level. a future study could also examine english-language teachers at the secondary and high-school levels and investigate whether the three groups of teachers differ significantly in relation to their perceptions about inset activities. a second suggestion should be proposed for a research that gathers qualitative data that provides a deeper understanding of the weaknesses of current inset programs. • • • references akyel, a. (2003). yabancı bir dil olarak i̇ngilizce eğitimi ve öğretiminde yaşanan sorunlar, çözümler ve gelişmeler [english language teaching as a foreign language and problems of teaching it,solutions and developments]., in doğan,i̇. (eds.), proceeding of the . early childhood education congress (pp. 97-103), i̇stanbul: özyurt matbaacılık. al-mutawa, n. a. (1997). evaluation of efl primary school teachers’ competencies in kuwait. evaluation and research in education, 11(1) 38-52. beldarrain, y. (2006). distance education trends: integrating new technologies to foster student interaction and collaboration. distance education, 27(2), 139–153. birman, b, f., desimore,l., porter, a. c., & garet, m. s. (2000). designing professional development that works. educational leadership, 57(8), 28-33. can, a. (2005). i̇ngilizce öğretmeni yetiştirme programının kazandırdığı konu alanı öğretmen yeterliliklerinin standartlara gore değerlendirilmesi[evaluation of english language teaching programme in regards of the competencies gained]. (unpublished phd thesis), university of ankara, turkey. chacon, c. (2005). teachers perceived efficacy among english as a foreign language teacher in middle schools in venezuela. teaching and teacher education, 21, 257-272. cohen, l. & manion, l. (1994). research methods in education (4th ed.). london: routledge. çoşkun, k. (2014). determining the in-service training needs of guidance teachers. pakistan journal of statistics, 30(6), 1389-1404. çalgan, s. p. (2008). the viewpoint of english teachers in private schools in in-service trainings (unpublished master’s thesis), university of beykent, i̇stanbul, turkey. çiftçi, e. (2008). an analysis of education given to in-service music teachers by ministry of national education in turkey and determining the needs for in-service training of music teachers (unpublished phd thesis),, gazi university, ankara, turkey. çimen,. s, çakır, i̇., & çimen, a. (2010). teachers’ views’ on the effectiveness of in-service courses on the new curriculum in turkey. european journal of teacher education, 33(1), 31-41. daloglu, a. (2004). a professional development program for primary school english language teachers in turkey: designing a materials bank. international journal of educational development,, 24(6), 677-690. demirtaş, t. z. (2008). an evaluation of the relationship between teachers’ perception of intuitional communication and in service training in primary schools (unpublished master’s thesis), yeditepe university, istanbul, turkey. in-service training of english language teachers / koç 463 division of research & development in education (earged). (2006) 21. yüzyıla girerken türk eğitim sisteminin ihtiyaç duyduğu çağdaş öğretmen profili[ a modern teacher profile needed in the turkish education system in the 21th century].: 2006. retrieved from http://earged.meb.gov.tr in 2010 eslami, z. r., & fatahi, a. (2008). teachers’ sense of self-efficacy, english proficiency, and instructional strategies: a study of non-native efl teachers in iran. teslej, 11(4)1-19. eslami, z. r. (2010). teachers’ voice vs. students’ voice: a needs analysis approach of english for academic purposes (eap) in iran, elt, 3(1), 3-11 gökdere, m. & çepni, s. (2004). üstünyetenekli öğrencilerin fen öğretmenlerinin hizmetiçi ihtiyaçlarının değerlendirilmesine yönelik bir çalışma [a study on the in-service needs of teachers of the gifted students].. gazi university journal of education faculty, 24(2), 1-14. güven, s. (2005). the profile and the perceptions of professional competences of the first stage primary school efl teachers (unpublished master’ thesis), mersin university, mersin, turkey. haznedar, b. (2003). çocukta ikinci dil egitimi: sorunlar, çözümler [child second language learning: problems, solutions], in doğan,i̇. (eds.), proceeding of the . early childhood education congress (pp. 119-130), i̇stanbul: özyurt matbaacılık. hüttner, j, mehlmauer-larcher, b., reich, s., & schiftner, b. (eds) (2012). theory and practice in efl teacher education: bridging the gap, bristol: multilingual matters. harlands, j. & kinder, k.(2014). teachers’ continuing professional development: framing a model of outcomes. professional development in education, 40(4), 669-682. i̇şpınar, d. a. (2005). study on teachers’ awareness of teaching english to young learners (unpublished master’s thesis), çukurova university,adana, turkey. kanlı, u. (2001). effectiveness of in-service training programs for secondary school physics teachers (unpublished master’s thesis), gazi university, ankara, turkey. karagöz, b. (2006). an investigation on art-teachers’ participation in ministry of national education’s in-service training activities and their reflections on program effectiveness (unpublished master’s thesis), gazi university, ankara, turkey. kaya, a., çepni, s. & küçük, m. (2004) . fizik öğretmenleri için üniversite destekli bir hizmet içi eğitim model önerisi [university-supported in-service education model proposal for physics teachers]., the turkish online journal of educational technologytojet, 3(1), 112-119. khandehroo, k., mukundan, j., & alavi, z. k. (2011). professional developmental needs of english language teachers in malasia. journal of international education research, 7, 45-52. kıldan, a.o. & temel, z. f. (2008). the effects of in-service training practices for preschool teachers in line with the constructivist approach to determine teachers’ perceptions. journal of kastamonu education faculty, 16(1), 25-36. krol, k., janssen, j., veenman, s., & linden, j. (2004). effects of a cooperative learning program on the elaborations of students working in dyads. educational research and evaluation, 10(3), 205237. hustler, d., mcnamara, o. jarvis, london, m., camphell, a.,& houson, j. (2003). teachers’ perceptions of continuing professional development. research report no 429, manchester metropolitan university. lamie, j. (2002). an investigation into the process of change: the impact of in-service training on japanese teachers of english. journal of in-service education, 28(1), 135-162. lucas, n. & unwin, l. (2009). developing teacher expertise at work: in-service trainee teachers in colleges of further education in england. journal of further and higher education, 33(4). maral, d. y. (2009). determination of classroom teachers' adequacy of assessment and evaluation and their in-service education requirements (unpublished master’s thesis), onsekiz mart university, çanakkale, turkey. international electronic journal of elementary education vol.8, issue 3, 455-466, 2016 464 michel, j. (1967). foreign language teaching: an anthology. the macmillan: new york. moore, m. g. (1989). three types of interaction. the american journal of distance education 3(2), 16. nicolaidis, k. & mattheoudakis, m. (2008). utopia vs. reality: the effectiveness of in service training courses for efl teachers. european journal of teacher education, 31(3)279-292. olivia, p. f. (1969). the teaching of foreign languages. prentice-hall; new jersey. o’sullivan, m. c. (2001). the inset strategies model: an effective inset model for unqualified and underqualified primary teachers in namibia. international journal of educational development, 21, 93-117. önen, f, mertoğlu, h, saka, m & gürdal, a. (2009). the effects of in-service training on teachers’ knowledge about teaching methods and techniques. ahi evran university journal of faculty of education, 10(3), 9-23. özbay, a. f. (2009). yapılandırmacılık kuramına dayalı olarak i̇ngilizce dersinin işlenişine ilişkin öğretmen görüşleri [english teachers ‘opinions on teaching english upon constructivist approach in english lessons ]. (unpublished master’s thesis), afyon kocatepe university, afyon, turkey. özdemir, v. a. (1998). study on efl teaching in i̇çel anadolu high school (unpublished master’s thesis), mersin university, mersin, turkey. özdemir, ö. (2007). i̇lköğretim birinci kademe i̇ngilizce öğretmenlerinin eğitim durumunda yöntemteknik ve araç-gereç (teknoloji) kullanma yeterlilikleri [the competency of primary school english teachers in the usage of methods-techniques and materials (technology]. (unpublished master’s thesis), çanakkale onsekiz mart university, çanakkale, turkey. öztürk, a. (2006). an analysis of elt teachers’ perceptions of some problems concerning the implementation of english language teaching curricula in elementary schools (unpublished master’s thesis), gaziantep university, gaziantep, turkey. polat, n. (2010). a comparative analysis of pre-and in-service teacher beliefs about readiness and self-competency. revisiting. teacher education for ells, system, 38, 228-244. reilly, d. h. & haworth, s. (2001).coordination of change for practicing. teacher education, 104(1), 12-16. ruba, p. a. (1985). chemistry teachers’ inservice needs: are they unique? journal of chemical education, 58(5), 430-431. salı, p. (2008). novice teachers’ perceived challenges and support needs in their journey to become effective teachers (unpublished phd thesis), anadolu university, eskişehir, turkey. sevinç, k. ü (2006). i̇lköğretim okullarında i̇ngilizce öğretiminde karşılaşılan güçlüklere ilişkin öğretmen dörüşlerinin değerlendirilmesi [the analysis of teachers' views regarding the difficulties in teaching of english as a foreign language in turkish primary schools]. (unpublished master’s thesis), dicle university,turkey. symeonidou, s. & phitiaka, h. (2009). using teachers’ prior knowledge, attıtudes, and beliefs to develop in-servıce teacher education courses for inclusion. teaching and teacher education, 25, 543-550. şahin, m. (1996) .the problems faced in the in-service training activities of the ministry of national education (unpublished master’s thesis) ,gazi university, ankara, turkey. taymaz, h, sunay, y. & aytaç, t. (1997). hizmet içi eğitimde koordinasyon sağlanması[achieving coordination in in-service teacher training]. milli eğitim dergisi, 133, 13-17. tekin, s. & ayas, a. (2006). determining chemistry teachers’ in service needs: the case of trabzon. journal of hacettepe university faculty of education, 31,169-178. in-service training of english language teachers / koç 465 tekin, s. & yaman, s. (2008). the evaluation scale of in-service education program: the development of teachers’ form. ahi evran university kırşehir education faculty journal (kefad), 9(3), 15-16. uşun, s. & cömert, d. (2003). okul öncesi öğretmenlerinin hizmetiçi eğitim gereksinimlerinin belirlenmesi.[.determining the in-service education needs of pre-school teachers]gazi education faculty journal, 23, 125-138. vygotsky, l. s.(1978).mind in society. harvard university press, cambridge. young, a. & lewis, c. w. (2008). teacher education programmes delivered at a distance: an examination of distance student perceptions. teaching & teacher education, 24(3), 601-609. international electronic journal of elementary education vol.8, issue 3, 455-466, 2016 466 appendix 1 general evaluation form for inset activities (english-language teachers) part a. demographic information 1. gender: □ female □ male 2. how much teaching experience do you have? □ 1-3 years □ 3 5 years □ 5-10 years □ 10+ years 3. which age group of students do you teach? (you may select more than one option.) □ elementary □ secondary □ high school part b. please indicate the extent to which you agree or disagree with each statement below. 1 2 3 4 5 strongly disagree disagree neutral agree strongly agree no. statement 1 the inset activities (seminars/workshops, etc.) were relevant to my needs. 2 the lecturer was well prepared and an expert in the field. 3 the inset activities were engaging and interactive. 4 the inset activities allowed for active participation. 5 the inset content was well organized. 6 the inset content was clear and comprehensive. 7 the inset activities (seminars/workshops, etc.) added new content to my current knowledge base. 8 the inset activities (seminars/workshops, etc.) provided me with new skills to add to my current language teaching skills. 9 i can apply the knowledge and skills i have gained in the inset activities in the classroom. 10 the overall evaluation of the inset program was satisfactory. 11 the organization of the inset program was satisfactory. part c. please comment on your personal experience with the inset activities. boylan.pdf international electronic journal of elementary education vol.1, issue 1, october 2008. ! issn:1307-9298 copyright © iejee www.iejee.com ! ! exploring elementary students’ understanding of energy and climate change colin boylan charles sturt university, school of education cboylan@csu.edu.au abstract as environmental changes become a significant societal issue, elementary science curricula need to develop students’ understanding about the key concepts of energy and climate change. for teachers, developing quality learning experiences involves establishing what their students’ prior understanding about energy and climate change are. a survey was developed to explore what elementary students know and understand about renewable and non-renewable sources of energy and their relationship to climate change issues. the findings from this survey are reported in this paper. key words: elementary students, energy, climate change introduction one of the current challenges facing people around the world focuses on energy. in particular, issues surrounding access to continuing energy sources, the rate of usage of known non-renewable energy resources, the use of renewable energy sources, the impact that society’s use of energy can have on the environment and earth’s climate are regularly reported in the media. for elementary students, these are key issues that will have a major impact on their quality of life both in the short term and in the longer term. for elementary teachers, the challenge is to teach about energy and climate change in ways that promote deep knowledge and deep understanding of these concepts. in the new south wales education system, elementary students (k-6) engage in the study of science and technology through a curriculum document that brings both science and technology studies together. the science and technology k-6 syllabus international electronic journal of elementary education vol.1, issue 1, october 2008 ! 2! ! (www.boardofstudies.nsw.edu.au) provides the direction and focus of developing teaching and learning experiences for elementary students. related literature student prior knowledge through the research of corney (2000), dawson (1997), driver (1985), osborne and freyberg (1985) and tytler (2002), some recurring themes about students’ prior knowledge of science related topics can be made. these themes are: 1) students bring to their studies their own ideas about science and environmental concepts; 2) these pre-conceived ideas are formed through prior experiences and they are very persistent; and, 3) the pre-conceived ideas are difficult to change. for the classroom teacher, the underlying message from these authors draws attention to how important it is for teachers to establish what students do know, do not know, and partially know about science concepts as the first step in quality teaching practices. these authors then suggest that elementary teachers design their teaching and learning programs in ways that build from what the students know and provide opportunities to engage in experiential learning practices that will lead to elementary students developing new and deeper understandings. understandings about energy as part of the learning in science project, the new zealand researchers concluded that the concept of energy is a difficult concept to teach (osborne and freyberg, 1985). kirkwood and carr (1988) investigated the concept of energy using both elementary and secondary students. their research found that children across the world typically perceived energy as something which: 1. is a general kind of fuel that does work for us. 2. is associated with living things often linked to terms including energetic, and human-centred. 3. is associated with moving things, e.g. fire, cars, ringing telephones. 4. can take on different forms as it travels through wires or chains of bicycles. 5. is a source of force or activity stored in objects, for example, water has energy in it so it can turn a water wheel. 6. is a storehouse used to make things work such as a battery. 7. can be obtained from food, the body, sun, and soil, it is regarded as an ingredient stored in them. 8. is a fluid-like material that flows from one body to another, as an electric current or a stream. 9. is given off like a waste product, for example, chemicals give off heat. exploring elementary students’!…& boylan ! 3! ! from this set of observations, it is clear that students do hold a variety of ideas about the nature of energy. the diversity and reported persistence of scientific and non-scientific understandings about energy has been shown to extend beyond school (trumper, 1997). in his study, trumper (1997) explored the ideas that pre-service elementary teachers hold about the concept of energy. his analysis revealed a number of persistent misconceptions that pre-service teachers hold including: 1. hold a number of different, alternative conceptual frameworks when describing physical situations involving energy, instead of the accepted scientific concept. 2. mostly think that energy is a concrete entity. 3. mostly do not accept the idea of energy conservation. 4. are ambiguous in their recognition of different types of energy. 5. mostly confuse the concepts of energy and force. in the investigation reported in this paper, the more specific link between energy and environment related issues is the central focus. very little research literature was found that specifically dealt with elementary students’ understanding of concepts such as sources of energy, climate change, and renewable and non-renewable energy sources. corney (2000) noted also that very little research had been conducted that sought to establish what students knew about environmental concepts. one study by papadimitriou (2004) reported that both students’ and adults’ understanding about environmental concepts, including climate change, are often incorrect as well as being very persistent to teaching interventions. in papadmitriou’s (2004) research with pre-service elementary teachers the following findings revealed a set of common misconceptions held by many of the students: 1. weather effects and climate change are often confused. 2. climate change is linked to air pollution and environmental pollution. 3. climate change is incorrectly associated with ozone layer depletion. through her research, the author sought to explicate the deep science conceptual understandings held by these pre-service teachers. she concluded that the concept of energy in its various forms (electromagnetic radiation, heat and chemical) was a difficult concept which these students did not fully understand. additionally, environmental terminology misconceptions were identified by boyes and stanisstreet (1997) which led to the inclusion of one question in the student survey (appendix 1: question 8) that sought to determine if students were able to recognise differences between these environmental terms: climate change; greenhouse effect and global warming. international electronic journal of elementary education vol.1, issue 1, october 2008 ! 4! ! the findings from boyes and stanisstreet (1997), kirkwood and carr (1988), papadimitriou (2004) and trumper (1997) research are reinforced by those reported by cavanagh (2007). he acknowledged that, as public concern about global warming increases, teachers realise the significance of teaching about these environmental issues and they ‘are carving out a larger place for those issues in science classes, particularly at the high school level.’ however, he noted that the need to increase the teaching time devoted to these issues has not been supported by appropriate curriculum materials. cavanagh (2007) stated that teachers faced challenges in finding accurate and studentfriendly classroom resources for teaching about these environmental topics. cavanagh (2007) identified the keystone centre (www.keystonecurriculum.org) as a useful high school curriculum resource designed to solve the problem of locating up-to-date and relevant resources. while the keystone centre resources are appropriate for high school students, he did not identify relevant elementary teacher resources. this omission in elementary teaching resources has been addressed through the e-learning materials that have been developed as part of the research reported in this paper. in summary, the literature suggests that students frequently define energy as the ‘ability to do work’ but their understanding is superficial. additionally, many students hold inaccurate ideas about the nature of energy which are very persistent. when energy is linked with environmental concepts such as climate change, little research has been conducted into this area. research that has been conducted suggests that students hold views that are confused, often inaccurate, and persistent. coupled to these topical and important concepts is that fact that teachers find that many textbooks have very little upto-date and relevant information about energy and climate change in them. one of the goals of this paper is to identify what elementary aged students know about energy and related environmental concepts. energy and climate change among the many concepts that elementary students encounter as part of their education in science, energy is one of the more important. currently society is becoming more aware of the importance of energy, the supply of energy, its continuing longevity, and its impacts on the natural and built environment. one particularly topical issue is the relationship between energy and climate change and how it is affecting our lives. the concepts of energy, renewable and non-renewable sources of energy, and climate change are introduced in the elementary curriculum in new south wales. these concepts are progressively extended through the new south wales years 7-10 junior secondary science curriculum and into a number of the year 11/12 syllabuses in biology, earth and environmental science, and senior science (see www.boardofstudies.nsw.edu.au/syllabus_hsc/syllabus-a-z.html ) in the elementary (k6) curriculum, the science and technology syllabus introduces students to ideas about the environment, human impacts on the environment, human use of earth’s resources, and the energy needs of society. exploring elementary students’!…& boylan ! 5! ! context for this study the impetus for the research reported in this paper arose from a state-wide educational initiative in environmental education within the state of new south wales australia. this initiative focused on developing students’ knowledge and understanding about energy and related environmental concepts as a priority teaching area for 2007. regional environmental education policy during 2007, the riverina region of the new south wales department of education and training implemented the riverina regional environmental education plan for its 197 schools. the major focus in this policy was the development of a teaching program about energy that sought to promote students’ understanding of the relationship between energy and key environmental concepts such as climate change, and renewable and nonrenewable energy sources. the riverina environmental education centre within the riverina region is the riverina environmental education centre (reec) which is one of 24 environmental education centres located throughout new south wales. the principle role of an environmental education centre is to assist schools, teachers and their students with curriculum based environmental fieldwork and to help schools become more environmentally friendly. the task of devising and implementing the region wide program about energy was allocated to the staff at the riverina environmental education centre. at the riverina environmental education centre, this regional focus was translated into developing learning activities that were based on a theme of energy and climate change specifically for elementary students. the plan involved two major initiatives: 1. a collaboratively planned program between charles sturt university staff, third year elementary teacher education students and reec staff that led to a major regional energy learning event; and, 2. the development of the e-learning resources on energy and climate change that supported the new south wales science and technology k-6 syllabus. the first element of the regional energy plan revolved around a set of six learning activities developed by reec staff. participating elementary students rotated around these activities and the csu elementary teacher education students became group leaders for specific activities. as part of the planning for this day, the author was asked to develop a short survey that would identify what the elementary students’ level of knowledge and understanding about energy sources, and renewable and non-renewable sources of energy was. this survey and its analysis is the primary focus of the reminder of this paper. this second initiative involved the development of a student self-paced, e-learning guide. riverina teachers were encouraged to incorporate these e-learning resources and international electronic journal of elementary education vol.1, issue 1, october 2008 ! 6! ! activities into their teaching programs. these e-learning materials are accessible to all teachers and students by accessing the reec website (www.reec.nsw.edu.au). the energy and climate change resource is accessed from the k-6 section of the reec website as shown in figure 1 below. figure 1: available elementary e-learning activities in the next figure a sample web page from the e-learning materials within the energy and climate change learning resource is shown. figure 2: a sample page from the energy and climate change e-learning resource exploring elementary students’!…& boylan ! 7! ! methodology instrumentation the energy and climate change survey was designed using guidelines for establishing what prior understandings students hold based on the seminal work of osborne and freyberg (1985). the specific questions in the survey were developed following analysis of: a) the syllabus learning outcomes in the science and technology k-6 syllabus (board of studies, 2006); b) misconceptions about energy and the environment identified from the literature analysis; and, c) the major ideas presented in the on-line elearning resource on energy and climate change contained on the reec website (www.reec.nsw.edu.au). the instrument, see appendix 1, included 8 questions that were designed to determine what the students’ level of understanding about the following concepts was: energy, climate change, and renewable and non-renewable sources of energy. participants 132 elementary aged students who participated in the major regional energy program of learning in 2007 were surveyed. 44 students were in years 3/4 (identified as stage 2 within the new south wales context) and 87 students were in years 5/6 (identified as stage 3 within the new south wales context). 56% of the students were boys and 44% were girls. overall, the response rates to individual questions on the survey were high (minimum respondents per question = 90%). no students omitted question 1; 1 student omitted question 2; 2 students omitted question 3; 2 students omitted question 4; 3 students omitted question 5; and, 1 student omitted question 6. for question 7 which contained 8 different sub-parts, the number of students who omitted a response to any one energy source ranged from 7 (hydro electricity) to 12 (nuclear electricity). the average nonresponses rate for this question was 9 students per energy source. for the final question, question 8, 6 students omitted a response to this question. statistical analyses included generating descriptive statistics and conducting chi-square tests to determine if response pattern differences between groups were evident: namely year 3/4 (stage 2) vs year 5/6 (stage 3); and, gender (boys vs girls). results students’ responses to q1 to q6 are presented in table 1 below. descriptive frequencies and their associated percentages are reported. further the data were divided into: a) educational stage which is derived from how the science and technology k-6 syllabus is organised namely, stage 2 (year 3/4) and stage 3 (year 5/6); and, b) gender, boys and girls. questions 1 3 were designed to reveal what students knew about the types of energy while questions 4 – 6 asked to students about where energy comes from. international electronic journal of elementary education vol.1, issue 1, october 2008 ! 8! ! table 1: response about energy (q 1 – q 6) question response choices all students number (%) stage 2 number (%) stage 3 number (%) boys number (%) girls number (%) is light coming from the bulb a type of energy? yes 112 (91) 37 (90) 76 (94) 61 (92) 47 (92) no 11 (9) 4 (10) 5 (6) 5 (8) 4 (8) is the person using energy when she runs? yes 121 (92) 36 (88) a 79 (99) a 65 (98) b 45 (90) b no 11 (8) 5 (12) a 1 (1) a 1 (2) b 5 (10) b when the radio is turned on, is the sound coming from the speaker a form of energy? yes 89 (69) 19 (48) c 64 (80) c 43 (67) 35 (69) no 41 (31) 21 (52) c 16 (20) c 21 (33) 16 (31) as wood burns, what type of energy is produced? electrical 0 (0) 0 (0) 0 (0) 0 (0) 0 (0) heat 31 (24) 10 (24) 18 (23) 16 (25) 10 (20) light 2 (1) 1 (2) 0 (0) 0 (0) 1 (2) light and heat 97 (75) 30 (74) 61 (77) 48 (75) 40 (78) the energy in our bodies comes from: sleeping 44 (34) 15 (37) 23 (29) 25 (39) 8 (16) eating food 49 (38) 11 (27) 36 (46) 29 (45) 18 (36) drinking water 36 (28) 15 (37) 19 (24) 10 (16) 24 (48) on our planet earth, all life depends on the sun for its energy. yes 109 (83) 34 (83) 66 (83) 55 (83) 40 (78) no 22 (17) 7 (17) 14 (17) 10 (17) 11 (22) key a: significant difference detected: chi–square statistic = 6.891, df=1, p<0.01 b: significant difference detected: chi–square statistic = 4.176, df=1, p<0.05 c: significant difference detected: chi–square statistic = 13.207, df=1, p<0.001 discussion of table 1 results. types of energy (q1 – q3) overall the results suggest that elementary (stage 2 and 3) students do know quite a deal about the different types of energy. in question 1, overall about 9 in 10 students correctly responded to this question. with question 2, the majority of students (92%) successfully recognised that when a person is running, energy is being expended / consumed. when the secondary analyses of this question were conducted it revealed that: a) more year 5/6 (stage 3) students correctly understood what was happening than exploring elementary students’!…& boylan ! 9! ! year 3/4 (stage 2) students; and, b) boys more frequently than girls correctly recognised that the person used energy while running. the third question revealed that about 2 in 3 students correctly knew that sound emanating from a radio was a form of energy. the secondary analyses conducted in this question also identified that year 5/6 (stage 3) students were significantly better at recognising that sound was a form of energy than the younger year 3/4 (stage 2) students. this finding suggests that these younger elementary students hold prior understandings about sound energy that are different to the accepted scientific explanations. this finding can also be linked back to the syllabus as sound production and energy are concepts that are first introduced in the physical phenomenon and using technology strands of the science and technology (k-6) syllabus for student investigation sometime during their studies over years 3 and 4. origins of energy (q4 – q6) the next three questions probed what students know about where does energy come from. these questions drew upon the some of misconceptions identified in the research by kirkwood and carr (1988) and trumper (1997). question 4 required the student to appreciate that both light and heat energy are produced when wood burns. it was pleasing to report that no student selected electrical energy as the product of burning wood. in fact, 3 in 4 students (75%) recognised that both light energy and heat energy are produced. almost all of the remaining students (24%) stated that only heat energy was produced. the misconception that burning wood in a fire does not produce light energy is a topic that elementary teachers need to include when teaching about burning. secondary analyses of the data revealed that this misconception is very persistent and consistent across both stages and gender. question 5 asked what students knew about where does the energy used in our bodies originate. the responses to this question revealed a surprising and unexpected set of results which were consistent across stages and gender. about 1 in 3 students (34%) believed that sleeping was where the energy in our bodies originated. similarly almost 1 in 3 students (31%) believed that drinking water supplied the energy our bodies needed. just over one-third of the students (38%) correctly stated that it was through eating food that our bodies gained the energy they needed. these findings indicate that the majority of students hold alternate ideas about the origins of energy in our body. for elementary teachers, the body and food are common topics taught in elementary science classes, they need to make more explicit the links between food as a source of energy and how our body processes food to provide the necessary energy for normal cellular and bodily functioning. the final question (q6) revealed that the majority of students (83%) knew that the sun was the source of energy for all life on earth. this finding was consistent across stages 2 and 3 and by gender. international electronic journal of elementary education vol.1, issue 1, october 2008 ! 10! ! renewable and non renewable energy sources students’ knowledge and understanding about how they classified different sources of energy as either renewable or non-renewable were specifically explored by question 7 of the survey. in the following table, the responses of the elementary students are reported. table 2: renewable and non-renewable sources of energy question which of the following are renewable and non-renewable energies? response choices all students number (%) stage 2 number (%) stage 3 number (%) boys number (%) girls number (%) hydro electricity renewable 89 (71) 25 (71) 59 (74) 45 (70) 36 (78) nonrenewable 36 (29) 10 (29) 21 (26) 19 (30) 10 (22) coal renewable 35 (28) 10 (29) 24 (30) 20 (31) 11 (26) nonrenewable 88 (72) 24 (71) 55 (70) 45 ( 69) 32 (74) natural gas renewable 63 (52) 20 (59) 35 (44) 35 (55) 18 (41) nonrenewable 59 (48) 14 (41) 44 (56) 29 (46) 26 (59) nuclear electricity renewable 71 (59) 27 (77) d 40 (53) d 38 (59) 24 (57) nonrenewable 49 (41) 8 (23) d 36 (47) d 26 (41) 18 (43) food renewable 69 (56) 21 (62) 42 (53) 35 (54) 27 (63) nonrenewable 54 (44) 13 (36) 37 (47) 30 (46) 16 (37) solar electricity renewable 109 (89) 35 (97) 66 (86) 59 (89) 38 (88) nonrenewable 13 (11) 1 (3) 11 (14) 7 (11) 5 (12) oil / petrol renewable 41 (34) 14 (39) 25 (33) 23 (36) 15 (34) nonrenewable 81 (66) 22 (61) 52 (67) 41 (64) 29 (66) wind generated electricity renewable 95 (77) 24 (69) 62 (78) 51 (78) 32 (73) nonrenewable 29 (23) 11 (31) 17 (22) 14 (22) 12 (23) key d: significant difference detected: chi–square statistic = 6.017, df=1, p<0.05 exploring elementary students’!…& boylan ! 11! ! discussion of table 2 results overall many students’ ideas about which energy sources are renewable and which are non-renewable revealed considerable confusion. on average, 7 in 10 students answered correctly to hydroelectricity being a renewable source of energy. this response rate was consistent across stage 2 and stage 3. a very similar correct response rate was found for coal as a non-renewable energy with 7 in 10 students being successful. the responses provided for natural gas were more interesting with only 5 in 10 students understanding that natural gas is a non-renewable source of energy. the misconception about natural gas being a renewable source of energy may be in part due to the word ‘natural’. students may interpret ‘natural’ as implying it must be renewable. on average 6 out of 10 students believed nuclear electricity was a renewable source of energy. upon further investigation, a significant difference was found in how students in stage 2 understood this example compared with stage 3 students with a larger proportion of the latter group holding the correct understanding. there is a 24% increase from stage 2 to stage 3 in the number of students correctly identifying nuclear electricity as a non-renewable source of energy suggesting that this improvement may be linked to specific topics about energy contained only in the year 5/6 (stage 3) syllabus learning outcomes. however the elementary students’ response to this example still emphasise that 5 in 10 students are unaware that nuclear electricity is a non-renewable source of energy. the implication for teachers when teaching about types of energy sources and whether they are renewable or non-renewable is clear. a detailed consideration about how and where nuclear electricity comes from needs to be included in the teaching sequence. determining if food was a renewable or non-renewable source of energy was also a question that revealed students held misconceptions. between 5 and 6 in every 10 students correctly identified food as a renewable source of energy. this finding when linked with question 5’s responses indicated that holistically the level of understanding about food as an energy source and whether food is a renewable or non-renewable source does not align with the accepted scientific understandings. one possible explanation for the high level of misconception associated with this question could be that students do not link eating food with the food chains, energy flow and life cycles. further, students may think that food must be non-renewable because when you eat it, it is gone. solar electricity produced the highest successful response rate with 9 out of 10 students understanding that it is a renewable source of energy. the high response rate can be attributed to the high media focus on solar power making it a ‘hot topic’ at the moment. however, oil/petrol consumption is also a high priority media event, yet only about 2 in 3 students responded correctly that they understood that oil/petrol is a nonrenewable source of energy. the final example revealed that 8 in every 10 students recognised wind generated electricity as a renewable source of energy. this response was consistent across both stage and gender. international electronic journal of elementary education vol.1, issue 1, october 2008 ! 12! ! environmental terminology the final question (q8) was included in the survey as the literature suggested that these environmental terms were not well understood by students. in table 3, the responses from the elementary students are reported. this question explored whether students recognised the three selected terms as being the same or different. table 3: knowledge about environmental terms question. response choices all students number (%) stage 2 number (%) stage 3 number (%) boys number (%) girls number (%) climate change, greenhouse effect and global warming all mean different things yes 74 (59) 21 (53) 51 (66) 40 (63) 28 (57) no 52 (41) 17 (47) 26 (34) 24 (37) 21 (43) no significant differences found between year 3/4 and 5/6 (stages 2 & 3) and between boys and girls. discussion of table 3 results only 6 in 10 students could correctly identify that climate change, greenhouse effect and global warming all mean different things. there was a slight improvement in the correct response rates from year 3/4 (stage 2) to year 5/6 (stage 3), however this change was not statistically significant. this response was surprising given the high level of current media coverage on these issues world-wide and specifically the australian media coverage of the ongoing drought (now into its 6 th year) in inland australia which has reached crisis point. conclusion this paper set out to report upon an investigation designed to establish what elementary students knew about sources of energy, climate change, and renewable and nonrenewable energy sources while attending an energy learning event at the riverina environmental education centre. the findings revealed that students’ ideas about the types of energy are still developing with their understanding about sound energy revealing some persistent misconceptions. further, most students realised that the sun is the major energy source for all life on earth but many students held misconceptions about food as an energy source for humans. up to half of the elementary students held specific misunderstandings about renewable and non-renewable energy sources. finally, many students were not clear about how the key environmental concepts of climate change, greenhouse emissions and global warming are different from each other. for elementary teachers, this investigation suggests a strategy for assisting students to overtly consider their tacit ideas about energy and climate change and the need to explicitly teach these concepts within real world contexts. for these teachers, this study highlights the important pedagogical practice of establishing what ideas students already hold about a concept at the start of their teaching program and developing a responsive teaching program to promote deep understandings of the exploring elementary students’!…& boylan ! 13! ! concept. the energy and climate change survey developed and used in this study provides one easy to use strategy that teachers can implement to assist them in finding out what concepts and misconceptions students in their class hold. references board of studies (2006). science and technology k-6. outcomes and indicators. sydney: office of the board of studies. boyes, j. & stanisstreet, m. (1997). children’s models of understanding of two major global environmental issues (ozone layer and greenhouse effect). research in science and technological education, 15(1), 19-29. cavanagh, s. (2007). lessons about climate change pose many challenges for science teachers. educational week, 27:10, 1 – 16. retrieved on 1 st november 2007, from http://www.edweek.org/ew/articles/2007/10/31/10warming.h27.html?print=1 corney, g. (2000). student geography teachers’ pre-conceptions about teaching environmental topics. environmental education research, 6(4), 313-329. dawson, c. (1997). science teaching in the secondary schools. sydney: longmans. driver, r. (1985). research into childrens’ ideas book 2. london: routledge. the key stone centre (2007). csi climate status investigations. retrieved on 4 th november 2007, from http://www.keystonecurriculum.org kirwood, v. & carr, m. (1988). learning in science project (energy) final report. science education research unit, university of waikato – hamilton teachers’ college hamilton, nz. osborne, r. &freyberg, p. (1985). learning in science: the implications of children’s science. melbourne: heinemann. papadimitriou, v. (2004). prospective primary teachers’ understanding of climate change, greenhouse effects and ozone layer depletion. journal of science education and technology, 13(2), 299-307. trumper, r. (1997). the need for change in elementary school teaching. educational research, 39(2), 157–174. tytler, r. (2002). teaching for understanding in science: student conceptions research and changing views of learning. australian science teachers’ journal, 48(3), 14-21. international electronic journal of elementary education vol.1, issue 1, october 2008 ! 14! ! appendix 1 energy and climate change please circle: i am a boy/girl in year ______ q1. when the torch is switched on, the light bulb glows. is the light coming from the bulb a type of energy? yes no q7. which of the following are renewable (r) or non-renewable (nr) energies? a) hydro electricity r nr b) coal r nr c) natural gas r nr d) nuclear electricity r nr e) food r nr f) solar electricity r nr g) oil/petrol r nr h) wind generated electricity r nr q2. is this person using energy when she runs? yes no q3. when the radio is turned on, is the sound coming from the speaker a form of energy? yes no q4. as the wood burns, what type of energy is produced? a) electrical c) light b) heat d) light and heat q8. climate change, greenhouse effect and global warming all mean different things. yes no q5. the energy in our bodies comes from: a) sleeping b) eating food c) drinking water exploring elementary students’!…& boylan ! 15! ! q6. on our planet earth, all life depends on the sun for its energy. yes no international electronic journal of elementary education, 2015, 7(2), 253-262. issn:1307-9298 copyright © iejee www.iejee.com restorative justice: a changing community response thomas g. ryan  nipissing university, canada sean ruddy near north district school board, canada received: 1 december 2014 / revised: 7 february 2015 / accepted: 8 march 2015 abstract our purpose herein is to demonstrate how restorative justice continues to unfold globally and we explain how the use of a restorative justice ideology and intervention leads to a common alternative, not only in criminal justice institutions, but also within social agencies, such as elementary schools, and the related social support systems. we draw attention to this emerging trend via current research and resources that enable us to put forward a definition, theoretical background and list the characteristic traits of this alternative mode of life consequence. finally, we argue that the use of restorative justice in schools is a focus that is really a paradigm shift within the landscape of the educational enterprise. keywords: school discipline, student conduct, restorative justice introduction in several countries such as new zealand, australia, chile, canada, and brazil the use of restorative justice ideology and intervention has developed to become a common alternative not only in the criminal justice institutions, but also within social agencies such as elementary and secondary schools, and the social support systems (mccluskey, lloyd, stead, kane, riddell, & weedon, 2008). to draw attention to this emerging trend we define restorative justice, its theoretical background and list the characteristic traits of this changing alternative mode of life consequence. the use of restorative justice in schools is a focus that is presented as a paradigm shift within the landscape of the educational enterprise. historically within education there have been numerous theories and strategies applied in order to deal with student misbehaviour, classroom management and school climate (lockhart, & zammit, 2005). within canada, specifically ontario, we report that up until february 2008, we were legislated to enforce a “zero tolerance” policy in our schools (ontario  thomas g. ryan, faculty of education, nipissing university, ontario, canada. e-mail: thomasr@nipissingu.ca http://www.iejee.com/ international electronic journal of elementary education vol.7, issue 2, 253-262,2015 254 ministry of education, 2007), as there were mandatory suspensions and expulsions for certain infractions of the policy. in february 2008, the ontario provincial government amended the safe schools act and everything changed. no longer were we dealing in absolutes and fact; we were dealing in all shades of gray within context. suspensions and expulsions were no longer mandatory and administrators now had to consider a number of factors prior to suspending. included in these factors was whether or not progressive discipline has been used. progressive discipline is a whole-school approach that utilizes a continuum of interventions, supports, and consequences to address inappropriate student behaviour and to build upon strategies that promote positive behaviours described above. when inappropriate behaviour occurs, disciplinary measures should be applied within a framework that shifts the focus from one that is solely punitive to one that is both corrective and supportive. schools should utilize a range of interventions, supports, and consequences that include learning opportunities for reinforcing positive behaviour while helping students to make good choices. (memorandum, 2007) with this ontario provincial government shift in philosophy when it came to student discipline, school boards and administrators had to look for new ways to work with all students. one of the options that existed was the application of a restorative justice way of life within the framework of school discipline. restorative justice defined when a student commits a wrongdoing within a school community or within the greater community, how is it handled? does the punishment fit the crime? “the criminal justice system is concerned about holding offenders accountable, but that means making sure offenders get the punishment they deserve. very little in the process encourages offenders to understand the consequences of their actions or to empathize with victims” (zehr, 2002, p. 16). is there another way to deal with the wrongdoing? there are many different ways to define restorative justice. zehr (2002) defines restorative justice as “a process to involve, to the extent possible, those who have a stake in a specific offense and to collectively identify and address harms, needs, and obligations, in order to heal and put things as right as possible” (p. 36). lockhart and zammit (2005) suggest: it is a valued-based approach to responding to wrongdoing and conflict, with a balanced focus on the offender, victim, and community. restorative justice focuses on transforming wrongdoing by healing the harm, particularly to relationships, that is created by the harmful behavior. (p. 7) the guiding questions when using a restorative approach, either formally or informally include the following: 1. who has been hurt? 2. what are their needs? 3. whose obligations are they? 4. what are the causes? 5. who has a “stake” in this? 6. what is the appropriate process to involve stakeholders in an effort to put things right? (stutzman amstutz & mullet, 2005, p. 14) restorative justice: a changing community response / ryan & ruddy 255 the common theme between the two definitions is that restorative justice brings the person who created the harm together with the person or persons who were impacted by the harm. this is in stark contrast to what, historically, we would do in an ontario school setting through a suspension or an expulsion, we would remove the student that caused the harm to the school community and, at no time, would we ever give the victim a voice. by changing this focus, restorative justice brings the person who committed the harm closer to the community by allowing them to make reparation for the harm caused. when we look at things with reparation in mind, we gain a better appreciation for the problem that exists with the traditional punitive model. zehr (1995) reports clearly via lists within a pragmatic chart under the heading “understandings of crime”, it is also very relevant to an educational setting. he breaks it down into a comparison chart between a “retributive lens” and “restorative lens”. retributive lens crime defined by violation of rules (i.e., broken rules) harms defined abstractly interpersonal dimensions irrelevant conflictual nature of crime obscured wounds of offender peripheral offense defined in technical, legal terms restorative lens crime defined by harm to people and relationships (i.e., broken relationships) harms defined concretely crime seen as categorically different from other harms state as victim state and offender seen as primary parties victims’ needs and rights ignored crime recognized as related to other harms and conflicts people and relationships as victims victim and offender seen as primary parties victims’ needs and rights central interpersonal dimensions central conflictual nature of crime recognized wounds of offender important offense understood in full context: moral, social, economic, political. (p. 184-185) in order to apply this chart to an educational setting, one would only have to change the word “crime” to “incident”. when an incident occurs, the restorative model allows for all of the stakeholders in the school community, on both the side of the victim and that of the offender, to be treated equally and fairly, and for their feelings and opinions to be openly communicated. thus the relationships that are necessary for students, teaching staff, administrators, support staff, and parents to be able to work together in the future are then more easily repaired. the retributive model forces distance between the offender and the victim, and between them and the school community. the restorative model forces all parties to bridge the distance created during an incident and allow for healing to begin. theoretical background: restorative justice when reviewing current literature, the origins of this mode surface in a unique and surprising manner. most researchers may conclude that most of the principles of restorative justice come naturally to some people. for example, if a child of mine threw a rock and hit a neighbor’s window, how would i react? instinctively, i would march them over to the international electronic journal of elementary education vol.7, issue 2, 253-262,2015 256 neighbor’s house, have them apologize, explain the actions, and have them let the neighbor know how they intend to pay for the broken window. in addition, they would have to earn my trust back prior to letting them out of the house again. this seems to match closely with the definitions of restorative justice that have been previously stated. surprisingly, the formal concept of restorative justice came out of victim-offender reconciliation in new zealand. the key moment came in 1989 when the children, young persons and their families act came to pass and they began using a modified maori circle in juvenile court. what they found was that by using this alternative program many court trials were averted and settled. in addition, those that were most affected by the crime the victims were involved in the process. in 1991, the program was then brought to australia and a program was developed in the city of wagga wagga. by 1994, formal conferencing began to spread into other sectors of society, including education (lockhart & zammit, 2005, p. 49). canada in canada, there have been a number of different models of restorative justice. the aboriginal people have used healing circles within their community. in 1974, initiatives from the mennonite church in the kitchener-waterloo region started the first victim/offender mediation program. there have also been family group conferencing models, originating from the new zealand model, which started happening in 1981. one of the largest programs in canada, the manitoba mediation services, handles up to 400 mediations per year (lockhart & zammit, 2005, p. 49-50). although new zealand started to incorporate restorative justice into its school communities in 1994, canada has had a much slower start. lynn zammit and art lockhart created the first school board training for the toronto district school board, based on the new zealand family group conferencing model, in 1995 (lockhart & zammit, 2005, p. 50). there have been numerous workshops and professional development sessions put on by various school boards to introduce this concept to both administrators and staff. in june 2008, the near north district school board situated in northern ontario fully trained all administrators to become restorative justice facilitators. the shift in provincial legislation has helped to put this philosophy in the spotlight in the education field. the authors were fortunate enough to attend the international institute for restorative practices world conference on october 22nd – 24th, 2008 in toronto, ontario. we were amazed that i was able to network with educational colleagues from around the world and, in particular, around the province and see how they are shaping their own philosophies around the new legislation. community in all of our research concerning restorative justice, and the philosophy behind it, the central theme is always community. wachtel (1997) defined community succinctly: community is not a place. rather, it is a feeling, a perception. when people see themselves as belonging to a community, they feel connected. they have a sense of ownership and responsibility. they feel they have a say in how things are run and a stake in the outcome. (p. 193) when there is friction or conflict within that community, which is inevitable, a restorative approach tries to use that conflict as learning and a healing experience. this concept of community is not without critique. restorative justice approaches to community safety and non-social behaviours, as argued in the literature, rejuvenate the notion that the ‘community’ has a very real interest in what is restorative justice: a changing community response / ryan & ruddy 257 happening to its members. however there can be a pronounced naivety or romanticism about ‘community’ within much restorative justice literature. (verity & king, 2008, p. 474) this can be viewed as exclusion and a challenge to the process that is often common with right wing political groups. an example of this would be the ontario zero tolerance safe schools act which simply punished and did not educate which emanated from within the ontario progressive conservative political party of the 1990’s. central to the community ideology is variable of relationships. we now know that “strong institutions that develop genuine positive relationships within the nexus that sustains individual and collective life seem essential to our capacity to build a civil society” (morrison, blood & thorsborne, 2005, p. 336). using the restorative philosophy, built on relationships, contributes to the shift away from punitive and moves towards the supportive. “what have more recently emerged is the recognition that restorative practice also needs to be proactive, immersing the school community in a pedagogy that values relationships and a curriculum that values social and emotional learning” (morrison et al., 2005, p. 338). as an educational system, i often think that relationships take a back seat to the curriculum. biffis and lockhart (2008) add, it’s the students, staff, parents and friends who make each day worthwhile. imagine the smiles, the hugs, the camaraderie to achieve common goals; rising to challenges, overcoming obstacles and setting new directions. that’s the feel good stuff and it’s the feel good stuff that forms the very heart of all relationships. (p.19) we believe that if you put the relationship ahead of the curricula, the entire curriculum will positively balance within a healthy climate. global paradigm shift in education restorative justice has been implemented within many countries and is now embedded into many schools globally. the unfolding of this mode is very much dependent upon the leadership of the country, ministry of education and school board or district and ultimately, the administration of the particular school. in 2004, the australian research centre conducted a study of 18 primary and secondary australian schools as they implemented a restorative justice philosophy. for a number of schools, they used conferencing, classroom circles, or formal community circles. shaw (2007) explains, for some teachers and administrators, the use of restorative practices represented a fundamental shift in thinking about school justice and discipline. the application of restorative practices may threaten some teachers with a perceived loss of power and control, particularly within frameworks that involve compliance with school rules regulated by punishment regimes and conferred power of teachers. however, the experience of participants suggested that punishments based on a high control, low support paradigm are less effective in changing negative behavior. (p. 131) this is a major change in philosophy for some people. international electronic journal of elementary education vol.7, issue 2, 253-262,2015 258 the social discipline window: high control to punitive with restorative neglectful not permissive for (o’connell, wachtel & wachtel, 1999, p.78) the impact on the educational setting seems clear. you would want to work in the “with” quadrant, high in supporting the student as well as high in control. having said that, the will to be punitive or punish is still prevalent within our schools. “formal restorative justice does not take any account of the historical and embedded power relations between teacher and pupil, adult and child, school and home. it cannot address the very particular risks for both sides if this power balance is challenged” (mccluskey et al., 2008 p. 206-207). this becomes particularly problematic if you are trying to introduce a restorative justice approach in a school that has a history of punishment. moreover, restorative justice philosophy and the embedded interventions are founded on the belief that misconduct is a fracture in a relationship; it demonstrates a disconnection and should not merely be described as a contravention of guidelines, code, rules or laws of the land (morrison et al., 2005). we embrace the notion that the very purpose of restorative justice practice is to restore and mend the fracture within the strained or provisionally dysfunctional relationship which has been damaged by the misbehaviour or crime. all behaviour has a purpose. indeed, restorative practices comprise an assortment of recognized (informal) and casual (informal) interventions (mccluskey et al., 2008). our recognized interventions are often referred to as stakeholder conferences connecting the victim and offender with families/ guardians and other school/community support agencies. the requirement to oversee and mediate the process cannot be overemphasized since we were trained as restorative justice conference facilitators who watch and listen for both positive and problematic overt and covert interpersonal signals. as conference facilitators we act to support, facilitate and guide the course of action. we must remind ourselves that we are there to mend a relationship, to repair a fracture, and act ethically. trust, respect, integrity and due care are common traits noted within the process or act, as it ensures both high support low restorative justice: a changing community response / ryan & ruddy 259 psychological and physical safety of the participants. we aim to assist the stakeholders to appreciate and understand who, and to what extent, individuals have been both effected and affected. we act in this manner to develop a space for compassion, empathy and understanding. the result is almost always a resolution accord that mends, repairs, and deconstructs damage caused. school health, culture and climate often will have a direct impact on whether or not implementing a restorative justice approach will be successful. if the entire discipline structure has been historically based on an increasing level of punitive consequences that eventually lead to the student leaving the school community, it would be a huge shift for that particular school. when you walk into a school, you can pick up on the school climate and its culture via clues quite quickly. morrison et al. (2005) explained, these cultural cues include: how management speaks to, and about, staff; how staff speak to, and about, students and parents; the patterns of communication within staff meetings and what is said immediately after meeting; how criticism and disagreement are handled (p. 339). positive modeling and engaging teachers in the restorative process is one way to move the school climate along the restorative continuum. school culture change will not happen overnight. the frustrating thing is that we are all at the mercy of the next government regulation, legislation or curricular change. in ontario, if and when the government changes, we might be back playing with a new set of rules, policies and curricula that revisits traditional punitive measures. summary there is sufficient support from around the world to suggest that the use of restorative justice philosophies will continue to grow in our canadian educational system. having said that, it is certainly not the fix all answer to the way students act at school. moreover, “a restorative approach is a philosophy or framework that can guide us as we design programs and make decisions within our particular settings” (stutzman, amstutz & mullet, 2005, p. 4). the personal style of the administrator or teacher will always come into play in dealing with the relationships in the educational setting. we believe the new legislation makes it incumbent on school boards and administrators to work towards a supportive environment for our students. making that happen is very difficult. embedding some of the key restorative approaches, from the continuum, into the school policy will help move it along. for some schools, this may mean significant change. another threat to this movement is sustainability. will some teachers see this as a “new” initiative that will go away in a few years? “restorative justice is not a map, but the principles of restorative justice can be seen as a compass pointing a direction. we believe, at a minimum, restorative justice is an invitation for dialogue and exploration” (zehr, 2002, p. 10). as we mentioned earlier, we believe that many great teachers instinctively build relationships and interact with students in a restorative way. by their daily interactions, they inspire and motivate students to unexpected levels. for many students, that may be the only positive interaction they have with an adult in their life. as educators, we have a profound impact on students’ lives and that is reinforced in every interaction we have with them. at a recent conference, we heard an educational leader speak about community. she asked the group to think of the worst student in your school. then picture that student ten years from now and he or she is your next door neighbor. would you do anything to intervene with this student now? should that intervention be a punitive one or an intervention with a foundation of support? international electronic journal of elementary education vol.7, issue 2, 253-262,2015 260 • • • thomas g. ryan is professor of education at nipissing university in north bay, canada. sean ruddy is a secondary school principal within the near north district school board and is a graduate of nipissing universities master of education program in north bay, ontario, canada. references policy program memorandum [memorandum]. (2007, october 4). retrieved november 3, 2008, from ministry of education web site: http://www.edu.gov.on.ca/ extra/ eng/ ppm/ 145.html biffis, a., & lockhart, a. (2008). the human touch: the heartbeat of extraordinary education. author. lockhart, a., & zammit, l. (2005). restorative justice: transforming society. toronto: inclusion press. mccluskey, g., lloyd, g., stead, j., kane, j., riddell, s., & weedon, e. (2008, june). 'i was dead restorative today': from restorative justice to restorative approaches in school. cambridge journal of education, 38(2), 199-216. doi:10.1080/03057640802063262 morrison, b., blood, p., & thorsborne, m. (2005, december). practicing restorative justice in school communities: addressing the challenge of culture change. public organization review, 5(4), 335-357. doi:10.1007/s11115-005-5095-6 o'connell, t., wachtel, b., & wachtel, t. (1999). conferencing handbook: the new real justice training manual. pipersville, pa: the piper's press. shaw, g. (2007, fall). restorative practices in australian schools: changing relationships, changing culture. conflict resolution quarterly, 25(1), 127-135. from academic search premier database. stutzman-amstutz, l., & mullet, j. h. (2005). the little book of restorative discipline for schools: teaching responsibility; creating caring climates. the little books of justice & peacebuilding. intercourse, pa: good books. verity, f., & king, s. (2008, october). responding to intercommunal conflict what can restorative justice offer?. community development journal, 43(4), 470-470. retrieved from academic search premier database. wachtel, t. (1997). real justice: how we can revolutionize our response to wrongdoing. pipersville, pa: the piper's press. zehr, h. (1995). changing lenses: a new focus for crime and justice. scottdale, pa: herald press. zehr, h. (2002). the little book of restorative justice: a bestselling book by one of the founders of the movement. the little books of justice & peacebuilding. intercourse, pa: good books. restorative justice: a changing community response / ryan & ruddy 261 appendix restorative justice / practices definintion /historical background paradigm shift community native canadian australian / new zealand aboriginals relationships changing school culture education system international electronic journal of elementary education vol.7, issue 2, 253-262,2015 262 www.iejee.com this page is intentionally left blank http://www.iejee.com/ international electronic journal of elementary education, 2015, 7(2), 189-198. issn:1307-9298 copyright © iejee www.iejee.com teacher efficacy in an early childhood professional development school ann epstein  university of wisconsin la crosse, united states gary l. willhite university of wisconsin la crosse, united states received: 13 february 2015 / revised: 26 february 2015 / accepted: 19 march 2015 abstract teacher efficacy is the belief teachers have in their ability to impact student learning. efficacy includes teacher confidence in instructional, management and collaboration skills. the following study addresses teacher efficacy in an early childhood professional development school (pds). the pds experience provides an opportunity for mentor teachers to share their knowledge with teacher candidates over extended placements, typically more than 100 hours. preschool through fourth grade teachers participated in pre and post surveys and in a focus group discussion. analysis revealed strong efficacy across instructional and management aspects of teaching but relatively weaker teacher confidence in assisting families to support their children’s success. findings also suggest that early childhood teachers in this pds setting believe it is their responsibility to nurture strong self-efficacy among their students. keywords: teacher self-efficacy, early childhood teacher efficacy, professional development school mentor teachers introduction teacher educator programs strive to engage pre-service teachers in highly effective learning experiences. teachers, the learning environment, and individual learners comprise three prominent components of this endeavor. the following pilot study provides insight regarding the intersection of these components. specifically, teacher beliefs in their ability to impact student learning (self-efficacy) is studied in the context of a particular type of learning environment (professional development schools). the study addressed efficacy among mentor teachers during the first year of implementation of an early childhood professional development school program.  ann epstein, ph.d., department of educational studies, 1725 state street, la crosse, wi, 54601. 608-785-8145. aepstein@uwlax.edu. http://www.iejee.com/ international electronic journal of elementary education vol.7, issue 2, 189-198,2015 190 albert bandura noted that, “…an efficacy expectation is the conviction that one can successfully execute the behavior required to produce the outcomes.” (1997, p. 193). selfefficacy is a person’s belief in his or her ability to succeed in a particular situation. bandura described these beliefs as determinants of how people think, behave, and feel (1994). the purpose of this study was to clarify components of mentor teachers’ self-efficacy in an early childhood professional development school (pds) setting. we asked, “how does being a mentor teacher in an early childhood pds affect teachers’ self efficacy?” literature review: pds impact on teacher candidate performance and mentor teacher development professional development schools were launched with the intention of providing both mentors and teacher candidates opportunities to develop effective teaching strategies and a deep understanding of why these strategies work (teitel, 2003; harris & van tassell, 2005; lee & hemer-patnode, 2010). building on the medical model of teaching hospitals, a group of university education deans founded professional development schools upon the premise that sound learning requires continuous reflection by both experienced teachers (mentors) and novice teachers (candidates) within a collaborative, respectful community (holmes, 1990). research clearly identifies positive outcomes for pds teacher candidates (castle, fox, & souder, 2006; darling-hammond, 2007; sandoval-lucero et al., 2011). for example, a study comparing pds and traditional campus-based field experiences revealed slightly higher performance by pds students during their field experience (ridley, 2005). performance was evaluated through blind scoring of lesson planning, lesson reflections, overall teacher effectiveness and content retention. the stronger performance of pds teacher candidates was not only maintained but grew stronger over time; during their first year of teaching the pds teacher candidates performed significantly better across the above dimensions than their peers who completed traditional field experiences. fulfilling the original pds intent, professional skills of mentor teachers are also strengthened. in a study addressing pre-service teacher preparation, in-service teachers’ professionalism, and children’s achievement (cobb, 2000), 85% of mentor teachers reported learning innovative teaching strategies. mentors also report a stronger understanding of culturally responsive teaching (mccormick, eick, & womack, 2013) as well as enhanced communication and collaboration skills (beaty-o’ferrall & johnson, 2010). a comprehensive analysis of “simultaneous renewal” of pds partners (shroyer et al., 2007) emphasized the importance of always placing student progress at the center of pds initiatives. a joint commitment to student achievement, along with sufficient resources and time, can result in renewed energy and professional satisfaction for mentor teachers and university personnel. literature review: teacher self-efficacy teachers’ beliefs in their ability to perform well have been researched in a variety of settings. one study of 1,430 teachers in traditional school settings revealed teacher confidence in their ability to implement effective instructional and classroom strategies leading to positive student engagement (klassen & chiu, 2010). however, teacher efficacy has not been widely addressed in professional development schools. particularly relevant to this pilot study, teachers in younger grades have been found to have stronger self-efficacy (tschannen-moran & woolfolk hoy, 2001). a study of early childhood teacher self-efficacy suggests that staff collaboration, student engagement, and consistent opportunities to participate in decision-making contribute to self-efficacy (guo et teacher efficacy in an early childhood professional development school / epstein & willhite 191 al., 2011). however, in a study of 48 early childhood teachers in 38 centers including head start and state-funded prek, student engagement and teacher experience did not appear to contribute to teachers’ self-efficacy (mcginty et al, 2008). teacher collaboration and being able to impact decision-making did correlate significantly with self-efficacy. collaboration and joint decision-making are required in pds settings. while time frames vary, mentor teachers typically guide pre-service teacher candidates several days each week over the course of an entire semester. they discuss instructional strategies, assessment procedures, individual learning styles, classroom guidance procedures and lesson planning. joint decision-making occurs regularly as mentors work closely with higher education liaisons to address day-to-day procedures and expectations as well as longterm goals. the pds setting allows for a more intensive field experience for teacher candidates. the additional time and intensity requires more mentoring and sharing of one’s own practice. this study sought to clarify how additional sharing affected mentor teachers’ views of their ability to impact children’s learning. setting and participants the pilot study was carried out during the first year of implementation of an early childhood professional development school collaboration in a mid-sized midwestern community. several successful pds programs were underway at the elementary, middle and high school levels. upon confirming interest among early childhood faculty, an invitation was offered to a school serving children from preschool through fifth grade. the school’s final enrollment for 2012-2013 was 395 children; 166 children received free lunch and 25 reduced lunch. the school served children who were learning english as a second language and included children with a range of exceptionalities. fourteen early childhood teacher candidates applied to the pds program. acceptance into the program required at least a 3.0 grade point average and completion of foundational courses regarding pedagogy, literacy and public education in the united states. pds teacher candidates were seeking certification to teach children from birth through age 11. they participated in classroom life every monday, wednesday and friday morning for fourteen weeks, logging a minimum of 100 hours. one teacher candidate was placed with each of 14 mentor teachers who volunteered for this inaugural year. the fourteen mentor teacher participants taught the following grade levels: preschool (3), kindergarten (3), first grade (3), second grade (2), third grade (1), and fourth grade (2). their teacher candidates had two additional field experiences before reaching their student teaching semester. therefore, mentor teachers were guiding candidates who were at the beginning of their pre-service field experiences. methodology and analysis a pre and post survey was used to study mentor teachers’ self-efficacy. the short form of the teachers’ sense of efficacy scale is comprised of twelve likert scale questions to assess teachers’ views of their ability to address typical aspects of teaching (tschannen-moran & woolfolk hoy, 2001). three open-ended questions were added to further study teachers’ views of their teaching skills. mentor teachers also participated in a focus group discussion that further addressed how the pds experience affected their views of the following professional areas: individual international electronic journal of elementary education vol.7, issue 2, 189-198,2015 192 relationships with students, classroom management, understanding of grade or age-level content, teaching strategies, assessment, and creativity. descriptive statistical analysis was employed to analyze likert item survey responses. constant comparative analysis (miles & huberman, 1994; guba & lincoln, 1981) was used to study mentor teacher comments offered during the focus group discussion, as well as their open-ended survey responses. the authors independently coded and categorized focus group comments and survey responses. to minimize subjectivity, they also identified themes independently. findings eight (of fourteen) mentor teachers completed the pre survey, three completed the post survey and five participated in the focus group discussion. response rates may have been affected by the newness of the program (pre survey) and end of the semester activities (post survey and focus group discussion). participants stated confidence in eleven of the twelve teaching skills (table 1). teachers’ responses to open-ended questions echoed their overall confidence as well as their frustrations in several aspects of teaching. the following teacher statement captures the positive influence teachers believe they have in assisting learners. i believe that each child can learn. i believe that each child has talents and a purpose in this world. if i can make a connection with each child, and build a classroom community i can get each child to believe they can be successful in school. when asked to provide additional comments regarding what aspects of their profession can sometimes make teaching difficult, mentors described the current culture of high stakes testing, a state-level initiative aimed at assessing teacher effectiveness, assisting children who have disruptive behaviors and collaborating effectively with families. one participant summarized several of these challenges. incorporating time into your day for assessment and reflection can be difficult. managing individual student behaviors that fluctuate from day to day requires you to be very flexible about your schedule and your plans. only three teachers provided post survey responses. they rated their level of influence at 9 (a great deal) for nine of the twelve questions. mentor teacher ratings for question 11 regarding their ability to influence families were similar to pre survey responses (table 2). five teachers participated in the focus group conversation. they were asked to share how being a pds mentor had affected their skills in the following areas: individual relationships with students, classroom management, understanding of grade or age-level content, teaching strategies, assessment, and creativity. analyses of their comments revealed the following themes. mentors valued the opportunity to share their knowledge with teacher candidates, and saw this as a way to validate their teaching strengths. teacher efficacy in an early childhood professional development school / epstein & willhite 193 table 1. summary of pre survey responses (all except question 11) 7 (quite a bit) 8 9 (a great deal) 1. how much can you do to control disruptive behavior in the classroom? 12% (1) 25% (2) 50% (4) 2. how much can you do to motivate students who show low interest in school work? 25% (2) 25% (2) 50% (4) 3. how much can you do to get students to believe they can do well in school work? 25% (2) 25% (2) 50% (4) 4. how much can you do to help your students value learning? 25% (2) 25% (2) 50% (4) 5. to what extent can you craft good questions for your students? 25% (2) 37% (3) 37% (3) 6. how much can you do to get children to follow classroom rules? 37% (3) 10% (1) 50% (4) 7. how much can you do to calm a student who is disruptive or noisy? 37% (3) 10% (1) 50% (4) 8. how well can you establish a classroom management system with each group of students? 25% (2) 25% (2) 50% (4) 9. how much can you use a variety of assessment strategies? 50% (4) 50% (4) 10. to what extent can you provide an alternative explanation or example when students are confused? 25% (2) 25% (2) 50% (4) 12. how well can you implement alternative strategies in your classroom? 25% (2) 25% (2) 50% (4) source: teachers’ sense of efficacy scale (tschannen-moran & woolfolk hoy, 2001) http://people.ehe.osu.edu/ahoy/research/instruments/#sense they rated their ability to assist families in helping children do well in school somewhat lower (table 2). table 2. summary of pre and post survey responses to question 11 regarding families 5 (some influence) 6 7 (quite a bit) 8 9 (a great deal) 11. how much can you assist families in helping their children do well in school? pre-survey 10% (1) 25% (2) 10% (1) 10% (1) 37% (3) post-survey (33% (1) 33% (1) 0 33% (1) source: teachers’ sense of efficacy scale (tschannen-moran & woolfolk hoy, 2001) http://people.ehe.osu.edu/ahoy/research/instruments/#sense international electronic journal of elementary education vol.7, issue 2, 189-198,2015 194 effective teaching is rooted in respectful relationships with individual learners. seeing pds students develop these relationships was particularly satisfying. mentor teachers revealed varying levels of comfort and skill, indicating that while most were comfortable with one teacher candidate, one mentor preferred providing guidance for both a pds student and a student teacher. enhanced reflection skills emerged as the strongest theme. a first grade teacher was among the focus group participants. she described how being a mentor strengthened her ability to reflect. i just found it great to actually analyze my teaching because some things come so naturally now that you don’t even realize that you’re doing it. so (with a pds student) you actually have to step back and see what process you take to get there and then whether that’s the right process or should i modify that process. then you will be able to explain it a little better to the pds student so that they can take their (own) steps to understanding it better. several focus group participants commented on the relationships teacher candidates developed with students. they described the sometimes uncomfortable process of moving from being a friendly helper to being a teacher with clear behavioral expectations. at the conclusion of the focus group discussion, a second grade teacher described the strong bonds her pds teacher candidate developed with students in the class. this resulted in difficulty saying good-bye. we had to go to “hug friday” because she (pds student) would leave monday and wednesday and it would take forever. they would all want to hug her goodbye. “we’ll hug her on friday, and we’ll high five her on the way out (on mondays and wednesdays).” it was just so… “don’t leave!” discussion participants in this pilot study viewed themselves as highly effective academic instructors. they also rated their management skills as strong. in response to open-ended survey questions, several remarked that working with individual students (particularly those with difficult behaviors) was both challenging and rewarding. collaborating with families was rated as the most challenging aspect of teaching. these findings were consistent across pre and post survey responses. while this was a short time span of three months, it provides some evidence of stability. the teaching range of the five participants ranged from preschool through second grade. all had been teaching for at least five years and two for over fifteen years. one of the more experienced participants discussed the challenge of balancing her guidance for a student teacher and for a pds student. she commented on the unique opportunities available with having both a student teacher and a pds student in her classroom. the camaraderie and the collaboration between the student teacher and the pds student is also an interesting one because the student teacher is much closer to where the pds student is in life and in their job and career and everything than i am. so they had good conversations. they talked about lessons and so that was really a plus for both of them and the student teacher feels like a mentor like i would feel toward the student teacher so that gives them (an additional) purpose. providing effective support for mentor teachers who prefer having both a student teacher and a pds student suggests an area for further study. teacher efficacy in an early childhood professional development school / epstein & willhite 195 the five focus group participants did not address partnering with families. however, it was highlighted by one of the eight mentors who completed the pre survey. her response to “please provide any additional comments regarding the type of things that sometimes make teaching difficult” follows. it is most difficult when one student is so disruptive. teaching sometimes cannot continue until outside assistance removes the student. also when a parent does not understand what happens in the classroom, misunderstands a situation, and is upset about it (teaching is difficult). combining pre and post responses, thirty-six percent of mentors rated themselves as having a moderate degree of influence in assisting families with helping their children do well in school (table 2). even though sixty-three percent gave themselves a score in the quite-a-bit to great-deal ranges of influence, working effectively with families emerged as the lowest area of self-efficacy. focus group participants did not bring up this topic; mentor teachers did note working with families as a challenge in response to an open-ended survey question. the national association for the education of young children holds effective collaboration with families as a critical area of effective teaching (copple & bredekamp, 2009). this finding suggests a possible area for mentor teacher professional development. limitations and recommendations limitations of this pilot study include a low post survey response rate. despite several email and in-person reminders, only three of the twelve mentor teachers completed the post survey. therefore, a comparison of pre and post responses is not possible. the wide grade range can be considered both a limitation and a strength. it is difficult to assess the selfefficacy of teachers from preschool through fourth grade with one instrument. while the basic components of effective teaching are similar, how these are carried out is difficult to capture through one set of prompts. however, providing opportunities for mentors across a wide grade range to participate in one pds setting can facilitate a rich sharing of effective practices and professional insights. the wide range of grades in this early childhood pds setting may provide a specific opportunity to address the one area of relative weakness regarding teacher efficacy: assisting families. all mentors in this study are fully certified; some are certified to teach children from birth through age eleven and others are certified for first grade through middle school. it is not known if responses to the family question (table 2) differ by certification range. early childhood teachers receive more training in the area of working with families. the collaborative nature of pds settings could provide an atmosphere for sharing expertise regarding effective family partnership strategies. a central tenant of self-efficacy is one’s belief in his or her ability to succeed. a corollary is that teachers who have strong self-efficacy pass this on to their students. a prek teacher captured this opportunity in her pre survey response. she stated the following when asked to explain why she viewed herself as being able to provide a great deal of influence in getting students to believe they can do well in school. my job as an early childhood teacher is to help my students believe in themselves and their abilities as a student. it is important that i send them off to kindergarten believing they can and will learn. i do this by frequently sharing with them all that they have learned and reminding them how smart they are. international electronic journal of elementary education vol.7, issue 2, 189-198,2015 196 findings of this pilot study suggest that not only do professional development school mentors have strong self-efficacy, preschool teachers pass this onto their own students. additional research could clarify if this holds true for mentors across elementary grade levels. • • • ann epstein, received her ph.d. in early childhood special education from the university of maryland in 1996. she is the early childhood program coordinator and the professional development school liaison for early childhood education at the university of wisconsin la crosse. her r esearch interests include accommodations for young children who have special needs, professional development school impacts on teacher candidate learning, and montessori education. gary l. willhite, received his ph.d. in curriculum and instruction with an emphasis in secondary english, literacy and supervision, from kansas state university in 1992. he has been the professional development school coordinator at the university of wisconsin la crosse since 2010. his research interests include self-efficacy, clinical experience, and professional development school initiatives. references bandura, a. (1997). self-efficacy: toward a unifying theory of behavioral change. psychological review 84(2), 191 – 215. bandura, a. (1994). self-efficacy. in v. s. ramachaudran (ed.), encyclopedia of human behavior (vol. 4, pp. 71-81). new york: academic press. (reprinted in h. friedman [ed.], encyclopedia of mental health. san diego: academic press, 1998). beaty-o’ferrall, m.e. & johnson, f.w. (2010). using supportive team building to promote improved instruction, student achievement, and collaboration in an urban professional development school. school university partnerships, 4(1), 56 – 64. castle, s., fox, r., & souder, k. (2006). do professional development schools (pdss) make a difference? a comparative study of pds and non-pds teacher candidates. journal of teacher education, 58(1), 65 – 80. cobb, j. b. (2000) the impact of a professional development school on pre-service teacher preparation, in-service teachers’ professionalism, and children’s achievement, action in teacher education, 22(3), 64–76.
 copple, c. & bredekamp, s., eds. (2009). developmentally appropriate practice in early childhood programs: serving children from birth through age 8. washington dc: naeyc. darling-hammond, l. (2007). the story of gloria is a future vision of the new teacher. journal of staff development, 28(3), 25 – 26. guba, e. g. & lincoln, y. s. (1981). effective evaluation: improving the usefulness of evaluation results through responsive and naturalistic approaches. san francisco: jossey-bass. guo, y., justice, l., sawyer, b. & tompkins, v. (2011). exploring factors related to preschool teachers’ self-efficacy. teaching and teacher education, 27, 961 – 968. harris, m. m. & van tassell, f. (2005). the professional development school as learning organization, european journal of teacher education, (28)2, 179–194. holmes group. (1990). tomorrow’s schools: principles for design of professional development schools. east lansing, mi: author. teacher efficacy in an early childhood professional development school / epstein & willhite 197 klassen, r. & chiu, m. (2010). effects on teacher self-efficacy and job satisfaction; teacher gender, years of experience, and job stress. journal of educational psychology, 102(3), 741 – 756. lee, y. a. & hemer-patnode, l. (2010). developing teacher candidates' knowledge, skills, and dispositions to teach diverse students. journal of instructional psychology, (37)3, 222 – 235. mccormick, t. m, eick, c.j., & womack, j. s. (2013). culturally responsive teaching: awareness and professional growth through a school-university collaboration. school university partnerships, 6(1), 6 – 14. mcginty, a. s., justice, l. m., & . rimm-kaufman, s. e. (2008). sense of school community for preschool teachers serving at-risk children. early education & development, 19, 361-384. miles, m. b. & huberman, a. m. (1994). qualitative data analysis: an expanded sourcebook. thousand oaks, ca: sage. sandoval-lucero, e., shanklin, n.l, sobel, d.m., townsend, s. s., davis, a. & kalisher, s. (2011). voices of beginning teachers: do paths to preparation make a difference? education, 132(2), 336 – 350. shroyer, g., yahnke, s. bennett, a. & dunn, c. (2007). simultaneous renewal through professional development school partnerships. the journal of educational research, 100(4), 211 – 224. teitel, l. (2003). the professional development school handbook: starting, sustaining, and assessing partnerships that improve student learning. thousand oaks, ca: corwin press. tschannen-moran, m., & woolfolk hoy, a. (2001). teacher efficacy: capturing and elusive construct. teaching and teacher education, 17, 783-805. international electronic journal of elementary education vol.7, issue 2, 189-198,2015 198 www.iejee.com this page is intentionally left blank http://www.iejee.com/ international electronic journal of elementary education, 9(3), 709-724, march, 2017 issn:1307-9298 copyright © iejee www.iejee.com the wicked problem of the intersection between supervision and evaluation ian m. mette a * bret g. range b jason anderson b david j. hvidston c lisa nieuwenhuizen d jon doty a a university of maine, usa b springfield public schools, usa c university of wyoming, usa d william woods university, usa received: 11 january 2017 / revised: 20 february 2017 / accepted: 28 february 2017 abstract the purpose of this research was to explore how principals in eight high-functioning elementary schools in one american school district balanced teacher supervision and evaluation in their role as an instructional leader. using the theoretical framework of ‘wicked problems’, to unpack the circular used to problematize teacher supervision and evaluation, the findings analyse how elementary principals in these eight buildings acknowledge the tensions and conflicts between supervision and evaluation, specifically as they relate to improving teacher instruction. specifically, the results of this study highlight not only the differences between supervision and evaluation, but also the intersection between the two functions, as well as how high-performing elementary school principals serve as an instructional coach rather than a manager of teachers. while the two functions of supervision and evaluation are inherently different, it is the acknowledgement of the intersection between the two functions that can allow building principals to progress as instructional coaches who can better develop human resources and create higher-functioning school systems. overall, this study points toward the importance of elementary principals having the instructional leadership skills to differentiate supervision and professional development need for teachers, which in turn influences the evaluation of a teacher is in her/his respective career. keywords: teacher supervision, teacher evaluation, instructional supervision, instructional leadership introduction across america teacher evaluation and supervision reform is underway, prompted in part by pressure from the federal no child left behind (nclb) flexibility waivers, race to the top, and every student succeeds act (essa). much has been written in the theoretical literature about what educators and scholars think makes for effective supervision (glickman, gordon, & ross-gordon, 2014; zepeda, 2012), evaluation (darling-hammond, * corresponding author: ian m. mette, university of maine 334 merrill hall orono, me 04401 united states of america. phone: 207-951-5659. e-mail: ian.mette@maine.edu http://www.iejee.com/ international electronic journal of elementary education, 9(3), 709-724, march, 2017 710 2013; platt, tripp, ogden, & fraser, 2000), and the merging of these two functions (marshall, 2013; marzano & toth, 2013), specifically if this is in the best interests of helping teachers grow as instructors and thus impact student achievement. within the literature, and to a greater extent with practitioners and laypeople, the terms evaluation and supervision are frequently and incorrectly used as synonyms. as early as the 1920s, the roles and interactions of personnel in schools gradually became more formalized as public scrutiny of education increased, causing supervision (supportive feedback to improve instruction) to become intertwined with evaluation (assessment of ability), to the point that now “supervision is usually understood as teacher evaluation in the schools” (hazi & ricinski, 2009, p. 2). in a study of 100 teachers and their principals, ponticell and zepeda (2004) found that “for all teachers and for the vast majority of principals, supervision was, quite simply, evaluation” (p. 47). with regards to teacher supervision, the meaning and definition is typically intended to target professional growth (i.e. formative feedback), which is often complex and usually situational, relying on a combination of knowledge, technical skill, and interpersonal skill (glickman et al., 2014). alternatively, with regards to teacher evaluation, the meaning and definition is primarily an administrative function (i.e., summative evaluation) to judge value, to determine whether the teacher meets minimum standards, and in some situations to determine positive or negative employment actions (zepeda, 2012). teacher evaluation, as detailed in the review of the literature, is a human resource function. thus, the purposes of evaluation and supervision are distinct; supervision is ongoing support for professional growth purposes while evaluation is primarily an assessment of performance. as a result, clear definition and review of the literature regarding teacher evaluation and supervision for teacher growth provide important insight to better understand school reform and school improvement efforts that are intended to improve student achievement, particularly as is relates to the differences in the definition and intention of teacher supervision and teacher evaluation. teacher supervision teacher supervision (i.e. formative feedback) is focused on ongoing support, teacher improvement, and teacher professional growth. supervision may include a wide variety of formal and informal efforts, including the clinical supervision model of conferencing before and after an observed lesson, staff development activities (large group, small group, or individual), portfolio development, goal-setting, data analysis, and reflection (ponticell & zepeda, 2004; eady & zepeda, 2007; hazi & ricinski, 2009). similarly broad, researchers note that supervision may be provided by a wide variety of people; including administrators, peer teachers, instructional coaches, and independent consultants (alila, määttä, & uusiautti, 2015; beach & reinhartz, 1989; oliva & pawlas, 2004; wiles & bondi, 2004). the ultimate purpose of supervision is to indirectly help student achievement by helping a teacher improve instruction, which require different instructional goals and strategies based on the strengths of individual teachers (glickman et al., 2014). thus, there is an inherent aspect of supervision that is situational in nature, namely the need to identify appropriate support that is dependent on a variety of factors, including instructional goals, strengths and needs of a teacher, the career stage of the teacher, and overarching organizational goals (beach & reinhartz, 1989; wiles & bondi, 2004; glickman et al., 2014). tension is noted between the desired collaborative, trusting relationship and conflicting functions when the supervisor is also an administrator (with responsibilities such as summative evaluation, resource allocation, and employment decisions). oliva and pawlas (2004) summarize this ongoing discussion in the literature, while recognizing that in small schools and small school districts resources and staff sizes are such that administrators supervision and evaluation / mette, range, anderson, hvıdston, nıeuwenhuızen & doty 711 must also be supervisors. wiles and bondi (2004) also note that collective bargaining has pushed the field of supervision into a management posture. thus, time limitations and tensions for those in dual administration/supervisory roles are major challenges facing the field of teacher supervision (ryan & gottfried, 2012). additionally, reduced flexibility poses challenges for school systems as states increase the number of mandates for testing, curriculum, and teaching, which create additional “intrarole conflicts” for supervisors tasked with reforms with which they potentially disagree (oliva & pawlas, 2004, p. 13). finally, supervisors face a challenge of moving from helper to judge, as they are frequently tasked with playing a role in teacher evaluation (beach & reinhartz, 1989). teacher evaluation teacher evaluation (i.e., summative evaluation) focuses on the organizational need for accountability, determining and documenting the level of a teacher’s performance over a specific time period (hazi & rucinski, 2009; platt et al., 2000). as posited by oliva and pawlas (2004), “summative evaluation…is assessment of teacher performance by an administrator for the purpose of making decisions about such matters as tenure, retention, career ladder, and merit pay” (p. 367). thus, teacher evaluation is typically formal and uses standardized criteria, forms, and procedures defined in policy to ensure accurate measurement. typically teacher evaluation cycles are time-bound, with summative evaluations usually occurring annually for probationary teachers, and typically once every one to three years for teachers who are tenured or on a continuing contract. at the conclusion of the cycle an evaluation cycle, if the summative evaluation determination is satisfactory, a new cycle typically begins, while an unsatisfactory summative evaluation may lead to development of an improvement plan or employment action such as dismissal. however, teacher evaluation produces tension between system rigidity and protection from arbitrary action, influenced by a desire for fairness (desander, 2000) and legal defensibility (glickman et al., 2014). koski (2012) describes this tension as: from the teacher’s or union’s perspective, such rigid and predictable systems minimize arbitrary administrative evaluations that may be based on personality conflicts, inappropriate factors, or sloppy observations. from the administration’s perspective, these rigid systems do not sufficiently account for a teacher’s impact on student performance, are time consuming due to paperwork, and do not give the principal sufficient flexibility to provide meaningful feedback and, ultimately, make decisions about the teacher’s future employment (2012, p. 86). similarly, darling-hammond (2013) and glickman et al. (2014) note that rigid systems may preclude tailoring helpful feedback to teacher needs. additionally, due to the fear of litigation, evaluators may limit potentially helpful or challenging feedback (desander, 2000) and develop a “norm of non-interference” with teachers (goldstein, 2007, p. 481). altogether, the evaluation process can result in negative side-effects in schools. evaluation is often perceived to be unhelpful in improvement and primarily for punitive purposes (darling-hammond, 2013; weisburg, sexton, mulhern, & keeling, 2009). the process can inadvertently discourage improvement through negative feelings, reduced participation, decreased willingness to alter behavior, and erosion of trust (glickman et al., 2010; marshall, 2013). teachers who perceive themselves as performing well are maddened when colleagues they perceive as poor performers receive evaluation ratings similar to their own (weisberg et al., 2009), or conversely, attempt to protect peers when dismissal of an ineffective teacher is sought (marshall, 2013; platt et al. 2000). theoretical framework: wicked problems education is a socially constructed institution, allowing it to be continually problematized based on various demands of society, including political, economic, and environmental international electronic journal of elementary education, 9(3), 709-724, march, 2017 712 pressures (southgate, reynolds, & howley, 2013). based on the notion of ‘wicked problems’, namely that they have no definitive formulation or solution, no correct or wrong answers, and no finite amount of solutions (margolis & buchanan, 1995), education faces a problem within the realm of instructional supervision and evaluation of teachers. rittel and weber (1973) conceptualized wicked problems as issues that employ circular logic to define social problems, which in turn hampers the ability for a group within a society (i.e. educators) to define and identify a problem, and consequently prevents change to produce more equitable practices. within instructional supervision and evaluation, the wicked problem that exists are the conflicts and tensions that arise between the two functions. supervision is needed to coach teachers and help them reflect on personal instructional goals, however it is a time-consuming process that also needs to be able to show results if teachers are not meeting the needs of students. evaluation is needed to produce high-quality school systems and retain quality teachers, however it is often a litigious process that can negatively impacts the climate and culture of school buildings and districts. thus, using the theoretical framework of wicked problems to unpack the discrepancies of supervision and evaluation, particularly how both supervision and evaluation can be used to support successful elementary school leadership, is necessary to better understanding the human process in creating better school systems. method this research is the final analysis of part of a large data set (mette, range, anderson, hvidston, & nieuwenhuizen, 2015; range, anderson, hvidston, & mette, 2013), in which the purpose of the study was intended to identify common supervision and evaluation traits shared by successful elementary principals in one urban district in a midwest state in america. for this study, successful principals were defined as principals that lead schools with high student achievement on state standardized tests for third and fourth grade math and communication arts. the district selected is one of the largest in the state with a district enrollment of almost 22,000 students, and the eight elementary schools were selected due to their previously mentioned high student achievement on the state standardized test. all eight elementary schools selected in this analysis are in the top 10% of student achievement in the state studied. specifically, the researchers of the study were interested in understanding 1) how teacher supervision and evaluation were conceptualized ideologically as well as implemented in action to drive continuous improvement of teachers, and; 2) how principals provided differentiated support for individual teachers based on experience and ability. thus, this study sought to inform the practice of teacher supervision and evaluation for elementary principals by examining and understanding the personal experiences of administrators working in high achievement elementary schools in the largest school district of an american midwestern state. as part of the qualitative analyses used in this study, the administrators of the eight elementary schools were interviewed to examine their perceptions of instructional supervision and evaluation, and specifically how these perceptions informed their practice (saldaña, 2013). great care was taken by the researchers to value the contributions and input of the participants, specifically the opportunity to work collaboratively with practitioners to increase the validity and authenticity of the conclusions (miles, huberman, & saldaña, 2014). a semi-structured, purposefully broad interview structure was used to allow for participant to help direct the flow of the interview as well as answering the research questions of this study. interviews with each elementary principal occurred her/his convenience, typically during the school day. all participants agreed to allow the researchers to use a digital voice recorder to document the interview and focus on the supervision and evaluation / mette, range, anderson, hvıdston, nıeuwenhuızen & doty 713 conversational approach to the interview. as a result, all interviews were transcribed verbatim and used for qualitative analyses. table 1. demographic information for elementary schools studied school/ participant id enrollment % free/ reduced lunch number of teachers principal gender years of principal experience teaching background of principal 1 152 54.5 8 female 16 classroom teacher 2 243 48.6 15 female 4 classroom teacher 3 346 27.7 21 female 5 classroom teacher 4 355 24.8 20 female 20 classroom teacher 5 399 26.8 26 female 9 classroom teacher 6 467 23.6 29 male 8 music teacher 7 518 17.2 30 female 7 classroom teacher 8 534 32.2 30 female 15 pe teacher initial coding was used to identify emergent themes, and the wicked problems theoretical framework was used to then conduct second cycle analysis (creswell, 2013). then, using an elaborative coding process, the researchers built on their understanding of the previous analyses from this large data set to help refine constructs teacher supervision and evaluation (saldaña, 2013). thus, through the data analysis of this study, the themes that emerged helped frame the leadership characteristics of the eight principals who lead high achieving schools, focusing on the conceptualization and implementation of teacher supervision and evaluation, as well as the differentiated support provided for individual teachers based on experience and ability. to provide additional validity and reliability to the study, the researchers gathered additional observational data and documents during the data collection process (miles, huberman, & saldaña, 2014). moreover, district documents related to the supervision and evaluation of teachers, state teacher standards, and the state educator evaluation model were collected and analyzed, and were particularly important in the context of this study. in doing so, the triangulation of data and collected documents provided multiple perspectives to inform the study (creswell, 2013), specifically regarding teacher supervision and evaluation in the selected midwest district. results two themes emerged from the interviews and analyses of this study regarding the conceptualization and implementation of teacher supervision and evaluation, as well as the differentiated support offered by high functioning elementary principals to teachers based on individualized ability and experience. one finding is there is overlap between the theoretical definitions of teacher supervision and evaluation, namely that both are intended to support and monitor instruction, target areas of ongoing improvement, and develop a collective building conscious of instruction. a second finding is the notion of a principal serving as an instructional coach, whereby she/he fosters trusting relationships with teachers, values feedback from teachers to target school improvement efforts, and international electronic journal of elementary education, 9(3), 709-724, march, 2017 714 differentiates individual teacher improvement efforts based the level of teacher expertise. thus, despite the wicked problem that creates tension and conflict between the intentions of supervision and evaluation, the high-achieving elementary principals in this study were able to reconceptualize their role to provide both and support teachers to grow and develop as professionals. supervision and evaluation: the venn diagram of instructional leadership the first component of this study was to understand how teacher supervision and evaluation were conceptualized ideologically by eight high-achieving elementary principals. of the eight principals who led high-achieving elementary schools from the selected urban school district, three common beliefs of the school leaders emerged regarding the overlap of supervision and evaluation. these factors inform the ongoing, day-to-day practices of their instructional leadership practices and vision for the school building. these notions of instructional leadership were largely influenced by the beliefs that the overlap of supervision and evaluation allowed principals to a) support and monitor instruction, b) target areas of ongoing improvement, and c) develop a collective building conscious of instruction that informs instructional practice. not only did these leadership beliefs value the ability to support and monitor the instructional environment of their respective school buildings, but they also informed principals on when they needed to provide more specific direction to struggling teachers and when they should allow teachers to drive their own professional development opportunities (see figure 1). moreover, it appears that because principals could focus on the strengths of teachers while simultaneously providing a supportive environment to promote individualized teacher growth, the school culture supported and valued a collaborative work environment that built a collective conscious of continual improvement, for both students and teachers. figure 1. supervision and evaluation: the venn diagram of instructional supervision support and monitor instruction. perhaps the most important aspect of the instructional supervision provided by the principals in this study was their ability to be present in classrooms in order to create awareness of the instruction being provided by teachers in their buildings. not only did this allow principals to provide teachers with quick, formative supervision and evaluation / mette, range, anderson, hvıdston, nıeuwenhuızen & doty 715 feedback in an informal manner, but it also helped them identify areas of improvement teachers might target and provide teachers with the support needed to successfully improve as an instructor. i think that can be the purpose behind it for an administrator…[to] look at the quality of what’s going on in your building and make sure that you’re maintaining certain standards. [p5] i try to be listening and if there’s things that i can get for them, support them with. i feel like their plates are very full, so if there’s anything that i can do to help facilitate and make things easier i’m happy to do that. i feel like visibility is probably one of my strengths. i’m in the classrooms pretty much daily in some format whether it’s a fly-by saying, ‘good morning’ or doing walk-throughs or just being very visible and present. [p7] by ensuring their presence in teachers’ classrooms, the elementary principals in this study were able to help support and monitor the learning environment in a supervisory, formative manner, which in turn was used to inform the eventual identifiers for each teacher’s summative evaluation. thus, principals used the information gathered from their walkthroughs to develop professional development plans that were then evaluated through ongoing observations. i think my greatest strengths are simply being visible, visibility in the classroom, being accessible to teachers and being willing to help and support them as they need it. and with evaluation, i think just giving feedback to teachers. [p3] as a result, principals used this concept to target ongoing improvement, which informed both supervision and evaluation. target areas of ongoing improvement. regardless of the ability of the individual teachers in their respective buildings, a common leadership trait of the principals in this study was the belief that all teachers are able to continually improve their instruction, whether they are considered high-functioning or low-functioning. through an unwavering focus on student achievement, principals provided instructional leadership for all teachers that impacted both the supervision and evaluation of instructors. i think ultimately it’s to improve instruction and then to improve student outcome[s]. so, you know, when you go in, during that whole evaluation process you’re looking at how, how can you be continually getting better and how can we continually get our scores up and get our achievement higher? [p4] thus, teachers were expected to be able to measure and increase student knowledge, and continually focus on how their instruction might improve with professional development opportunities. for teachers that were novice instructors or struggled to engage in selfreflection, the administrators provided more principal-directed professional development opportunities. conversely, those teachers that were high-functioning and able to selfreflect on how to improve their instruction, the elementary principals in this study gave them more leeway to select and implement their own professional development activities. as commented earlier, however, all improvement efforts were monitored by principals, regardless of teacher ability. in targeting areas of improvement, principals leading these high-achieving schools reflected on the importance of how to tactfully approach individual teachers, as well as collective faculties, and the targeted improvement efforts. i think it’s real critical that i read the situation, i think not just my building, but individuals, and know if it’s the time that they need to be pushed, and i really need to support and really push them out there to go to the next level. or, is it a time that we need to just kind of hold right there and practice to get better at what we’re doing or is it a time to really pat them on the back and celebrate for a little bit and let us get our feet back under us before we push international electronic journal of elementary education, 9(3), 709-724, march, 2017 716 again? and i think that’s my job, is to be able to know that timing. timing is everything so to speak. [p8] well, when i’m doing walkthroughs or going through the evaluation cycle or when i’m doing my surveys with the teachers, i try to pick things that are going to be relevant and meaningful that ties to our school improvement plan…so we’re not kind of doing just the scattered gun approach, but… [rather] that intentional focus. [p7] whether building-wide or individual focused, the elementary principals in the study understood the need to provide situational instructional leadership, thereby acknowledging there is not a one-size-fits-all approach to teacher supervision and evaluation. through the reinforcement of targeting areas of growth to support ongoing improvement, the high-functioning principals were able to develop a collective building conscious about instruction and student engagement. develop a collective building conscious of instruction that informs vision. a third component of the intersection of supervision and evaluation provided by principals in this study is the notion of creating a collective faculty conscious in regards to selecting school improvement efforts, while still valuing the improvement efforts of individual teachers. the ability to principals to provide their own input to influence various aspects of school improvement efforts, but also value the voice and feedback from teachers themselves, seemed to create a balance of input from both administrators and teachers that influenced the instructional vision of the school. we formed our collaborative learning teams, our whole faculty study groups and … the leadership team here at school chose what the overall, overarching theme was going to be for our learning this year. but then each whole faculty study group could pull out a piece of what was under that umbrella to fit what they wanted to study the most either at their grade level….they really got to choose where their learning was going to be focused, and i think that’s important to give them that ownership just like the kids in the classroom, you know? [p6] we’re trying to put all of our time, not all of our time, but a lot of our energy in our professional learning in how we can improve that nonfiction reading piece to improve scores overall in all subject areas. so that will be our site goal, and then you know, if there is an individual, you know, like individual plans that need to be made based on observations…then that will just be done on a one-on-one basis. [p4] by allowing teachers to have input on areas of improvement, principals in this study attempted to address the wicked problems of the conflicts and tensions between supervision and evaluation by developing a culture that valued the professional opinions of teachers while still providing necessary leadership and direction-setting from the principal position. the principal as an instructional coach the second component of this study was to understand how high-functioning elementary principals provided differentiated support for individual teachers based on experience and ability. a strong theme that emerged was the idea of these elementary principals seeing themselves as a coach in promoting instructional supervision. thus, building administrators who provided leadership to the eight high-achieving schools seemed to provide leadership as a coach and promoted the notion of teamwork in order to foster continual teacher improvement and greater student achievement. this was largely influenced by a) fostering strong, trusting relationships among teachers and administrators, b) valuing feedback from teachers to target school improvement efforts and identify necessary professional development, and c) differentiating individual teacher improvement efforts based on level of expertise to ensure. these factors were pervasive among the interviewed principals, suggesting a mixture of shared leadership and direct supervision and evaluation / mette, range, anderson, hvıdston, nıeuwenhuızen & doty 717 improvement efforts to targeting continual teacher improvement in order to attain or maintain high student achievement. by differentiating the supervision and evaluation of teachers based on their level of function, these elementary principals were able to empower highly effective teachers to have input and buy-in to school improvement efforts, but also balance overall school improvement by providing leadership to struggling teachers. moreover, the principals seemed to perceive their function as a leader to allow teachers to grow and knowing when to provide direct interventions to help teachers improve, rejecting the notion that evaluation is about “getting rid” of underperforming teachers as some school reform efforts suggest, but rather valuing the importance of providing supervision to build an inclusive school culture that is supportive of change, growth, and experimentation to meet the ever changing needs of students. building a team with a strong foundation: trusting relationships. a major component of principals’ leadership styles in the eight high-functioning schools focused on fostering trusting relationships with teachers. by developing a supervision cycle that focused on providing feedback built on trusting relationships, principals were able to provide feedback that targeted specific instructional improvement efforts, and thus valued a team mentality, allowing the principal to focus efforts as a coach in terms of celebrating strengths and helping teachers target improvement of individual weaknesses. one of the principals explained her greatest strength in providing teacher supervision and evaluation as the idea of being a coach. well for me, i’m a coach at heart…but i really approach it that way because, you know, you’re taking a whole group of people with a lot of different personalities and a lot of different strengths and weaknesses and you’re trying to mesh them together for a common goal. and my whole purpose here is to bring out everybody’s strengths and, you know, [identify] weaknesses while we help each other get better in those areas, if we elevate everybody’s game, so to speak…. but i’m just trying to coach them to bring out the best in them, build some trust with them so that they’ll also get outside their comfort zone to work on their weaknesses and know that it’s not a ‘gotcha’ situation, it’s a ‘i’m not going to be upset with you unless you’re refusing to try’. [p8] another principal described her role as an instructional leader by commenting on the importance of focusing on the individualized growth and improvement of teachers: it’s about their personal growth and development…. i also think a part of that strength is that relationship building with teachers. i think being an effective coach you have to have a relationship and a buy-in and a respect for them. trust building i think would probably be tied to that relationship building. [p5] thus, the principals of these highly effective elementary schools commented on the importance of building trusting relationships and being supportive of individualized growth in order to serve as a coach for teachers to help target ongoing instructional improvement within their buildings. as a result of their approach to providing instructional leadership, the elementary principals were able to target specific growth plans for individual teachers despite the highly prescriptive nature of district supervision and evaluation requirements. by focusing on creating strong, trusting relationships, principals were able to create an environment that focused on the ability to improve and valued personal growth, development, and goal setting. for me, i use a cognitive coaching model as much as i can with the constraints of our evaluation system that we have to use…. it’s my experience it’s really all about teacher growth…. i think being an effective coach you have to have a relationship and a buy-in and a respect for them. trust building i think would probably be tied to that relationship building. [p5] international electronic journal of elementary education, 9(3), 709-724, march, 2017 718 i spent a lot of time just carrying around a coffee cup as opposed to a clipboard or writing anything down and just walking in and out of rooms just to begin to establish that i was going to be coming in but it wasn’t to be invasive. [p8] consequently, the effectiveness of the elementary principals in this study to play the role of a coach as the instructional leader is predicated on the ability to build meaningful, trusting relationships with faculty prior to engaging in critical feedback. in doing so, these principals disrupt some of the circular logic of the wicked problem that separates supervision from evaluation, and in doing so helps identify solutions to the problem of placing blame on teachers for low student achievement. instead, focus is placed on trust between teachers and principals. the feedback loop: valuing the voice of teachers. in the eight high achieving schools, the openness and willingness for principals to gather and use teacher feedback to inform decisions on selecting professional development opportunities to improve student achievement seemed to address some of the issues of the wicked problem caused by the conflicts of supervision and evaluation. the elementary principals in this study were able to involve teachers in setting their own goals and providing support to help accomplish agreed upon improvement efforts. principals also commented on the importance of valuing the knowledge of teachers and giving them the respect to allow individuals to improve as a professional. [i use] more of a coaching type role and really, ‘hey, what can i do to support you?’ because quite honestly they know more about how to teach their content than i do, and so i just try to ask those questions that focus them on their goals and where they want to go more so than i do about the nitty gritty…. [p1] subsequently, principals in this study internalized a team approach by valuing teacher’s opinions and working collectively to produce a supervision and evaluation environment that focused on increasing or maintaining high student achievement rather than reinforcing a culture of compliance. additionally, principals in this study clearly valued the voice of teachers as a collective faculty to help determine targeted professional development opportunities. the ability and the desire for these principals to listen to the voice of their faculty not only increased their supervisory awareness; it also helped choose professional development responses to building-wide student achievement concerns (see figure 2). well, again i think that takes a lot of listening. you have to be willing to really listen to the teachers about what do they, you know, what are they missing that they feel like they need professional development for. just last week we formed our collaborative learning teams… [and] the leadership team here at school chose what the overall, overarching theme was going to be for our learning this year. but then each whole faculty study group could pull out a piece of what was under that umbrella to fit what they wanted to study the most either at their grade level or if they’re a multiage group, you know, the vertical alignment in that process, but they really got to choose where their learning was going to be focused, and i think that’s important to give them that ownership just like the kids in the classroom you know. if you want student input so that that guides their learning the you’ve got to have the teacher input to guide their learning you know? [p6] all those times that i’m in and out [observing classrooms] i’m talking with teachers and getting [feedback], it helps me identify where we need additional training and…so that they can keep giving me feedback…. i mean, i don’t want to miss anything because the communication lines aren’t open with us…. it gives them a chance to actually evaluate and, you know, where are they still feeling like they need additional support or where do they feel they need additional administrator support because, you know, sometimes those aren’t the same thing. [p8] supervision and evaluation / mette, range, anderson, hvıdston, nıeuwenhuızen & doty 719 thus, by allowing teachers to give feedback on what support might be needed to help target school-wide improvement efforts, the elementary principals in this study were able to further develop and support a team atmosphere that valued the voices of teachers and kept the overall effort of the school on student achievement. figure 2. the revised cycle of teacher supervision and evaluation (as adapted from zepeda, 2012) high or low-functioning? the basis for differentiation. differentiation of supervision and evaluation provided by principals in this study occurred based on the perceived level of teacher expertise and need. specifically, the observations of each individual teacher by the principals in this study determine how direct a principal would be when reviewing proposed teacher professional development plans. teachers who were high performing, able to reflect on practices, and highly self-motivated, tended to be allowed to develop individualized professional development and needed little additional administrative support. conversely, teachers who were low performing, unable to readily reflect on instructional practices, and showed low levels of motivation to improve tended to receive much more direct, principal-determined professional development to target areas of improvement (see figure 2). teacher teacher-directed (high-functioning) principal professional development supervision evaluation creates principal awareness principal-directed (low-functioning) non-renewal feedback new hires international electronic journal of elementary education, 9(3), 709-724, march, 2017 720 i took a cognitive coaching approach with all of them, but with the higher performing teachers i sort of, if they were doing great i kind of asked them what do you, what do you want to work on and just let them go because they were very self-motivated, they knew what they wanted to work on and they were always trying to get better whereas those who were just getting started or had not, were not as high as performers i maybe would try to direct them a little bit in areas that i saw as needing some support. [p3] i think a lot of it though is the conversations i have with the staff i probably use more of a cognitive coaching approach with those staff members that are those master teachers and really just try to coach them more than i do someone who might be struggling or who might really be new to the profession and just needs a lot of scaffolding where i might be a little bit directive, i still try to use that cognitive coaching approach but sometimes you just need to say, you just got to say it and so i think you just kind of have to figure that out with your staff but i think that i probably fall back on those cognitive coaching strategies and how i have conversations with different staff members. it really just kind of depends on where they are in their quest towards mastery. [p1] by differentiating support for individualized professional development opportunities, the elementary principals in this study were able to better able to target areas of improvement for those teachers struggling while still valuing continual growth and development by all teachers in their respective building. while more autonomy was given to teachers who were highly motivated to improve, and who were self-reflective enough to target their own areas of improvement, principals in this study expected growth from all teachers, regardless if they were low-performing or high-performing. i think that when you are…working with someone who may need a little bit more support, that support is going to be a little bit more directive as far as, ‘ok these are the things that i need to see going on in your classroom’ maybe because there’s just a low level of consciousness about those things and you need to draw their attention to it specifically. whereas if you have a teacher that is a little bit more with it, and just a good solid teacher, it may be more directed by them as far as the areas of growth that they’re looking for in themselves and what i may go in and monitor for, you know, help them monitor and grow. [p5] thus, principals in this study seemed to have an expectation that all teachers would continually improve their instruction, and as a result, improve the overall ability of the school to provide high quality education to students. in doing so, the elementary principals in this study address the wicked problem of society placing blame on teachers for low student-achievement, but rather focus on the importance of instructional ability and motivation to continuously improve. discussion this qualitative study was conducted to understand elementary principals’ views about teacher supervision and evaluation in eight high performing elementary schools, including how supervision practices created high performance and ultimately informed teacher evaluation. the results add to the literature concerning teachers’ formative supervision and summative evaluation, specifically principals’ responsibilities to engage in instructional leadership to build the capacity of teachers. in sum, the results can be summarized as, 1) beliefs about the ideologies of teacher supervision and evaluation become intertwined when implemented in action, and; 2) the role of principal as an instructional coach can be accomplished through building strong relationships with teachers and valuing teacher feedback to provide differentiated professional development opportunities. the elementary principals in this study highlight a gap between theory and practice by conceptualizing the ability to provide supervision and evaluation independently, but also supervision and evaluation / mette, range, anderson, hvıdston, nıeuwenhuızen & doty 721 acknowledging there is an intersection between the two functions, which literature has suggested are mutually exclusive (glickman et al., 2014; hinchey, 2010). using a venn diagram framework to analyze leadership perceptions of supervision and evaluation, principals in the study described notions of instructional leadership that were largely focused on the intersection of the two functions, namely the need to support and monitor the instructional environment, the targeting of ongoing improvement efforts buildingwide as well as individually, and valuing teacher opinion when informing the vision of the school building as it pertains to instruction. the ability of the principal to be an effective instructional leader by helping teachers focus on goals of lessons and identifying areas of improvement is inherently related to supervision (formative feedback), but interestingly enough, principals in this study suggest a connection to evaluation that is informed by ongoing supervision, one that takes into account supervision and influences the yearly evaluation (summative feedback). thus, while the two functions are separate, they inform one another, and as such can be considered a wicked problem (rittel & weber, 1973), specifically the tension between the two functions as in-depth supervision of instructional practices and professional development are necessary to make evaluative decisions about ongoing employment. perhaps the most important factor to point towards school success in this study was how principals supported their faculty as an instruction coach, specifically with targeting the constant improvement of instruction within their respective buildings. these successful elementary principals were able to challenge their teachers to accept the notion that continual professional development was an expectation, and through building strong, trusting relationships, as well as valuing teachers’ feedback, principals were able to differentiate improvement efforts for individual teachers. by understanding the individualized needs of teachers, developing a sense of trust to continually improve instruction as a collective faculty, and addressing agreed upon areas of improvement efforts, these principals were able to ensure all teachers received targeted improvement efforts through differentiated supervision that guided improvement efforts regardless of the level of teacher expertise in the name of increasing student achievement (range, mckim, mette, & hvidston, 2014; sergiovanni & starrat, 2002; memduhoglu, 2012). conclusions the literature regarding teacher supervision and teacher evaluation suggests separate functions that focus on different aspects of developing a school staff (ponticell & zepeda, 2004; eady & zepeda, 2007; hazi & ricinski, 2009). supervision is focused on the ongoing development of teachers through the use of formative feedback (glickman et al., 2014; zepeda, 2012) while evaluation is focused on the measurement of teachers’ abilities to instruct at a high level through a summative process (darling-hammond, 2013; platt, tripp, ogden, & fraser, 2000). however, based on this study of eight high-performing schools in one large district, there appears to be a gap between theory and practice. herein lies the wicked problem of supervision and evaluation – not only might there be overlapping of beliefs about the function of supervision and evaluation (the intersection between the two functions), but there differences of opinion about how instructional leadership is provided, as seen by the friction between theory and practice. the definition of what is supervision as a theory is something that has been debated for years among scholars (burns, jacobs, & yendol-hoppey, 2015; glickman, 2013; goldsberry, 2014). however, there is a real need to acknowledge that practice can inform theory as much as theory can inform practice. in this case, high-functioning principals who support teachers and who have high student achievement aren’t simply teaching to a standardized test. instead, they are focused on developing teachers on an individual basis, and are attempting to create a school climate where continuous growth and development international electronic journal of elementary education, 9(3), 709-724, march, 2017 722 are embedded within everyday practice. moreover, these eight elementary principals are not implementing a top-down agenda; rather they are suggesting that teachers, regardless of their years of experience, have expertise in different areas and are responsible for reflecting on their own growth. if teachers cannot, or are unwilling to, embark on this reflection, then it becomes the principal’s job to ensure this growth happens to maximize student success. given these observations, researchers would be wise to continue to study exemplar instructional leaders to understanding more about how practice might drive the development of new theory. as such, future research could examine how the theory of supervision, the formative feedback provided to teachers intended to promote growth as an instructor, can exist in practice within the current high stakes agenda of school accountability that predominately focuses on evaluation as a human resource function. moreover, the with additional research, focusing on how supervision can serve as a human resource function to grow teachers, not simply remove them if ineffective, might be considered highly beneficial to practitioners and rethink the current paradigm of supervision as a theory. these research efforts have policy implications for state departments of education throughout america, particularly as new evaluation and professional growth systems (mette & fairman, 2016). in light of the findings from this study, one question is begged to be asked, for both researchers and practitioners: if the instructional leadership offered by principals helps teachers improve their instruction, and student achievement is positively impacted, does it really matter if the process is labeled as supervision or evaluation (hazi, 2016)? in practice, the terms are often used incorrectly as synonyms, with a large amount of energy focused on the evaluation of teachers and little support offered in the way of instructional coaching, which contributes to the wicked problem of teacher supervision and evaluation. in theory, the terms are delineated and considered mutually exclusive, however there appears to be a widening gap between the acceptable definitions of researchers and practitioners. within these differences of opinions, however, none of the discrepancies matter if educators, as a whole, cannot agree upon common inroads to improve the american public education system and reduce the asinine approach to american public school reform. research has shown that school improvement efforts are quite simple – the focus on and monitoring of instruction if crucial, and teachers need to be able to provide feedback to and accept feedback from principals regarding professional development opportunities in order to function as a learning organization (datnow & castellano, 2001; marks & nance, 2007; monk, 2008; senge, 1990). these eight elementary principals offer an example of how this can be accomplished, and both researchers and practitioners should take note. • • • references alila, s., määttä, k., & uusiautti, s. (2015). how does supervision support inclusive teacherhood? international electronic journal of elementary education, 8(3), 351-362. beach, d. m., & reinhartz, j. (1989). supervision: focus on instruction. new york: harper & row. burns, r., jacobs, j., & yendol-hoppey, d. (2015, october). in search of a common framework of preservice teacher supervision using meta-analysis. paper presented at the annual meeting of the council of professors of instructional supervision, ft. worth, tx. creswell, j. w. (2013). qualitative inquiry and research design: choosing among five approaches (3rd ed.). thousand oaks, ca: sage. supervision and evaluation / mette, range, anderson, hvıdston, nıeuwenhuızen & doty 723 darling-hammond, l. (2013). getting teacher evaluation right: what really matters for effectiveness and improvement. new york: teachers college press. datnow, a., & castellano, m. e. (2001). managing and guiding school reform: leadership in success for all schools. educational administrative quarterly, 37(2), 219-249. desander, m. k. (2000). teacher evaluation and merit pay: legal considerations, practical concerns. journal of personnel evaluation in education, 14(4), 307-317. eady, c. k., & zepeda, s. j. (2007). evaluation supervision, and staff development under mandated reform: the perceptions and practices of rural middle school principals. the rural educator, 28(2), 1-7. glickman, c. d. (2013, october). good and/or effective schools: what do we mean? symposium conducted at the annual meeting of the council of professors of instructional supervision, state college, pa. glickman, c. d., gordon, s. p., & ross-gordon, j. m. (2014). supervision and instructional leadership: a developmental approach (9th ed.). boston: allyn & bacon. goldsberry, l. (2014, october). where goes supervision as a field of study? considering purpose, practice, and results. paper presented at the annual meeting of the council of professors of instructional supervision, atlanta, ga. goldstein, j. (2007). easy to dance to: solving the problems of teacher evaluation with peer assistance and review. american journal of education, 113(3), 479-508. hazi h. m. (2016, october). a letter to the field: an elder proposes reconciliation with teacher evaluation. paper presented at the annual meeting of the council of professors of instructional supervision, tampa, fl. hazi, h. m., & rucinski, d. a. (2009). teacher evaluation as a policy target for improved student learning: a fifty-state review of statute and regulatory action since nclb. education policy analysis archives, 17(5), 1-19. hinchey, p. h. (2010). getting teacher assessment right: what policymakers can learn from research. boulder, co: national education policy center. koski, w. s. (2012). teacher collective bargaining, teacher quality, and the teacher quality gap: toward a policy analytic framework. harvard law & policy review, 6, 67-90. margolis, c. & buchanan, r. (1995). the idea of design. cambridge, ma: mit press. marks, h. m., & nance, j. p. (2007). contexts of accountability under systemic reform: implications for principal influence on instruction and supervision. educational administration quarterly, 43(1), 3-37. marshall, k. (2013). rethinking teacher supervision and evaluation: how to work smart, build collaboration, and close the achievement gap. new york: john wiley & sons. marzano, r. j., & toth, m. (2013). teacher evaluation that makes a difference: a new model for teacher growth and student achievement. alexandria, va: ascd. memduhoglu, h. b. (2012). the issue of education supervision in turkey in the views of teachers, administrators, supervisors, and lecturers. educational sciences; theory and practice, 12(1), 149-156. mette, i. m., & fairman, j. (2016). piloting pe/pg systems in maine school districts: lessons learned. augusta, me: maine education policy research institute. mette, i. m., range, b. g., anderson, j., hvidston, d. j., & nieuwenhuizen, l. (2015). teachers’ perceptions of teacher supervision and evaluation: a reflection of school improvement practices in the age of reform. educational leadership review, 16(1), 16-30. miles, m. b., huberman, a. m., & saldaña, j. (2014). qualitative data analysis: a methods sourcebook (3rd ed.). thousand oaks, ca: sage. international electronic journal of elementary education, 9(3), 709-724, march, 2017 724 monk, d. h. (2008). reflections and commentary from the field: connecting the reform of administrator preparation to the reform of teacher preparation. educational administration quarterly, 44(2), 282-295. no child left behind (nclb) act of 2001, pub. l. no. 107-110, § 115, stat. 1425 (2002). oliva, p. f., & pawlas, g. e. (2004). supervision for today’s schools (7th ed.). new york: john wiley and sons. platt, a. d., tripp, c. e., ogden, w. r., & fraser, r. g. (2000). the skillful leader: confronting mediocre teaching. acton, massachusetts: ready about press. ponticell, j. a., & zepeda, s. j. (2004). confronting well-learned lessons in supervision and evaluation. nassp bulletin, 88(639), 43-59. range, b. g., anderson, j., hvidston, d. j., & mette, i. m. (2013). how teachers’ perceive principal supervision and evaluation in eight elementary schools. journal of research in education, 23(2), 65-78. range, b. g., mckim, c., mette, i. m., & hvidston, d. j. (2014). aspiring principals’ perspectives about teacher supervision and evaluation: insights for educational leadership preparation programs. educational leadership review, 15(1), 1-17. rittel, h., & webber, m. (1973). dilemmas in a general theory of planning. policy sciences, 4, 155169. ryan, t. g., & gottfried, j. (2012). elementary supervision and the supervisor: teacher attitudes and inclusive education. international electronic journal of elementary education, 4(3), 563571. saldaña, j. (2013). the coding manual for qualitative researchers (2nd ed.). thousand oaks, ca: sage. senge, p. (1990). the fifth discipline: the art and practice of the learning organization. new york: doubleday. sergiovanni, t. j., & starratt, r. j. (2002). supervision: a redefinition (7th ed.). new york: mcgraw hill. southgate, e., reynolds, r., & howley, p. (2013). professional experience as a wicked problem in initial teacher education. teaching and teacher education, 31, 13-22. weisberg, d., sexton, s., mulhern, j., & keeling, d. (2009). the widget effect. brooklyn, ny: the new teacher project. wiles j, bondi j (2004) supervision: a guide to practice (6th ed.). upper saddle river, nj: pearson, prentice hall. zepeda, s. j. (2012). instructional supervision: applying tools and concepts (3rd ed.). larchmont, ny: eye on education. international electronic journal of elementary education, 2016, 8(3), 333-350 a study to determine the mental models in preschool children’s conceptualization of a desert environment berat ahi̇  kastamonu university, turkey received: 19 november 2015 / revised: 29 december 2015 / accepted: 17 january 2016 abstract this study aimed to determine mental models and identify codes (schemes) used in conceptualizing a desert environment. the sample for this study consisted of 184 – out of a total population of 3,630 children in preschool education in the central district of kastamonu, turkey. within the scope of this study, the children were initially asked to draw a desert-themed picture, followed by a semistructured interview to seek their opinions about the drawing and clarify what a desert environment meant to them. according to the findings, the children referred to 38 different codes relevant to the conceptualization of a desert environment; the most frequently used were the sun (f= 160, 86.9%), sand (f= 100, 54.3%), cacti (f= 74, 35.3%) and camels (f= 52, 28.6%). during the interview phase, 33 children described a desert as a place where there is no life, although a significant number of the children (f= 65, 39.1%) did describe a desert as a place where plants and animals live. moreover, the sun and its rays were disproportionately bigger in size, in order to emphasize the excessive heat associated with the specific ecosystem found in a desert environment; to reinforce this, humans drenched in sweat, the absence of trees and the prevalence of cacti and exotic wildlife, including camels, scorpions and lizards, were all features of the children’s drawings. based on these findings, it was inferred that the mental models in some of the children (f= 72, 39.1%) were scientifically informed, with a degree of accuracy, about a desert environment. on the basis of the findings, it is considered that determining mental models in children in relation to different ecological concepts can be beneficial to teachers and curriculum programmers involved in environmental education. keywords: desert, mental model, draw and explain, child, ecological concept. introduction after every natural disaster, we are reminded that all the ecosystems in the biosphere are under threat as a result of the human exploitation of natural resources. considering that the biosphere is essential to the preservation of all life forms (callenbach, 2012), such a threat poses a shared and significant problem for humanity. an ecosystem is a natural system that originates from the regular and balanced interaction between all living and non-living things (abiotic elements), and which is static-oriented and self-renewing (dinç  berat ahi, kastamonu university, faculty of education, department of pre-school teacher education, 37100 kastamonu, turkey. phone: +90 506 5951002, email: bahi@kastamonu.edu.tr international electronic journal of elementary education vol.8, issue 3, 333-350, 2016 334 & özkaya, 2015; kışlalıoğlu & berkes, 2012). in numerous international forums, a wide range of administrative and scientific precautions have been taken in order to find solutions to the increasing number of environmental problems facing the planet (özsoy, özsoy & kuruyer, 2011). according to the intergovernmental panel on climate change, factors, such as temperature, rain and soil moisture, wind, humidity, and nutrient cycle, determine the structure and features of an ecosystem (noble, 2014). about 45% of the earth’s surface is covered with desert and semi-desert areas (shekhawat, et al., 2012). most deserts occur between 20o and 40o latitude (noble, 2014). although it would appear that temperature determines the basic features of deserts, they are generally characterized by temperate climates with low annual rainfall, high evaporation, and large seasonal and daily temperature contrasts. they are also home to a significant number of endemic plants, reptiles, and fish (national fish, wildlife and plants climate adaption strategy, 2012). therefore, like forests or oceans, deserts need to be protected against man-made climate change. that said, in recent years, it has been forests, oceans and seas that have generated the most environmental concern; less attention has been given to desert ecosystems, which are also critical to the preservation of all life forms (fenk, et al., 2014). it has been reported that the rise in global temperatures (li, chen, van der tol, luo & su, 2014), leading to the melting of icebergs (barrett, et al., 2008), the increase in rainfall levels, dry air, and changes in humidity rates, pose a significant threat to the fragile life forms found in deserts (united states global change research program, 2009). exposure to tourism has also resulted in negative implications for the desert ecosystems. bbc nature (bardo, 2012) and the independent (connor, 2006) have published articles that bring this situation to light, warning that the human impact of tourism as well as off-road racing are particularly harmful to mammals and plants found in the desert. moreover, desert ecosystems are unhelpfully compared with the notion of desertification, in which terrain that becomes infertile and unable to support living things is referred to as a desert (abella, chiquoine, newton & vanier, 2015): a notion that is both ignorant and anthropocentric in respect of desert ecosystems (world resource institute, 2005). there is a risk, however, that such a notion may infiltrate the education system and allow the perception that deserts are worthless terrain to prevail (judson, 2011). in order to prevent against such a scenario, individuals need to acquire scientific-based knowledge at the earliest opportunity in life. as part of the belgrade workshop (1975), six core aims were agreed in relation to environmental education (unesco, 1975), including raising awareness and shaping positive attitudes from the start of early childhood education (kahriman-öztürk, olgan & güler, 2012). awareness, or the state of being aware, when examined in detail, involves the interaction between sensory and cognitive processes with schemes (becker, kleinböhl & hölzl, 2012). knowledge, conditioning and attention are required for awareness to develop properly (ross & nelson, 1973). in piaget’s (1970) study, it is understood that learning occurs by an individual’s existing prior knowledge interacting with new data in order to create new structures of knowledge. it is also understood that learning has a strong relation with the individual’s own world of concepts (piaget, 1970). learning biological and essentially ecological concepts follows a similar process. knowledge concerning biological concepts can be found in the consciousness from early childhood and is defined as “naive biology” (hatano & inagaki, 1997). this kind of knowledge means that children can establish definitions that are compatible with the biological world, in a simplistic sense, allowing them to understand the nature of biological entities and processes (inagaki & hatano, 2006). moreover, such knowledge relies on cognitive structures that are resistant to change (sungur, tekkaya & geban, 2001). carey (1985) emphasized that children cannot explain biological concepts scientifically before they are a study to determine the mental models in preschool children’s / ahi 335 10 years old (akt. prokop, kubiatko & fančovičova , 2007). vosniadou and brewer (1994), meanwhile, proposed that, when learning involves structured concepts of relative complexity, children segment the concepts’ structure into units and compartmentalize them within mental structures. similar cognitive processes are performed when learning about ecological concepts. it is particularly challenging, however, to learn about structured concepts, such as forests, deserts and the oceans, that involve terms such as habitat and abiotic elements (hatano & inaagaki, 1997; palmer, 1994; strommen, 1995). in other words, while concepts can be acquired more instinctively in early childhood, the scientific knowledge needed to properly explain them emerges in later years (hatano & inagaki, 1997; inagaki & hatano, 1993). according to the constructivist approach, cognitive structures are schemes that emerge through heredity and regenerate quickly after the birth, before they are categorized and constructed (gilbert, pietrocola, zylbersztajn & franco, 2000). these structures establish a relation between cognitive schemes and reality; this schematic approach to dealing with reality is called a mental model (gilbert, 2004). gentner and stevens (1983, s. 1) characterize mental models as “structures that are related to the knowledge of the human about the real world and knowledge processing phase”. given that memory units involve symbols that reflect knowledge, rather than knowledge itself, the determination and structure of mental models play a significant role in the process of learning (brunning, schraw & norby, 2014). furthermore, developments in cognitive psychology have led to the use of mental modeling in the educational field (gentner & stevns, 1983). in johnsonlaird’s (1983; s. 397) view, the most significant reason for expansion into education is that “mental models play a great role in the mental reflections of concepts, expressing problems, organizing the events, the ways of perceiving the world, and understanding the social and psychological situations in daily life”. when we think of mental models as the reflection of reality in an individual’s minds, every drawing that the individual makes about a subject, concept or phenomenon can be used in evaluating the mental models at work (moseley, desjean-perrotta & utley, 2010). by activating the imagination and thinking processes, drawing allows for an efficient way of understanding an individual’s mental structures (vygotsky, 1971). furthermore, drawings can reflect not only the knowledge schemes, but also those of feelings and instincts (yavuzer, 2010). by drawing a picture, a child may reveal their feelings, values (günindi, 2015), and mental models in visual terms, which can help determine their true thoughts, desires, and wishes (coates, 2002; einarsdottir, dockett & perry, 2009; leonard, 2006; piperno, biasi & levi, 2007). when a drawing is analyzed appropriately, the concept within it, no matter how complicated, is effective in revealing the schemes that exist in the drawer’s mind and the relationship between these and other schemes, as well as the drawer’s cognitive structures (schafer, 2012). the technique of drawing a picture has been increasingly used in determining mental models. the literature concerning mental models within the context of education focus generally on concepts related to astronomy (kurnaz, kıldan & ahi, 2012; nobes et al., 2003; panagiotaki, nobes & potton, 2009; samarapungavan, vosniadou & brewer, 1996; straatemeier, van der maas & jansen, 2008; saçkes & korkmaz, 2015; vosniadou & brewer, 1992; 1994). studies that are specifically concerned with environmental education tend to focus on the concept of “green” issues (ahi, 2015; liu & lin, 2015; moseley, desjean-perrotta & utley, 2010; shepardson, wee, priddy & harbor, 2007). palmer’s (1994) work is widely considered to be the pioneering study into how children perceive concepts related to the environment and environmental problems, while subsequent studies have tended to focus on how different ecological and biological concepts are perceived (braund, 1998; prokop, kubiatko & fančovičova , 2007; international electronic journal of elementary education vol.8, issue 3, 333-350, 2016 336 shepardson, 2002). the concept of the forest has especially dominated research in this area (ergazaki & andriotou, 2010; strommen, 1995; ahi, balcı & özcan, 2015). while judson’s (2011) study sought to determine mental models in relation to the concept of the desert, it is more concerned with how deserts are perceived in different cultures, rather than being appreciated as an eco-system. for strommen (1995), determining the level of knowledge in children and young people about the complexity of ecological and environmental structures ought to inform how educational programs are designed. taking the aforementioned studies into account, it can be considered that determining how knowledge in the mind, which creates perceptions in accordance with developments in the neuroscientific field, is stored, and in which schemes are used to explain a concept, has more kinship with recent developments in the cognitive field, than efforts to determine perceptiveness towards a concept. the aim of this study, then, is to determine the mental models in children in preschool education in relation to the concept of the desert environment, and in so doing answer the following questions.  what are the codes (schemes) that are used by children in their mental models about the concept of the desert environment?  what are the mental models in children about the concept of the desert environment? method this study is a qualitative study which aims to determine the mental models in children who are in preschool education about the concept of the desert environment. the study is also a phenomenological study based on social structuralist philosophy. according to creswell (2007), social structuralist philosophy provides the historical background to all kinds of concepts and phenomena that are experienced by individuals in daily life, and these concepts and phenomena, along with new experiences, are shaped by individuals. a phenomenological study can also be a qualitative study given that it aims to determine the reactions of individuals towards a large variety of concepts and phenomena; in a more profound sense, it also seeks to establish the experience, knowledge and perception about phenomena from an individual’s own perspective (fraenkel & wallen, 2009; wiersma & jurs, 2005). this study was conducted using a modified analytic induction approach. a modified analytic induction approach creates a narrowly scoped study question, whereby the study is expanded until it complies with the universal model in the way that it involves the inspected phenomenon (wiersma & jurs, 2005). in this study, it was assumed that the children’s knowledge about the concept of a desert was determined by their mental models. in response, the mental models concerning the concept, which are of interest to a social structuralist based phenomenological study, were examined thoroughly. codes were identified and explanatory remarks collected from the participants in the course of the research. population and sample the population for this study were children in preschool education within the central district of kastamonu, turkey. according to the turkish ministry of national education (2015), 3,630 children in total were in preschool education in this district during the 2014-2015 school year when the study was conducted. according to fraenkel and wallen (2009), although a minimum of 100 participants are enough to create a sample, attention should be paid to the generalization issue. this study was conducted with 184 children and, while the sample can be considered as representative of the study population, the researcher acknowledges that the number of the participants was a limitation of the study. in response, the researcher took great care in ensuring that the demographic variables a study to determine the mental models in preschool children’s / ahi 337 were compatible with the population (fraenkel & wallen, 2009). the demographic characteristics of the children in the sample group are demonstrated in table 1. table 1. demographic characteristics of the children in the sample girl boy total f % f % f % age 4 34 18.5 28 15.2 62 33.7 age 5 age 6 42 10 22.8 5.4 59 11 32.1 6.0 101 21 54.9 11.4 total 86 46.7 98 53.3 184 100.0 47.6% of the 3,630 children in the total population were girls, while 52.4% were boys (mne, 2015). in terms of gender, compatibility between the population and the sample group is considerably high. in both the population and the sample, the age range was between four and six years of age. in terms of age distribution, the largest age group in the population consisted of those aged six, with those aged four being the smallest; in the sample, however, the largest age group consisted of those aged five, while those aged six were the smallest. however, the difference in age distribution did not pose a significant problem given that no inferential claims were made on the basis of age. acquisition of data in the study, the data were gathered using the draw and explain technique. in other words, the pictures that the children drew and the remarks they made about them during the interview stage, constituted the research data for the study. the technique of draw and explain is one of the methods that is frequently used in the study of mental models (liu & lin, 2015; moseley, desjean-perrotta & utley, 2010; shepardson, wee, priddy & harbor, 2007). in this technique, the drawing establishes the codes (schemes) that form mental models, while the interview explores the experience and knowledge behind the mental model. the data was collected in preschools in the central district of kastamonu, turkey, during the spring semester of the 2014-2015 school year. a mutually convenient time for conducting the research was agreed with school management and teachers beforehand. the researcher was only present in the school during the appointed times. in order to minimize the impact upon the children’s preschool day, the research took place during art activity sessions, thereby locating the process within a typical class setting. the pictures were drawn on tables, with four children sat at each; only crayons or chalk were permitted for the drawing at the direction of the teachers. during the drawing stage, interaction between the children was restricted in order to prevent them from influencing each other and exchanging ideas. this was followed by the interview stage, which took place with each child individually in the playroom area, where the children are felt to be most at ease. the children who did not wish to speak with the researcher were included in the interviews with their teachers. the data acquisition protocol is demonstrated in table 2. international electronic journal of elementary education vol.8, issue 3, 333-350, 2016 338 table 2. data acquisition protocol drawing a picture of a desert landscape 1) ask the child to draw what comes to his/her mind when thinking of a desert. 2) discuss the codes in the picture with the child about what they signify, and ask them to label these codes in writing on the drawing. 3) discussed whether the codes drawn by the child have any relation to each other. questions asked during the verbal explanation 1) what comes to your mind when you think of a desert? 2) what is a desert like in your opinion? 3) is there life in a desert? 4) are the codes on your drawing related to each other? data analysis initially, the code determination was performed using the collected data. within this process, the researcher recorded the codes identified in each picture. later, the interviews in relation to each picture were analyzed, and themes regarding the remarks were collated and compared with the existing literature. already, correlation between these themes and those identified in earlier studies (judson, 2011; moseley, desjean-perrotta & utley, 2010; shepardson, wee, priddy & harbor, 2007) was observable. the children’s remarks were then placed alongside the relevant themes by the researcher. another researcher, who is an expert on environmental education, conducted the same process simultaneously, and the theme lists of both the researcher and the expert were compared using the kappa (κ) measure in spss; a high compatibility was detected with .93κ. an integrated list was then sent to another environmental education expert with the remarks, following which .92κ was achieved. accordingly, in both comparative processes, a high compatibility was detected (pallant, 2011). findings using the data collected in the study, the codes that were identified in the children’s drawings are set out in table 3 below. according to table 3, 38 different codes relating to the concept of the desert were identified in the children’s drawings. when analyzing the codes according to ages, the age five group produced the maximum number of codes (38), followed by age 4 (21 codes), and finally, age 6 (15 codes). the most significant variance in the codes was for categories relating to animals and abiotic factors. among the codes in the drawings, the sun (f= 160, 86.9%), sand (f= 100, 54.3%), cacti (f= 74, 40.2%), humans (f= 65, 35.3%) and camels (f= 52, 28.6%) were more prevalent than other codes. the fact that the most-drawn codes were also the most essential schemes within the concept of a desert ecosystem is noteworthy. in addition, the pyramid code (f= 28) within the artificial environment category is among the codes that also appears relatively often. another notable finding was that the highest number of codes was drawn by the children in the age five group, while the lowest number of codes was drawn by the children in the age six group. a study to determine the mental models in preschool children’s / ahi 339 table 3. codes(schemes) identified in the drawings regarding the concept of the desert codes age 4 age 5 age 6 total f % f % f % f % human 15 8.2 37 56.9 13 20.0 65 100.0 animal bird camel lizard butterfly insect fox ant spider cat snake bee lion scorpion rabbit dog 5 17 0 0 1 0 0 2 0 0 0 0 1 0 1 45.5 32.7 0.0 0.0 20.0 0.0 0.0 25.0 0.0 0.0 0.0 0.0 50.0 0.0 33.3 6 29 2 1 4 4 3 6 3 3 1 1 1 1 2 54.5 55.8 66.7 100.0 80.0 80.0 100.0 75.0 100.0 100.0 100.0 100.0 50.0 100.0 66.7 0 6 1 0 1 1 0 0 0 0 0 0 0 0 0 0.0 11.5 33.3 0.0 20.0 20.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 11 52 3 1 5 5 3 8 3 3 1 1 2 1 3 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 plants cactus tree palm tree bush grass flower 19 15 0 3 5 14 25.7 68.2 0.0 42.9 41.7 66.7 45 7 4 4 7 7 60.8 31.8 100.0 57.1 58.3 33.3 10 0 0 0 0 0 13.5 0.0 0.0 0.0 0.0 0.0 74 22 4 7 12 21 100.0 100.0 100.0 100.0 100.0 100.0 abiotic factors sun cloud star sky sea dune sand soil oasis 54 16 0 6 0 2 35 5 0 33.8 34.0 0.0 21.4 0.0 8.0 35.0 29.4 0.0 88 22 6 18 5 21 52 12 2 55.0 46.8 100.0 64.3 100.0 84.0 52.0 70.6 100.0 18 9 0 4 0 2 13 0 0 11.3 19.1 0.0 14.3 0.0 8.0 13.0 0.0 0.0 160 47 6 28 5 25 100 17 2 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 100.0 fruit (date palm) 0 0.0 1 100.0 0 0.0 1 100.0 natural events rainbow rain 0 1 0.0 25.0 4 1 100.0 25.0 0 4 0.0 50.0 4 4 100.0 100.0 artificial environment house car pyramid indian tent 8 2 0 0 53.3 16.7 0.0 0.0 6 10 19 0 40.0 83.3 67.9 0.0 1 0 9 1 6.7 0.0 32.1 100.0 15 12 28 1 100.0 100.0 100.0 100.0 international electronic journal of elementary education vol.8, issue 3, 333-350, 2016 340 mental model theme: the place where plants and animals live the codes in the picture: sun, camel, sand, cactus, bush picture 1. c6’s desert drawing (girl, age 4) it was also reported that the reason why the sun was disproportionately larger in size and colored orange in some of the drawings was to emphasize the excessive temperatures associated with a desert environment. it was also stated that the reason for drawing cacti or trees with a bent shape was because it is difficult to find water in a desert. in 24 of the human figures that were drawn, the faces were unhappy while in one picture the human figure appears to be sweating. likewise, there are unhappy faces on the animal drawings. furthermore, the children made more frequent use of the sand code (f= 100, 54.3%) than the soil code (f= 17, 9.2%); some of them even drew dunes (f= 25, 13.5%). the colors used in the drawings were mainly various shades of red and yellow, while some of the trees (f= 11, 5.9%) were drawn in the shape of palm trees rather than less exotic kinds of tree. it was also noted that every child in the sample perceived deserts as permanently hot. based on the codes identified in the drawings, it can be said some of the children (f= 48, 26.0%) perceived the concept of the desert as a specific ecosystem, which indicates a scientifically-based mental model. mental model theme: the place where plants and animals live the codes in the picture: sun, camel, cactus, sand picture 2. c43’s desert drawing (boy, age 5) a study to determine the mental models in preschool children’s / ahi 341 the remarks made by the children in relation to their drawings and the categories regarding the mental models based on these remarks are set out in table 4 below. table 4. mental model themes regarding the concept of desert categories age age 4 age 5 age 6 total f % f % f % f % a place where animals and plants live 23 31.9 40 55.6 9 12.5 72 100.0 there is no life 11 33.3 18 54.5 4 12.1 33 100.0 a place where plants live 9 37.5 14 58.3 1 4.2 24 100.0 a place where animals, plants, and humans live 6 30.0 9 45.0 5 25.0 20 100.0 living/nonliving (everyday life) 6 33.3 11 61.1 1 5.6 18 100.0 a place where animals live 6 40.0 8 53.3 1 6.7 15 100.0 a place of nature, a natural area 1 50.0 1 50.0 0 0.0 2 100.0 it can be seen from table 4 that the children in the sample group, in explaining their drawings, define the desert mostly (f= 72, 39.1%) as a place where animals and plants live. some of the children (f= 33, 17.9%), however, stated that there is no life in the desert. considering the table in terms of age groups, the theme of a place where animals and plants live was emphasized the most across all age groups. commenting generally on table 4, it can be stated that some of the children (f= 72, 39.1%) possessed a scientificallyaccurate mental model, while the other 112 children (60.8%) have a non-scientific-based model. with regard to the definition that a desert is ‘a place where animals and plants live’, c12 stated that, “[....] this place is so hot that humans can’t take it. animals are stronger than humans, so they can live there. the plants there have thorns. animals eat them, also they eat each other. but there can’t be cities there”. another child, c96, said that, “humans can’t live there, there is sand everywhere. but flowers live in the soil, and also some animals live in the sand, like scorpion. so they [plants and animals] live in the desert”. remarks made in reference to the desert being a place where ‘there is no life’ generally concerned the notion that a desert is too hot to live in, as exemplified by c48’s comments that, “deserts are very hot places. they are so hot that everything gets dry. rain gets dry in the air. look at my drawing. rain got dry in the air. since it is hot, living beings can’t live there”. international electronic journal of elementary education vol.8, issue 3, 333-350, 2016 342 mental model theme: the place where humans, plants and animals live the codes in the picture: sun, cloud, camel, pyramid, cactus, human picture 3. c45’s desert drawing (boy, age 6) the remarks about ‘a place where plants live’ were based on two different beliefs. the first was that plants were not regarded as living beings. c13 remarked that “only plants live in desert and that’s because they are not alive”, while c19 believed that plants “are not like animals or us, they can’t breathe. they are not affected by heat. so they exist in desert, but not many”. the second belief was that plants survive by absorbing water from the soil. as c57 stated, “roots of plants are far beneath the soil, since they are not affected by the heat, they live in the desert”, while c171 commented that “plants feed on the water in the soil. there is so much soil in the desert. so they take the water in the soil for themselves. but humans and animals can’t”. mental model theme: the place where plants and animals live the codes in the picture: sun, cloud, sand and dunes, human, cactus, camel picture 4. c4’s desert drawing (girl, age 5) 20 children who participated in the study described a desert as a place where humans, animals, and plants live. some of the remarks made concerning this category were that:  “deserts are not like cities, but all the living beings live there.” (c5)  “deserts are very hot and people sweat very much, (...) and animals go outside at night. and there are cactuses, they store the water. anyone can live there.” (c174) a study to determine the mental models in preschool children’s / ahi 343  “not as many as in here, but there are living beings in the desert, (...) and humans live there.” (c65) meanwhile, 18 children did not regard a desert environment as a much different from their everyday living environment, reflected by their remarks in the living/non-living category:  “the desert is not a different place. there are cars, roads, houses. it is a place like here.” (c123)  “there are jeeps in there, roads are covered with sand, but there are houses, people don’t go out of their houses because it is hot there.”(c93) 15 children who participated in the study defined the desert as a place where only animals live. a commonly held view was that animals are much more capable of enduring harsher climatic conditions than humans and plants, along with a belief that only animals are able to live in the desert for this reason. c76, for example, claimed that, “there is very little water in the desert. humans can’t live without water. and plants can endure for a very short time. (...) animals are very powerful. they can endure the lack of water. there are only animals in the desert”. some of the remarks in this category emphasize the capacity for animals to adapt to a desert environment. c138 gave an impressive explanation in biological terms by stating that, “humans and plants can’t hide from the sun by going beneath the soil, but animal can. camels store water on their backs, and drink it from there. so they can live there”. only two children defined a desert environment as a natural habitat. one of the remarks in relation to this category came from c6, who said that, “deserts are places where there are no humans. humans go there to picnic or visit (...) they are natural places, like forests, where there is no one”. discussion the aim of this study was to determine mental models in children about the concept of a desert environment. in the first instance, the codes that inform the mental models relating to such an environment were identified. according to the findings, most of the children (f= 160, 86.9%) included the sun code, with codes for sand (f= 100, 54.3%), cacti (f= 74, 40.2%), humans (f= 65, 35.3%), and camels (f= 52, 28.6%) also significant used. 38 different codes were identified in total from the children in the sample. all codes were classified under seven different categories, each with sub-categories as follows: 15 different codes in the animal category, nine different codes in the abiotic element category, six different codes in the plant category, four different codes in the artificial environment category, and two different codes in the natural event category, along with single-coded categories for humans and fruit. in terms of the variety of codes and the frequency of their use, resonances exist between this study and previous studies in the field. for example, barraza’s (1999) study also found that children depicted the sun, animals and trees when asked to produce drawings about their concept of the environment. in the (2007) study carried by shepardson, wee, priddy and harbor, a significant number of the participating children also included the sun, animals and plants in their drawings, while özsoy and ahi’s (2014) study showed not only the significant prevalence of the sun, animals and humans in children’s drawings about today’s environment but also that of the future. taking all these findings into account, it is reasonable to claim there are common codes specific to the environment that frequently inform the mental models in children regarding the desert ecosystem. the main explanation for this is provided by mason and langenheim (1957), in that the concept of the desert and the concept of the environment are to some extent synonymous in ecological terms, which allows for the identification of common schemes in mental models that are not complicated by any differences between scientific disciples. considering that mental models are active cognitive structures related to the international electronic journal of elementary education vol.8, issue 3, 333-350, 2016 344 mental models of different structures (freca & moreira, 2000), it can be concluded that children store the concepts of both the desert and the environment associatively in their minds. the presence of the human code in drawings regarding the environment should be interpreted carefully. it has been established in most of the similar studies in the existing literature that the human figure is pictured either rarely or routinized and unrelated (ahi, 2015; özsoy & ahi, 2014; liu & lin, 2015; moseley, desjean-perrotta & utley, 2010; shepardson, wee, priddy & harbor, 2007). in this study, while some of the children (f= 65, 35.3%) included humans as a code, only 20 children during the interview stage described the desert as a place where humans, animals and plants live. when the drawings were examined more closely, it was observable that human figures were either drawn alone or riding a camel in most of the drawings. this finding has parallels with other studies in the literature (littledyke, 2004; loughland, reid, & petocz 2002; shepardson et al., 2007; yardımcı & bağcı kılıç, 2010). according to this study’s findings, while very few children (f= 20, 10.8%) have a false mental model regarding people living in the desert, a significant number of them see humans as part of the desert environment even though they are unable to describe humans’ relationship with that environment clearly. one of the main reasons for this is that the linear development of children is not at a sufficient level to make sense of such complexity at a young age (yavuzer, 2010). as dunlop et al. (2000) stated, another reason could be how an individual’s perspective towards environment is manifest. in other words, the children who were able to accept the desert environment as including living and non-living phenomena and systems include humans in their drawings, whereas those who viewed humans as a more superior than other living things did not include humans (brechignac, 2011; herrmann, vaxman & medin, 2010). after analyzing the codes acquired during the study thoroughly, the results are impressive. some of the children gave inferences about their perception of the desert in their drawings. this is particularly true where the drawing of the sun was concerned. for example, the common depiction of the sun as orange, whose rays are thick and tall, emphasized that the desert environment is perceived as extremely hot. one of the significant findings reported by ahi (2015) is that children include depictions of soil when asked to draw something about the concept of the environment. in this study, however, it was noteworthy that the children drew sand in their desert drawing instead of soil. moreover, even in the studies that have sought to determine the perception and mental model regarding the environment (ahi, 2015; özsoy & ahi, 2014; shepardson, wee, priddy & harbor, 2007) or evaluate knowledge regarding a forest habitat (strommen, 1995; ergazaki & andriotou, 2010), camels, spiders, foxes, lizards and scorpions are hardly depicted; in the desert drawings, however, the camel was particularly prevalent. where humans and animals were drawn, they were depicted as sweating excessively and with unhappy faces; even the sun was, on occasion, given an unhappy face in response to the perception of extreme heat. what these findings indicate is that the children in the sample group perceived the concept of a desert environment as a different type of ecosystem with specific conditions of its own. the remarks made by the children concerning their drawings further support this conclusion. for example, although most of the children (f= 72, 39.1%) defined the desert as a place where plants and animals live, some (f= 33, 17.9%) defined it as a place where there is no life. the findings from the interview stage correspond to the results of judson’s (2011) study, involving older children, which found that 78% of the participants in the control group and 82% in the test group defined the desert as a place where plants and animals live. for judson (2011), this kind of response demonstrated that the children who participated in his study possessed mental models that were appropriate to the concept of a study to determine the mental models in preschool children’s / ahi 345 a desert environment. accordingly, it is considered that the children who participated in this study also possessed scientifically-based mental models about the concept. although the age groups in this and judson’s (2011) studies differ, perceptions about a desert environment do not appear to change with age (alerby, 2000). indeed, as liu and lin (2015) stated, mental models are structures affected by experience and culture rather than age. although cultures change perceptions towards the relation between humans and the natural environment (loughland, reid & petocz, 2002), common perceptions have been found in all cultures when it comes to the core environmental concepts (liu & lin, 2014). this helps to explain why the findings of this study are similar to studies conducted in different cultures. in light of the findings, the following comments are made. firstly, when seeking to determine a mental model, a number of different methods can be used other than the draw and explain technique. while drawing and explaining techniques have their distinctive advantages, it can be argued that working with different techniques may reveal different types of mental model. regarding the dynamic structure of mental models, it is important to observe how a mental model developed. understanding how and where the children structure the knowledge that they acquire should be beneficial both to the curriculum programmers and teachers. one of the basic aims of environmental education is that children and other learners possess accurate knowledge about concepts regarding the environment. furthermore, understanding the relation between knowledge and mental modeling in an individual’s mind will inform developments in the fields of neurocognitive and cognitive psychology. for this reason, it is believed that studying concepts regarding ecology within different disciplines and how they can be applied in the classroom will improve the effectiveness of environmental education. • • • references abella, s. r., chiquoine, l. p., newton, a. c. and vanier, c. h. (2015). restoring a desert ecosystem using soil salvage, revegetation, and irrigation. journal of arid environments, 115, 44-52. ahi, b. (2015). okul öncesi eğitim programına kaynaştırılan çevre eğitimi programının çocukların çevre kavramı hakkındaki zihinsel model gelişimine etkisi. (unpublished doctoral dissertation). gazi üniversitesi, eğitim bilimleri enstitüsü, ankara. ahi, b., balcı, s. & özcan, h. (2015). children and the environment: young children’s understanding of forest and deforestration. paper presented in 2015 international business & education conferences (2-6 ağustos 2015, new york, usa). alerby, e. (2000). a way of visualising children’s and young people’s thoughts about the environment: a study of drawings. environmental education resarch, 6(3), 205-222. bardo, m. (2012, september 7). desert diversity cut by ‘human activities’. bbc nature. retrevied from http://www.bbc.co.uk/nature/19466470 barraza, l. (1999). children’s drawings about the environment. environmental education research, 5(1), 49-66. international electronic journal of elementary education vol.8, issue 3, 333-350, 2016 346 barrett, j. e., virginia, r. a., wall, d. h., dorans, p.t., fountain, a.g., welch, k. a. & lyons, w.b. (2008). persistent effects of a discrete warming event on a polar desert ecosystem. global change biology, 14, 2249-2261. becker, s., kleinböhl, d. & hölzl, r. (2012). awareness is awareness is awareness? decomposing different aspects of awareness and their role in operant learning of pain sensitivity. consciounes and cognition, 21, 1073-1084. braund, m. (1991). children’s ideas in classifying animals. journal of biological education, 25(2), 103–110. braund, m. (1998). trends in children’s concepts of vertebrate and invertebrate. journal of biological education, 32(2), 112–118. brechignac, f. (2011). technology and the forces of nature: a lesson of humility calling for ecocentrism. integrated environmental assessment and management, 7(3), 409-410. bruning, r. h., schraw, g. j. & norby, m. m. (2014). bilişsel psikoloji ve öğretim. (z. n. ersözlü ve r. ülker, çev.). ankara: nobel. callenbach, e. (2012). ekoloji. (çev. e. özkan). ankara: sinek sekiz yayınevi. coates, e. (2002). ‘i forgot the sky’ children’s stories contained within their drawings. international journal of early years, 10(1), 21-35. connar, s. (2006, june 5). desert life threatened by climate change and human exploitation. the indipendent. retrevied from http://www.independent.co.uk/environment/desert-lifethreatened-by-climate-change-and-human-exploitation-481116.html creswell, j. w. (2007). qualitative inquiry & research design: choosing among five approaches. (2nd edition). london: sage. dinç, m. & özkaya, a. (2009). ekosistem. çevre bilimi (eds. m. aydoğdu & k. gezer) içinde. ss.35-70. ankara: anı. dunlap, r. e., van liere, k. d., mertig, a. g. & jones, r. e. (2000). measuring endorsement of the new ecological paradigm: a revised nep scale. journal of social issues, 56(3), 425-442. einarsdottir, j., dockett, s. & perry, b. (2009). making meaning: children’s perspectives expressed through drawings. early child development and care, 179(2), 217-232. ergazaki, m. & andriotou, e. (2010). from “forest fire” and “hunting” to disturbing “habitats” and “food chains”: do young children come up with any ecological interpretations of human interventions within a forest? research in science education, 40, 187-201. fenk, w., zhang, y., jia, x., wu, b., zha, t., qin, s., wang, b., shao, c., liu, j. & fa, k. (2014). impact of environmental factors and biological soil crust types on soil respiration in a desert ecosystem. plosone, 9(7), 1-10. fraenkel, j. r. & wallen, n. e. (2008). how to design and evaluate research in education. (7th edition). new york: mcgraw-hill. gentner, d. & stevens, a., l. (1983). mental models. east sussex: taylor & francis. gilbert, j. k. (2004). models and modelling: routes to more authentic science education. international journal of science and mathematics education, 2, 115-130. gilbert, j. k. pietrocola, m., zylbersztajn, a. & franco, c. (2000). science and education: notions of reality, theory and model. j. k. gilbert ve c. j. boulter (eds.), in developing models in science education, (pp.19-40). dordrecht: kluwer academic publishers. greca, i., m. & moreira, m. a. (2000). mental models, conceptual models, and modelling. international journal of science education, 22(1), 1-11. günindi, y. (2015). preschool children’s perceptions of the value of affection as seen in their drawings. international electronic journal of elementary education, 7(3), 371-382. a study to determine the mental models in preschool children’s / ahi 347 hatano, g. & inagaki, k. (1997). qualitative changes in intuitive biology. european journal of psychology of education, 12(2), 111-130.
 herrmann, p., waxman, s. r. & medin, d. l. (2010). antropocentrism is not the first step in children’s reasoning about the natural world. pnas, 107(22), 9979-9984. inagaki, k. & hatano, g. (1993). young children’s understanding of the mind-body distinction. child development, 64, 1534-1549. inagaki, k. & hatano, g. (2006). young children’s conception of the biological world. current direction in psychological science, 15(4), 177-181. johnson-laird, p. n. (1983). mental models. cambridge: harvard university. judson, e. (2011). the impact of field trips and family involvement on mental models of the desert environment. international journal of science education, 33(11), 1455-1472. kahriman-öztürk, d., olgan, r. & güler, t. (2012). preschool children’s ideas on sustainable development: how preschool children perceive three pillars of sustainability with the regard to 7r. educational sciences: theory & practice, special issue, 2987-2995. kışlalıoğlu, m. & berkes, f. (2012). çevre ve ekoloji. (13. basım) i̇stanbul: remzi. kurnaz, a., kıldan, a. o. & ahi, b. (2012). mental models of pre-school children regarding the sun, earth and moon. the international journal of social sciences, 7(1), 136-143. leonard, m. (2006). children’s drawings as a methodological tool: reflections on the develop plus system in northern ireland. irish journal of sociology. 15(2), 52-66. li, l., chen, x., van der tol, c., luo, g. & su, z. (2014). growing season net ecosystem co2 exchange of two desert ecosystems with alkaline soils in kazakhstan. ecology and evolution, 4(1), 14-26. littledyke, m. (2004). primary children’s view on science and environmental issues. examples of environmental cognitive and moral development. environmental education research, 10(2), 217-235. liu, s. c. & lin, h. (2014). undergraduate students’ ideas about nature and human-nature relatiopships: an empirical analysis of environmental worldviews. environmental education research, 20(3), 412-429. liu, s. c., & lin, h. (2015). exploring undergraduate students’ mental models of the environment: are they related to environmental affect and behavior? the journal of environmental education, 46(1), 23-40. loughland, t., reid, a., & petocz, p. (2002). young people’s conceptions of environment: a phenomenographic analysis. environmental education research, 8(2), 187-197. mason, h., l. ve langenheim, j., h. (1957). language analysis and the concept “environment”. ecological society of america, 38(2), 325-340. ministry of national education. (2015). national education statics. formal education [adobe digital editions version]. retrieved from http://sgb.meb.gov.tr/ istatistik/ meb_istatistikleri_orgun_egitim_2014_2015.pdf moseley, c, desjean-perrotta, b. & utley, j. (2010). the draw an environment test rubric (daet-r): exploring pre-service teachers’ mental models of the environment. environmental education research, 16(2), 189-208. national fish, wildlife and plants climate adaption strategy. (2012). desert ecosystem. retrieved from http://www.wildlifeadaptationstrategy.gov/pdf/desert_ecosystems_paper.pdf nobes, g., moore, d., g., martin, a., e., clifford, b., r., butterworth, g., panagiotaki, g., & siegal, m. (2003). children’s understanding of the earth in a multicultural community: mental models or fragments of knowledge?. developmental science, 6(1), 72-85. international electronic journal of elementary education vol.8, issue 3, 333-350, 2016 348 noble, i. r. (2014). deserts in a changing climate: impacts (climate change 2014 synthesis report). retrevied from http://www.ipcc.ch/pdf/assessment-report/ar5/syr/syr_ar5_final_full.pdf özsoy, s. & ahi, b. (2014). elementary school students’ perceptions of the future environment through artwork. educational sciences: theory & practice, 14(4), 1-26. özsoy, s., özsoy, g. & kuruyer, g. (2011). turkish pre-service primary school teachers' environmental attitudes: effects of gender and grade level. asia-pacific forum on science learning and teaching, 12(2), 1-21. pallant, j. (2011). spss survival manual. australia: allen & unwin. palmer, j. a. (1994). acquisition of environmental subject knowledge in pre-school children: an international study. children’s environments, 11(3), 204-211. panagiotaki, g., nobes, g. & potton, a. (2009). mental models and other misconceptions in children’s understanding of the earth. journal of experimental child psychology, 104, 52-67. piaget, j. (1970). genetic epistemology. new york: norton & company. piperno, f., di biassi, s., & levi, g. (2007). evaluation of family drawings of physically and sexually abused children. european child & adolescent psychiatry, 16(6), 389-397. prokop, p., kubiatko, m. & fančovičova , j. (2007). why do cocks crow? children’s concepts about birds. research in science education, 37, 393-405. ross, l. e. & nelson, m. n. (1973). the role of awareness in differential conditioning. psychophysiology, 10(1), 91-94. saçkes, m. & korkmaz, h. i̇. (2015). anaokulu çocuklarının dünyanın şekline ilişkin zihinsel modelleri. i̇lköğretim online, 14(2), 734-743. samarapungavan, a. vosniadou, s. & brewer, w., f. (1996). mental model of the earth, sun and moon: indian children’ s comologies. cognitive development, 11, 491-521. schafer, n. (2012). finding ways to do research on, with and for children and young people. geography, 97(3), 147-154. shekhawat, n. s., phulwaria, m., harish, rai, m. k., kataria, v., shekhawat, s., gupta, a. k., rathore, n. s., vyas, m., rathore, n., vibha, j. b., choudhary, s. k., patel, a. k., lodha, d. & modi, r. (2012). bioresearchers of fragile ecosystem/desert. proc. natl. acad. sci., india, sect. b biol. sci., 82(2), 319-334. shepardson, d. p. (2002). bugs, butterflies, and spiders: children’s understanding about insects. international journal of science education, 24(6), 627–643. shepardson, d. p., wee, b., priddy, m. & harbor, j. (2007). students’ mental models of the environment. journal of research in science teaching, 44(2), 327-348. straatemeier, m., van der maas, h. l. j. ve jansen, b. r. j. (2008). children’s knowledge of the earth: a new methodological and statistical approach. journal of experimental children psychology, 100, 276-296. strommen, e. (1995). lions and tigers and bears, oh my! children’s conceptions on forests and their inhabitants. journal of research in science teaching, 32(7), 683–689. sungur, s., tekkaya, c. & geban, ö. (2001). the contribution of conceptual change texts accompanied by concept mapping to students’ understanding of the human circulatory system. school science and mathematics, 101(2), 91-101. united nations educational, scientific and cultural organizationunited nations environment programme. (1975). the international workshop on environmental education. belgrade final report. paris: unesco. united states global change research program. (2009). global climate impacts in the united states. (eds. t.r. karl, j.m. melillo, and t.c. peterson). cambridge university press. a study to determine the mental models in preschool children’s / ahi 349 vosniadou, s. & brewer, w. f. (1992). mental models of the earth: a study of conceptual change in childhood. cognitive psychology, 24, 535-585. vosniadou, s. & brewer, w. f. (1994). mental models of the day/night cycle. cognitive science, 18, 123-183. vygotsky, l. s. (1971). the psychology of art. cambridge: the mit. wiersma, w and jurs, s. g. (2008). research methods in education: an introduction. (9th edition). boston: pearson. world resource institue. (2005). ecosystems and human well-being. wasginton dc. wri press. retrevied from http://www.millenniumassessment.org/documents/document.355.aspx.pdf yardımcı, e. & bağcı kılıç, g. (2010). çocukların gözünden çevre ve çevre sorunları. elementary education online, 9(3), 1122-1136. yavuzer, h. (2010). resimleriyle çocuğu tanıma. (10. baskı). i̇stanbul: remzi. international electronic journal of elementary education vol.8, issue 3, 333-350, 2016 350 www.iejee.com this page is intentionally left blank http://www.iejee.com/ microsoft word iejee_4_3_427_440 international electronic journal of elementary education, 2012, 4(3), 427-440. issn:1307-9298 copyright © iejee www.iejee.com the effect of curriculum for developing efficient studying skills on academic achievements and studying skills of learners semra demi̇r∗∗∗∗ erciyes university, turkey mehmet kilinc ministry of national education, turkey ali dogan ministry of national education, turkey received: october, 2011 / revised: april, 2012 / accepted: june, 2012 abstract purpose of this study is to examine the effect of “development of efficient studying skills curriculum” on academic achievements and studying skills of 7th grade primary school students. in this study, pretest-post-test from experiment models and semi-experimental model with control group were preferred. the reason for the preference is our wish to make a comparison between the group on which curriculum was implemented (experiment group), and the group on which curriculum was not implemented (control group) in terms of academic achievement, and acquiring efficient studying skills. study population of this research covers 7th grade students from refika küçükçalık primary school in kocasinan district of kayseri which is located in the middle of turkey during 2011-2012 academic year (8 units, 320 students). sample of the study was determined according to purposive sampling which is one of non-probability sampling types. obtained data were analysed employing covariance analysis (ancova). as a result, this research indicated that students can acquire efficient studying skills by means of curriculum for developing efficient studying skills and they increase their academic achievements thanks to these studying habits. in this sense, if quality of education is desired to be increased, students with high level of academic achievements are intended and growing youth is expected to compete with the young population of other states with the effect of globalization, it is necessary to make students acquire efficient studying skills. keywords: study skills, curriculum development, academic achievements ∗ semra demir, , erciyes university, faculty of education, department of educational sciences, 38030 kayseri / turkey. e-mail: sdemir@erciyes.edu.tr &. phone: +90 3522076600 / 37267 fax: +90 352 437 88 34 gsm: 05336300878 e-mail: sdemir@erciys.edu.tr international electronic journal of elementary education, 2012, vol.4, issue 3, 427-440 428 introduction learning takes place through experiences influencing psychological functions which lead to differences in behaviours. there are five elements in learning which are learner, learning, learnt, teacher, and learning environment. taking these five elements in terms of influencing learning shows that last two elements do not affect learning directly but they have an indirect effect since they influence other elements. thus, elements influencing learning can be approached in three different groups which are factors regarding the learner, factors regarding the learning materiall, and factors regarding the learning method (bacanlı, 2005). these define the quantity and quality of learning by interacting with each other. while factors regarding the learner can be compiled as level of readiness for learning, age, intelligence, motivation level, interests and skills, characteristics, cognitive styles, and learning styles; perceptual distinguishability, semantic connotation, and conceptual grouping are related to learning material. on the other hand, structure of the subject to be learnt, the duration of time allocated for learning, feedback, and active participation are the factors associated with learning method (senemoğlu, 2005). in the past, people used to think that efficient learning depends mainly on teacher’s teaching methods. however, today it is a commonly accepted fact that permanent and significant learning can be achieved only through learner efforts and contributions (benson, 2001). from this point of view, learners can catch up with the intended achievement standards only by perceiving the given information accurately, analysing, and internalizing them. this process is briefly called studying (bacanlı, 2005). studying behaviours are skills acquired during school years and they are difficult to change in the later periods of life. the most important factor influencing learner’s achievements possesses the skills necessary for efficient studying (küçükahmet, 2002). endowing learners with efficient studying skills means both making them study deliberatively and understand their courses and aiding them to succeed in their exams. learners studying properly value their skills and abilities in the best way. learners without efficient studying skills have low level of achievement both in schools and professional lives since they cannot get the reward in return to the effort and time spent for learning (yılmaz, 1987; teker, 2002). learners without efficient studying skills or learners with wrong studying skills are considered as one of the most basic reasons for failure in schools (türkcan & öcal, 2003). yeşilyaprak (2000) and türkoğlu et al., (1996) expressed that many students cannot succeed though they spend loads of time for studying. the motive behind this fact was suggested as not having the efficient studying skills. in addition, it was emphasized that even students who can be considered as successful do not possess adequate level of efficient studying skills. students who cannot improve efficient studying and learning skills can be incapable of motivating themselves for learning, planning their learning processes and evaluating them as the grade goes up. this situation negatively influences their school achievements and therefore their life after school (yıldırım, doğanay & türkoğlu, 2000). studying skills are influenced by learner motivation and their full and accurate use of time. besides, learners’ wish to study or not, difficulties in studying conditions, resistance against frustration and showing intimidation when faced with failure are factors influencing learning as well (fidan, 1996). having unsuccessful learners at the end of education process leads to considerable amount of waste in investments made for education as well as supporting the worry about not being able to raise human force that society demands. it is known that learner failures are not limited with educational life but have much more wide effect. currently, it is accepted that student failures is a problem growing each day. it is expressed that one of the biggest the effect of curriculum for developing efficient studying skills / demir, kılınc & dogan 429 underlying reasons for student failures is inadequacy of studying skills and attitudes (küçükahmet, 2002). there are many various definitions for studying skills in literature; however, studying skills are generally defined as “using specific methods efficiently to learn something” (thomas, 1993; uluğ, 2000; yıldırım, doğanay & türkoğlu, 2000). studying skills are generally examined under the titles of planned studying, organizing study environment, efficient reading, listening lecture, note-taking, efficient writing expression, and doing homework (dodge, 1994; smith, 2000; thomas, 1993; uluğ, 2000; yeşilyaprak, 2000; yıldırım, doğanay & türkoğlu, 2000). although these skills are independently important, they are not independent from each other. strategies in relation to these skills are supportive of each other and high level of success can only be achieved by using all of them responsively together. (crow, 1968; gettinger & seibert, 2002; uluğ, 2000; yıldırım, doğanay & türkoğlu, 2000). of studying skills, planned studying means having a purpose and this prevents students from disarrangement (demir, 2011). setting the period for studying is characterized by paying attention to studying courses at times close to the day and time of course lectures, making daily, weekly or monthly plans, providing flexibility to shift studying periods in case of sudden obstacles, defining durations in accordance with the course characteristics and knowing for certain what to study beforehand. the concept of studying environment covers the environment both in and outside of school where learners continue their activities regarding the school. preparing for studying can be expressed as organizing the place in such a way that it facilitates studying in terms of environment and the adopting the atmosphere suitable to read both physically and psychologically in terms of person (baltaş, 1998). skills for reading and making sense of what you read, which have a place in educational process, are two of the most important elements contributing meaning to people’s lives. reading “broadens and deepens one’s horizon by teaching new words, gaining new comprehensions, creating dreams and improving creativity” (akyol, 2006, p. 29; binbaşıoğlu, 1995). this process is influenced by past experiences of the reader, ability to use language, skill for making meaning of unknown words, and purpose of reading (yüksel, 1997). efficient listening is not only hearing what is being said. it is an active process intended for spending effort to consider what is said as important, comprehend, and evaluate (türkoğlu, doğanay & yıldırım, 1996). the concept of understanding is mainly related to listening skill. listening skill, which is one of the basics in the development of native language skills, means “understanding, interpreting, evaluating, and organizing the suggested thoughts in the speech as well as detecting the relations between them and assorting the ones significant enough to store in the mind” (taşer, 1996, p. 214). 42% of human communication covers listening and students listen both each other and their teacher in school for about 2, 5-4 hours. these results indicate that there is a strict relationship between learning in school and listening (göğüş, 1978, p. 227). yalçın, (2002, p. 45) claims that learners should be given a systematic listening education starting from the level they start to understand their teacher. note-taking is the process of writing the summary of information regarding a subject on a proper place to be used in the future. note-taking and benefiting from the taken notes have a distinct and special importance from other learning materials in terms of efficient learning. this enables learners to think actively about the lectured subject and convey the information s/he internalized in an appropriate form on the paper. international electronic journal of elementary education, 2012, vol.4, issue 3, 427-440 430 from this aspect, note-taking increases learner concentration for listening the course. similarly, since the information is processed and interpreted while reading, note-taking increases the efficiency of individual study. thus, note-taking is of great importance in terms of both lecture and individual study. notes taken during the courses aid students in determining what was important in the lecture (ertuğrul, 2004, pp. 116-117). santangelo et al. (2007) stated that writing is extremely powerful means for communication since it creates a bound between times and distances, collects, and protects information, provides information about newly organized or spreading subjects, and creates a flexible environment for expressing yourself in terms of art, spirit and politics. emphasizing that writing is a skill inseparable with communication and handled in educational activities, temur (2011) denoted that past experiences are one of the elements influencing the efficiency of writing process. temur (2011) said that some of the primary school beginners acquire important skills regarding reading and writing (vocal awareness, writing awareness) while some of them cannot. active participation in lectures is possible only through active learning. active learning is a process in which learner bears the responsibility of learning process, learner is granted with the opportunity to make decisions and self-arrangements in relation to various aspects of learning process and learner is forced to use his/her mental abilities with various educational tasks. theoretical background of active learning is based on constructivism. homework is one of the teaching methods outside of the school. it is also one of the most efficient instruments in giving importance, and meaning for old learning experiences, and acquiring new learning experiences (tan & erdoğan, 2004). salend and schliff (1998, p.398) consider homeworks as one of the methods for teaching and skill development. they explain the reasons for giving homework as follows; facilitating learning by making applications, individualizing learning for both slow learners and advanced learners, completing the subjects which are left unfinished in schools outside of school time, making learners acquire independent studying habits, and delivering what was taught in class to parents. the fact that there is a difference between achievement levels of learners who acquired necessary methods, and habits for an efficient study, and learners who either possess none or limited amount of those habits was supported by researches. researches regarding this topic revealed that students lack in terms of studying skills (bay, tuğluk & gençdoğan, 2005; kesiktaş, 2006; koşar, 2004; küçükahmet, 1987; uluğ, 1981; yıldırım et al., 2000) and indicated that a studying skills-aware education increases academic achievement, and self-confidence (gall et al, 1990). additionally, knowing these skills and using them in this process make important contributions to shortening the period for learning, saving the information, and developing skills for using (gall et al, 1990). besides, researches proved that there is a positive relationship between learner studying skills and academic achievements (eliot, godshall, shrout & withy; 1990; bay, tuğluk, gençdoğan, 2005). benjamin (1991) searched studying strategies of successful students to detect whether they are different from failed students in terms of quality in processing their knowledge. findings indicated that successful students are more active, purposeful and flexible in using their strategies and more satisfied with their academic performances (eg., tümkaya & bal, 2006). to gall (1990), today many educators accept that “teaching learning” and therefore “developing studying skills” are as much important as teaching traditional courses (such as mathematics and turkish). for instance; studying skills has become one of the courses either for primary or secondary school level in many states of usa. or studying skills are being taught to learners in the form of seminars. as for higher education, studying skills has started to be given as a course in undergraduate programmes since 1920s. these courses became the effect of curriculum for developing efficient studying skills / demir, kılınc & dogan 431 widespread since 1960s in many universities and some of the universities included it as a compulsory course. also, private sector put too much stress on studying skills and studying skills seminars by private consultancy institutions regarding this sector are becoming widespread (türkoğlu, doğanay & yıldırım 1996, p. 2-3). studying skills, which is given as a course in different grades in various states, is not a course covered in turkish curriculum yet. there is also limited number of research made on this matter (bay et al, 2005; koşar, 2004). this indicates the lack of an important database regarding this matter in turkey and the importance of this paper. in addition, it expected from the findings of this study to shed a light upon the authorities of ministries, school managers, teachers and parents. purpose of this study is to examine the effect of “development of efficient studying skills curriculum” on academic achievements and studying skills of 7th grade primary school students. questions below were tried to be answered to achieve this purpose: 1is there a significant difference between post-test scores of students from experiment and control groups which they got from the entire academic achievement test? 2is there a significant difference between post-test scores of students from experiment and control groups which they got from the entire efficient studying skills habit inventory? method research model in this study, pre-test-post-test from experiment models and semi-experimental model with control group were preferred. the reason for the preference is our wish to make a comparison between the group on which curriculum was implemented (experiment group), and the group on which curriculum was not implemented (control group) in terms of academic achievement, and acquiring efficient studying skills. this experimental pattern enables making such comparisons (karasar, 2006, p. 110).two of the present groups are appointed randomly to operation groups in semi-experimental pattern without random appointment (fraenkel & wallen, 2000, p. 283). in this way, examinees are similarly matched with each other as much as possible. the fact that central education is being applied in turkey, where research took place, and classes are formed by school managers played a role in making this preference. that is because in turkey, it is not possible for a researcher to form his/her own classes and use random appointment while making this process with specific number of students in classes for it is against the education system in turkey. therefore, of these two classes which were formed by school management at the same grade, one was selected as experiment and the other was selected as control group. first hypothesis of the research is “there is a significant difference between post-test scores of students from experiment and control groups which they got from the entire academic achievement test”, and the second hypothesis is “there is a significant difference between post-test scores of students from experiment and control groups which they got from the entire efficient studying skills habit inventory”. population and sample there are two populations emphasized in scientific researches. one of these is general population which covers whole units wanted to be generalized with a large scope. study population is the one which is available. the population about which the researcher will make comment is the study population. practically, researchers are made with study international electronic journal of elementary education, 2012, vol.4, issue 3, 427-440 432 population. thus, it is inevitable to generalize the results to this population (karasar, 2006). study population of this research covers 7th grade students from refika küçükçalık primary school in kocasinan district of kayseri which is located in the middle of turkey during 20112012 academic year (8 units, 320 students). sample of the study was determined according to purposive sampling which is one of non-probability sampling types. in purposive sampling, which is also called as judgement sampling, researcher uses his/her own judgement about whom to select and includes those who are most appropriate to the purpose of the research (balcı, 2004: 90). taking these principles into consideration, 7-f class of the above mentioned school was accepted as experiment group while 7-b class was accepted as control group. some of the qualities included in independent variables of experiment and control groups are as follows: 51,51% of students are female and 48,48% is male. 7,57% of students is single child, 27,27% of students has one sibling apart from himself, 42,42% has two siblings apart from himself, 18,18% has three siblings apart from himself and 4,54% has four or more siblings apart from himself. the rate of students whose mothers are primary school graduates is 45,45%, the rate of students whose mothers are secondary school graduates is 27,27%, the rate of students whose mothers are high school graduates is 24,24% and the rate of students whose mothers are university graduates is 3,03%. the rate of students whose fathers’ education level is primary is 18,18%, the rate of students whose fathers’ education level is secondary is 31,81%, the rate students whose fathers’ education level is high school is 33,33% and the rate of students whose fathers’ education level is university is 16,66%. 6,06% of students are from low, 77,27% is from medium and 16,66% is from high level income families. 16,66% is living with extended family and 83,33% is living with a nuclear family. besides, while 28,78% of experimental objects is attending training centre, 71,21% is not. data collection instruments as data collection instruments, “studying habits inventory” developed by (1) uluğ (1981) and achievement tests for 7th grade students of refika küçükçalık primary school (1st pilot and 2nd pilot exams) (2) were employed. there are explanations below in relation to them. (1)studying habits inventory: studying habits inventory consists of 60 questions in total. first 50 questions of these are related to intended habits (starting studying, making plans, note-taking, summarizing, preparing for exam, studying individually, and in groups, organizing studying environment, efficient reading, and listening). last 10 questions in inventory were prepared as control items to check the accuracy of the answers. reliability of studying habits inventory was defined as 0.79 correlation coefficient which was obtained as a result of test-retest method applied on experiment group containing 16 people in 1981. (2)achievement tests for 7th grade students of refika küçükçalık primary school: achievement tests are tests which are periodically applied in primary schools by ministry of national education to prepare students for placement tests which are implemented for placing students in high schools and to test student achievements. in this study, first and second achievement tests from the abovementioned tests were employed. the effect of curriculum for developing efficient studying skills / demir, kılınc & dogan 433 collecting data table 1. data collecting process as seen in table 1, students were applied studying habits inventory and academic achievement test for 7th grade students of refika karaçalık primary school at the beginning of experimental operation. afterwards, students in experimental group were applied curriculum for developing efficient studying skills. curriculum for developing efficient studying skills was completed in four weeks with two applications in each week which made 8 applications in total. meanwhile, control group students were not put subject to any operation. after the implementation of curriculum for developing efficient studying skills, studying habits inventory and achievement test for 7th grade students of refika karaçalık primary school were applied as post-test and the results were compared. curriculum for developing efficient studying skills: curriculum for developing efficient studying skills, which was planned to support development of students’ efficient studying skills, was prepared in accordance with the systematic below. initially, target for curriculum were determined. related literature was taken into consideration while determining the targets. afterwards, scope of curriculum for developing efficient studying skills was set. setting of the scope was based on review of related literature. at this stage, the place where education will take place, implementer and experimental objects were determined. the scope was formed in such a way that it will aid acquisition of the targets. also, it is appropriate for different learning and teaching activities, and reasoning. at the beginning of each activity, student opinions regarding that day’s topic were taken and they were informed about the subject. students were asked to discuss lectured subject with the group and to compare their old knowledge with the new information they had learnt. they were made to conduct reasoning by presenting various sample events. curriculum for developing efficient studying skills employed lecturing, question-answer, discussion, sample event, and brainstorming methods during the education process. white board, photocopy notes, projection machine, and related slide shows were utilized as learning material. targets of curriculum for developing efficient studying skills are as follows: (1) knowledge of determining target. (2) organizing the study environment and starting to study. (3) conceiving the obstacles against efficient studying. (4) conceiving the steps of planning time. (5) conceiving the steps of efficient reading. (6) conceiving the steps of efficient listening. (7) proper note-taking and doing homework. (8) comprehending the importance of repeating. (9) being able to evaluate curriculum for developing efficient studying skills. as for the scope of the curriculum it contains; (1) determining target. (2) organizing the study groups pre-test process post-test experiment group studying habits inventory 1st academic achievement test (pilot test) implementing efficient studying skills curriculum for 4 weeks studying habits inventory 2nd academic achievement test (2nd pilot exam) control group studying habits inventory 1st academic achievement test (pilot test) studying habits inventory 2nd academic achievement test (2nd pilot exam) international electronic journal of elementary education, 2012, vol.4, issue 3, 427-440 434 environment. (3) starting to study. (4) planning. (5) efficient reading. (5) efficient listening. (6) note-taking. (7) repeating. (8) evaluating the curriculum. data analysis obtained data were analysed employing covariance analysis (ancova). “ancova is used to determine whether there is a statistically significant difference between groups” (büyüköztürk, 2007: 47-48). therefore, “if the researcher is focused on whether the experimental operation was effective or not in a pattern with pre-test/post-test control group, the most appropriate statistical operation is the single factor ancova in which pretest is considered as the common variable” (büyüköztürk, 2004, s. 106). due to this quality of ancova, data of this research were analysed using this method. at least .05 level of significance was taken as basis in these analyses. findings and commentary is there a significant difference between post-test scores of students from experiment and control groups which they got from the entire academic achievement test? this sub-problem, which was taken as covariant that is changing with pre-tests, was tested via covariance analysis (ancova). to this end, arithmetic average of pre-test and post-test scores of students from experiment and control groups which they got from “the exam for 7th grades implemented by refika küçükçalık primary school” were corrected according to pretest with standard deviation and arithmetic average values regarding post-test were determined. in this sense, pre-test arithmetic average of experiment group is 217,0 and standard deviation is 46 while arithmetic average of control group for pre-test is 217,3 and standard deviation is 46,6. post-test arithmetic average of experiment group is 228,2 and standard deviation is 38,8 while arithmetic average of control group for post-test is 217,6 and standard deviation is 39,6. arithmetic average of experiment group for post-test which was corrected according to pre-test scores is 228,40 while this score was calculated as 217,51 for control group. this data show that post-test average scores of experiment group which were corrected according to pre-test named “the exam for 7th grades implemented by refika karaçalık primary school” are higher than those of control group. ancova test was applied in order to reveal how to interpret this difference between groups statistically. results of this test are given in table 2. table 2. ancova analysis results applied for entire academic achievement test source kt sd ko f p model 89971,429 2 44985,714 268,303 ,000 covariant 88096,356 1 88096,356 525,42 ,000 group 1956,165 1 1956,165 11,667 ,001 error 10563,074 63 167,668 total 100534,502 66 when table 2 is examined, it is seen that there is a significant difference between post-test academic achievement score averages of groups which were corrected according to re-test academic achievement test (f(1-63)=11.667, p <.05). ancova model defining this is significant [f(2;63)= 268,303, p <.05]. this finding indicates that academic achievement test scores of students change significantly depending on the implemented experimental operation. the effect of curriculum for developing efficient studying skills / demir, kılınc & dogan 435 is there a significant difference between post-test scores of students from experiment and control groups which they got from the entire efficient studying skills habit inventory? this sub-problem, which was taken as covariant that is changing with pre-tests, was tested via covariance analysis (ancova). to this end, pre-test and post-test arithmetic average of students from experiment and control group which they got from “studying habits inventory” were corrected according to pre-test with standard deviation and arithmetic average values regarding post-test were determined. in this sense, pre-test arithmetic average of experiment group is 28,3 and standard deviation is 4,8 while pre-test arithmetic average of control group is 27,3 and standard deviation is 7,6. post-test arithmetic average of experiment group is 34,3 and standard deviation is 5,4 while post-test arithmetic average of control group is 28,6 and standard deviation is 6,3. post-test arithmetic average of experiment group which were corrected according to pre-test scores is 34,2 while this score is 28,7 for control group. this data shows that experiment group has higher post-test average scores which were corrected to pre-test named as “studying habits inventory” than control group. ancova test was implemented to see if this difference between groups is statistically significant. results of the related test are given in table 3. table 3. ancova analysis results implemented for entire efficient studying habits inventory source kt sd ko f p model 564,710 2 282,355 8,064 ,001 covariant 40,528 1 40,528 1,158 ,286 group 497,307 1 497,307 14,204 ,000 error 2205,775 63 35,012 total 2770,485 65 when table 3 is examined, it is seen that there is a significant difference between groups in terms of post-test score averages they got from efficient studying habits inventory which were corrected according to pre-test efficient studying inventory scores of groups [f(163)=14.204, p <.05]. ancova model defining this is significant [f(2;63)= 8,064, p <.05]. this finding indicates that student scores regarding efficient studying habits inventory change significantly according to the applied experimental operation. conclusion and discussion in relation to first sub-question of research, a significant difference on behalf of experiment group was detected between academic achievement levels of experiment and control group according to analysis results obtained from achievement test scores. as a result of comparison made between post-test scores of experiment and control group, first hypothesis of the research which was “there is a significant difference on behalf of experiment group between academic achievements of experiment and control groups” was confirmed. that post-test scores of experiment group are significantly different from control group is an indicator of the fact that curriculum for developing efficient studying skills was effective on increasing 7th grade students’ achievements. that teaching studying skills increases academic achievements of students was stated in different sources and previous research findings supported this claim were taken into consideration while creating the first hypothesis of the research. when related literature was reviewed; bol, warkentin, nunnery and o’connel (1999) detected a positive relationship between college studying habits survey and achievements of college students. gonzales stated that there is a medium or high level of relationship between spanish origin university students’ studying habits and their grade point averages (gonzales,1984, p. 2). özbey (2007) international electronic journal of elementary education, 2012, vol.4, issue 3, 427-440 436 revealed that secondary school students with high level of mathematics achievements are more conscious than other students with lower level of mathematics achievement (özbey, 2007, p. 97). subaşı (2000) stated in his research that education regarding efficient studying habits increases students’ academic self-concept levels and academic achievements. jegede and jegede (1997) denoted in their study that studying habits and academic motivation are related to each other positively. these results show that students’ academic achievements increase as their efficient studying habits increase. in relation to second sub-problem of the research, it was concluded that there is a significant difference between post-test studying habit inventory score averages which were corrected according to pre-test studying habits inventory scores of groups. this result expresses that there is a significant change in students’ studying habits inventory scores depending on the applied experimental operation. this case confirmed the hypothesis that “there is a significant difference on behalf of experiment group in terms of efficient studying habits of experiment group and control group”. these results can be evaluated as such; curriculum for developing efficient studying skills enable students organize the study environment and use some specific methods effectively such as efficient reading, listening lectures, note-taking, efficient writing and doing homework. in this sense, it is possible to say that it helps increasing student motivations. second hypothesis of research was created taking into account the fact that various sources state that students do not possess efficient studying skills adequately. various research findings support this case (fletcher, 1980; uluğ, 1981; subaşı, 2000; kaya, 2001). abovementioned researchers revealed that education, seminars and guidance regarding improvement of studying habits positively influence studying habits and attitudes. besides, other findings obtained as a result of literature review showed that students do not use proper studying habits, more importantly; they fail since they cannot demonstrate studying behaviours (yenilmez & özbey 2007). in a research conducted by zeyrek et al (1990), it was revealed that students are moderate in relation to the wish to study (eg., memiş 2007). another point to be emphasized is that 55% of students state that they consider the factor “i fail because i do not know how to study” as an important determinant for their failures (olcay & döş, 2009). derviş (1993) in his study examining the effect of group work on student studies came to the conclusion that group work positively influence achievement and it makes students acquire the habit of planned studying, self-confidence and sense of responsibility. in addition, it was revealed by various researches that students with proper studying skills and habits are academically successful as well (uluğ, 1981; schultz, 1989; elliot et al, 1990; slate et al, 1990; jones et al, 1993; lawler, slate & jones, 1993; agnew et al, 1993; gordon, 1997; ley & young, 1998; sünbül et al, 1998; carter, 1999; arslantaş, 2001; memiş, 2007, özsoy, memiş & temur, 2009). as a result, this research indicated that students can acquire efficient studying skills by means of curriculum for developing efficient studying skills and they increase their academic achievements thanks to these studying habits. in this sense, if quality of education is desired to be increased, students with high level of academic achievements are intended and growing youth is expected to compete with the young population of other states with the effect of globalization, it is necessary to make students acquire efficient studying skills. to this end, legal regulations can be made for the preparation of curriculum for developing efficient studying skills. teachers of all grades can be given seminars regarding the development of efficient studying skills. in departments educating teachers, information concerning the how to teach efficient studying skills can be covered by pedagogical formation courses. besides, activities in relation to implementation of this curriculum can be the effect of curriculum for developing efficient studying skills / demir, kılınc & dogan 437 practiced in practical courses. in addition, repeating this study with different groups, different content and different educational approaches may be beneficial as well. . . . references agnew, n. c., slate, j. r., jones, c. h., & agnew, d. m. (1993). academic behaviors as a function of academic achievement, locus of control, & motivational orientation.nacta journal, 37(2), 24-27. akyol, h. (2006). türkçe öğretim yöntemleri. ankara: kök. altunışık, r., çoşkun, r., bayraktaroğlu, s. ve yıldırım, e. (2007). sosyal bilimlerde araştırma yöntemleri spss uygulamalı. sakarya: sakarya. arslantaş, m. (2001). orta öğretim öğrencilerinin ders çalışma alışkanlıkları non published master thesis. dicle üniversitesi sosyal bilimler enstitüsü, diyarbakır. atılgan, m. (1998)."üniversite öğrencilerinin ders çalışma alışkanlıkları ile akademik başarılarının karşılaştırılması", non published master thesis. gaziantep üniversitesi). bacanlı, h. (2005). “gelişim ve öğrenme”, ankara:nobel. balcı, a. (2004). sosyal bilimlerde araştırma yöntem, teknik ve i̇lkeleri. ankara: pegema. baltaş, acar, (1998). öğrenme ve sınavlarda üstün başarı, 15. basım, i̇stanbul: remzi. bay, e., tuğluk, m.n., gençdoğan, b. (2005) üniversite öğrencilerinin ders çalışma becerilerinin i̇ncelenmesi (kâzım karabekir eğitim fakültesi örneği), elektronik sosyal bilimler dergisi. 4, 14: 94105. benson, p. (2001) teaching and researching autonomy in language learning. pearson education limited, england. binbaşıoğlu, c. (1995). “okuma eylemi, okumak ne yapmak demektir?”. çağdaş eğitim, 23:210, s.42-48. bol, l. , warkentin, r. , nunnery, j. a. & o’connel, a.(1999). college student activities and their relationships to study context reffrence course and achievement. college student journal, (vol,33).[online]: retrieved from 10-06-2012: http://findarticles.com/p/articles/mi_m0fcr/is_4_33/ai_62839476/ büyüköztürk, ş. (2004). sosyal bilimler i̇çin veri analizi el kitabı (4. baskı). ankara: pegem a. büyüköztürk, ş. (2007). deneysel desenler ön test-son test kontrol grubu desen ve veri analizi (2. baskı). ankara: pegema. carter, d. d. g. (1999). the relationship of study habits, attitude, and motivation to academic achievement in a selected course of study at an historically black university. morgan state university, the faculty of the school of graduate studies and research, umi number: 9945902. crow, a. (1968). educational psychology: an outline with questions and answers. totowa: a littlefield, adams & co. demir, s. (2011). öğretim etkinliklerinin planlanması. sınıfta etkili öğretim ve yönetim (etkinlikler ve örnekler), (ed: dilek erbaş), ankara:data. derviş, n. (1993). grup çalışmalarının öğrenci başarısına i̇lişkin öğrenci ve öğretmen görüşleri, non published master thesis. hacettepe üniversitesi. ankara. dodge, j. (1994). the study skillshhandbook: more than 75 strategies for better learning new york: scholastic inc. international electronic journal of elementary education, 2012, vol.4, issue 3, 427-440 438 elliot, t., godshall, f., shrout, j. and withy, t. (1990) problem-solving appraisal, self-reported study habits and performance of academically at-risk college students. journal of counseling psychology, 37, 203-207. ertuğrul, h.(2004). öğrencinin başarı kılavuzu, i̇stanbul: nesil. fidan, n. (1996). okulda öğrenme ve öğretme. ankara, alkım. fletcher l. (1980). remedial study skills, study habits program, annual meeting of the massachusetts reading association, 11th, worcester, ma, april 10-11. fraenkel, j. r. & wallen, n. e. (2000). how to desing and evaluate research ineducation. mcgraw-hill. gall, m.d., gall, j.p., jacobsen, d.r. & bullock, t.l (1990). tools for learning: a guide to teaching study skills. alexandria, va: ascd. gettinger, m., & seibert, j. k. (2002). contributions of study skills to academic competence. school psychology review, 31(3), 350-365. gonzales, t. m. (1984). locus of control and study habits-attitudes scales as predictor of academic achievement of specially admitted (eop) hispanic university students. university of washington usa. gordon, d. (1997). the relationships among academic self concept, academic achievement, and persistence with self attribution, study habits, and perceived school environment. the faculty of purdue university, doctoral thesis, umi number:9818955. göğüş, b. (1978). türkçe ve yazın eğitimi, ankara: gül. jegede, j.o. & jegede, r.t. (1997). effects of achievement motivation and study habits on nigerian secondary school students’ academic performance. the journal of psychology. (september, 131).[online]: retrieved on 10-06-2012: http://web.ebscohost.com/ehost/pdf?vid=14&hid=120&sid= 9550dcad2c4c-4a9e-8a4bdf96cf5b05c6%40sessionmgr102. jones, c.h., slate, j.r., marini, i. and dewater, b.k. (1993), academic skills and attitudes toward intelligence', journal of college student development, 34(6), 422-4. karasar, n. (2006). bilimsel araştırma yöntemleri, 16. baskı. ankara: nobel. kaya, m. (2001). lise 1. sınıf öğrencilerine verimli ders çalımsa alışkanlıkları kazandırmada grup rehberliğinin etkisi, malatya. i̇nönü üniversitesi sosyal bilimler enstitüsü, non published master thesis. kesiktaş, a.dolunay. (2006). ders çalışma becerileri ve özel gereksinimli öğrenciler. ankara üniversitesi eğitim bilimleri fakültesi yayınları. özel eğitim dergisi. 2006, 7(1) 37-48. koşar, e. (2004) uludağ üniversitesi, “eğitim fakültesi öğrencilerinin çalışma tutum ve alışkanlıkları üzerine bir i̇nceleme.” 12. eğitim bilimleri kongresi, bildiriler kitabı. 1, 165-182. küçükahmet, l. (1987). öğrencilerin çalışma alışkanlıkları ve tutumları, üniversite öğrencileri üzerine bir araştırma. ankara: ankara üniversitesi eğitim bilimleri fakültesi yayınları. küçükahmet, l. (2002). öğretimde planlama ve değerlendirme. ankara: nobel. lawler-prince, d., slate, j. r., & jones, c. h. (1993). academic behaviors of preservice elementary and/or special education teachers: a preliminary study. louisiana education research journal, 18, 109-118. ley, k. & young, d. b. (1998). self regulation behaviours in underprepared (developmental) and regular admission college students. contemporary educational psychology, 23, 42-64. memiş, (demiroglu) a. (2007). öğrencilerin çalışma oryantasyonlarını etkileyen demografik faktörler. türk eğitim bilimleri dergisi, bahar, 5(2), 291-321. neuman, w. l. (2008). toplumsal araştırma yöntemleri nitel ve nicel yaklaşımlar i-ii. (s. özge, çev.). i̇stanbul: yayınodası. the effect of curriculum for developing efficient studying skills / demir, kılınc & dogan 439 olcay, a. ve döş, i̇. (2009).ortaöğretimde başarıyı olumsuz etkileyen unsurların öğrenci boyutuyla tespitine yönelik bir uygulama. gaziantep üniversitesi sosyal bilimler dergisi 8(1):131 -155. özbey, n.(2007). i̇lköğretim öğrencilerinin ders çalışma alışkanlıklarının bazı değişkenler açısından i̇ncelenmesi. non published master thesis. eskişehir osmangazi üniversitesi fen bilimleri enstitüsü: eskişehir. özsoy, g., memiş, a., ve temur, t. (2009). metacognition, study habits and attitudes, international electronic journal of elementary education, 2(1), 154-166. robertson, k, a. (1999). etkili dinleme. (çev.: e. sabri yarmalı). i̇stanbul: hayat yayınları. salend, s. j. ve j. schliff. (1988). the many dimension of home work. academic threapy. 23. 397-403. santangelo, t., karen, r. h., graham, s. (2007). “self-regulated strategy development: a validated model to support students who struggle with writing”, learning disabilities: a contemporary journal, 5, 1-20. schultz, r. a. (1989). differences between academically successfull and unsuccessfull students in an intrusive academic advising program, the graduate college of the oklahoma state university, the degree of doctor of philosophy, umi number:9019517. selçuk, z. (1997). eğitim psikolojisi gelişim ve öğrenme. ankara: pegem yayınları. senemoğlu, n. (2005). “gelişim öğrenme ve öğretim”, ankara: gazi. slate, j. r., jones, c. h., & charlesworth, j. r. (1990). relationship of conceptions of intelligence to preferred teaching behaviors. action in teacher education, 12, 25-29. subaşı, g. (2000).“verimli ders çalışma alışkanlıkları eğitiminin akademik başarı, akademik benlik kavramı ve çalışma alışkanlıklarına etkisi”, eğitim ve bilim, 4, 117. sünbül, a. m., tüfekçi, s., kocaman, y., arı, m. & karagözlü, m. (1998). üniversite öğrencilerinin çalışma alışkanlıklarının bazı değişkenler açısından i̇ncelenmesi. 7. eğitim bilimleri kongresi, 1. cilt. smith, c. b. (2000). reading to learn: how to study as you read. bloomington: eric clearinghouse on reading, english and communication, indiana university. taşer, s. (1996). konuşma eğitimi, i̇stanbul: papirus yayınları. tan, ş. ve a. erdoğan. (2004). öğretimde planlama ve değerlendirme. ankara: pegem yayıncılık. temur, t. (2011). i̇lköğretim 1. sınıf öğrencilerinin kalem tutma şekilleri ile kavrama ve sıkıştırma kuvvetlerinin betimlenmesi, kuram ve uygulamada eğitim bilimleri, educational sciences theory and practice, 11(4) güz. teker, n. (2002). uzaktan eğitim öğrencilerinin ders çalışma stratejilerinin karşılaştırılması, eğitim bilimleri ve uygulama, 1(1), 49-66. thomas, a. (1993). study skills. eugene, oregon: oregon school study council, university of oregon. (eric document reproduction service no. ed 355 616). tümkaya, s. ve bal, l. (2006). çukurova üniversitesi öğrencilerinin ders çalışma alışkanlıklarının bazı değişkenler açısından i̇ncelenmesi, ç.ü. sosyal bilimler enstitüsü dergisi, 15(2), 313-326. türkcan, g., & öcal, g. (2003). verimli ders çalışma teknikleri. çoluk çocuk, 31, 26-27. türkoğlu, a., doğanay, a., ve yıldırım, a. (1996). ders çalışma becerileri, adana: baki kitapevi. uluğ, f. (1981). verimli çalışma alışkanlıkları kazandırma konusunda yapılan rehberliğin okul başarısına etkisi. non published master thesis. ankara üniversitesi. uluğ, f. (2000). okulda başarı: etkili öğrenme ve ders çalışma yöntemleri (7. basım). i̇stanbul: remzi. yalçın, a.(2002). türkçe öğretim yöntemleri, ankara: akçağ. yenilmez, k. ve özbey,n. (2007). i̇lköğretim öğrencilerinin ders çalışma alışkanlıklarının bazı değişkenler açısından i̇ncelenmesi. ai̇bü, eğitim fakültesi dergisi, 7(2),1-16 international electronic journal of elementary education, 2012, vol.4, issue 3, 427-440 440 yeşilyaprak, b. (2000). eğitimde rehberlik hizmetleri. ankara: nobel. yıldırım, a., doğanay, a., & türkoğlu, a. (2000). okulda başarı i̇çin ders çalışma ve öğrenme becerileri. ankara: seçkin. yılmaz, h. (1987). rehberlik ve psikolojik danışma. konya: atlas kitabevi. yüksel, a. (1997). ders çalışma tekniklerinin öğrenci başarısına etkisi. non published master thesis. abant i̇zzet baysal üniversitesi sosyal bilimler enstitüsü, bolu. international electronic journal of elementary education, 2015, 8(2), 317-332. issn:1307-9298 copyright © iejee www.iejee.com mathematics curriculum in ireland: the influence of pisa on the development of project maths liz kirwan university college cork, ireland abstract this article interrogates the extent to which the organization for economic cooperation and development (oecd) through its programme for international student assessment (pisa) influenced the development of project maths, a new second-level mathematics education policy in ireland. it argues that the irish government, in its revision of mathematics education policy, was strongly influenced by pisa and that concern with the country’s ‘average’ placement in the international assessment was instrumental in defining the direction of the revision. it traces the genetic imprint of pisa on the development of curriculum policy, the new mathematics syllabus, its content and assessment. it argues that project maths sets out to follow closely the pisa conceptual framework. however, the analysis finds that project maths is not a mini-pisa but that the programme is comprised of two distinct approaches, on the one hand retaining the abstract, symbolic mathematics of sections of the pre-existing curriculum, while on the other emphasizing a pisa-like approach to pedagogy and to real-life problem solving. keywords: mathematics education, pisa, project maths, policy development, ireland introduction the discourse of the reform of mathematics education takes place within the fields of education, politics and economics. within this discourse educational change is framed as occurring within a rapidly changing economic environment which includes developing knowledge economies, technological advances and shifts in labour patterns. this changing context, it is argued, calls for radical changes in education. mathematics, in particular, is mobilized in the rhetoric as the engine of innovation, the means to developing problem solving skills and independent thinking, the key to the success of knowledge economies. within this discourse it is argued ‘policy can no longer be “thought” or “thought about” within the limits of the nation state and national boundaries’ (ball, 2009, p. 537). although national governments still retain their power to develop their own policies, education  liz kirwan, school of education, university college cork, ireland, phone: +353 86 87 45 864 e-mail: epkirwan@gmx.com http://www.iejee.com/ international electronic journal of elementary education vol.8, issue 2, 317-332, 2015 318 policies in general tend to reflect a distinct set of values ‘whose authority is allocated at the intersection of global, national and local processes’ (rizvi & lingard, 2010, p. 3). one major influence in this ‘internationalisation’ of education policy is the organisation for economic cooperation and development (oecd) (grek, 2009). since the late 1990s the oecd has positioned itself as an agent in national education policies through its development of comparative indicators and its generation of data for steering educational systems (carvalho, 2012). in particular, through its development of the programme for international student assessment (pisa) it has become a technology of government, a tool in the reform of education policy. pisa is a triennial international survey which aims to evaluate education systems worldwide by testing the knowledge and skills of 15 year-old students (oecd, 2015). the programme is formulated to produce policy-orientated and internationally comparable indicators of student achievement (oecd 1999a, 7) together with quantifiable measures of human capital (oecd 1999a, 11). it does this by testing the knowledge and skills of student populations at the end of compulsory schooling. the ‘challenges of today’s knowledge societies’ (oecd, 2002, p. 9) underlie the oecd/pisa problematisation of education systems and in this respect pisa professes to examine ‘the extent to which young people have acquired… knowledge and skills’ (oecd, 1999b, p.11) and their ability to use them to meet real-life challenges. the oecd itself has carefully constructed these ‘knowledge’, ‘skills’ and ‘competencies’ under a programme begun in 1997 – the definition and selection of competencies (deseco) project. deseco forms the theoretical underpinnings of oecd/pisa (deseco project, 2005, p.3; oecd, 2009, p.7). through its assessment of oecd-defined skills and competencies (deseco project, 2005; oecd, 2001), and its prioritisation of oecd-defined ‘literacies’ in mathematics, science and reading (oecd, 1999a), pisa provides a ‘legitimate’ framework within which states can undertake education policy change. it is a powerful policy instrument (grek, 2009; ozga, 2012; rizvi & lingard, 2010) supplying data which contribute to a shift in the definition of knowledge under the influence of the knowledge society construct (carvalho, 2012). it provides support for a move away from traditional school-based learning (cosgrove, shiel, sofroniou, zastrutzki, & shortt, 2004). pisa’s stratified league tables have the effect of reducing ‘educational excellence to test scores’ (clarke, 2012, p. 301). through its production of internationally agreed objectives, norms and standards it exerts a form of ‘soft power’ (bieber & martens, 2011, p. 101). this paper examines the extent of the influence of oecd/pisa on the development of mathematics education policy in ireland. the central questions are: to what extent has recent mathematics education policy at second-level been informed by oecd/pisa and in practical terms, does the new second-level mathematics curriculum, project maths, bear the genetic imprint of oecd/pisa and if so, to what extent? method the research involves an analysis of documents relating to the development of project maths up to its national ‘roll-out’ to all schools in 2010. the mass of empirical data for this analysis will come from official government publications or government-commissioned publications, oecd reports, national and international reports, departmental papers, studies, strategic plans, review documents, mathematics syllabi etc. – that is, the set of documents and acknowledged texts which relate in any way, either partly or in full, to mathematics education policy, and to the reform of mathematics curricula at second-level in ireland. the documents analyzed emanate from a diverse range of sources – local, national and global, the published acknowledged narrative. michel foucault uses the felicitous phrase ‘a history of the present’ (foucault, 1977, p. 31) to describe this kind of undertaking. in his work it is an analysis of how things have come to be as they are. mathematics curriculum in ireland / kirwan 319 the main body constitutes an analysis of the conceptual framework on which project maths is based. the framework for this analysis mirrors that of the oecd and considers the presence or otherwise, in project maths, of the three ‘dimensions’ of pisa which correspond to ‘process skills, knowledge and understanding, and context of application’ (oecd, 2002, p. 12). the concept of mathematics education present in project maths will be compared with pisa’s concept of mathematical literacy. thus the article will consider the extent to which the new programme has been influenced by the philosophy underlying pisa, its approach to mathematical literacy and to its assessment. mathematics education in ireland in ireland the study of mathematics is mandatory at primary school. at secondary school all students study the subject in junior cycle (to approx. age 15), while at senior cycle upwards of 86%1 of students study it and the vast majority of these take mathematics in the leaving certificate examination. mathematics syllabi are developed and implemented centrally under the auspices of the national council for curriculum and assessment (ncca). at second-level the subject is examined nationally at the end of junior cycle and again at the end of senior cycle by means of two state examinations, the junior certificate and the leaving certificate. these are constructed, administered and corrected centrally by the state examinations commission (sec). in 2010 project maths, a radically new second-level programme of mathematics education, was introduced to all second-level schools in ireland2 (full implementation of the project would be completed in 2015). according to official sources the project ‘involves empowering students to develop essential problem-solving skills for higher education and the workplace by engaging teenagers with mathematics set in interesting and real-world contexts’ (ncca, 2015). it replaces previous mathematics programmes3 whose foundations lay in the ‘new maths’ system introduced in the early 1960s (ncca, 2002) where the curriculum involved formal ‘comparatively abstract and symbolic mathematics’ (oldham, 1991, p. 127) and was ‘characterized by emphasis on structure and rigour’ (ncca, 2002, p. 4). over the years mathematics education ‘functioned in an inherently conservative rather than an inherently innovative manner’ (oldham, 2001, p. 275). successive revisions attempted to ‘fix’ syllabus related problems rather than provide an opportunity to review courses thoroughly or to give consideration to an appropriate philosophy for mathematics in the new millennium (shiel, cosgrove, sonfroniou, & kelly, 2001). for example, the junior certificate syllabus revision carried out during the 1990s, which, as we will see, has been the subject of many official pisa-related reviews and criticisms, was the outcome of a process whose limited remit specified that ‘the outcomes of the review would build on current syllabus provision and examination approaches rather than leading to a root and branch change of either’ (ncca, 2002, p. 6). however, the revised syllabus was accompanied by ‘some changes in emphasis: or rather, in certain cases, for some of the intended emphases to be made more explicit and more clearly related to rationale, content, assessment, and … methodology’ (ncca, 2002, p. 17). 1 percentage based on figures for 2014 received from the state examinations commission 2 the pilot phase of project maths was introduced to 24 schools in 2008. 3 the pre-2010 leaving certificate syllabus was introduced in 1992 and the pre-2010 junior certificate syllabus was introduced in 2000 international electronic journal of elementary education vol.8, issue 2, 317-332, 2015 320 when the revision was introduced to schools in 2000 it was accompanied by an inservice programme for teachers which aimed to radically change how mathematics was taught, replacing traditional didactic methods of instruction with ‘active learning methodologies’ (ncca, 2002, p. 25). this move, i contend, was in part a response to a benchmarking report produced by the irish council for science, technology and innovation (icsti) (1999) – initiated on foot of ireland’s decision to position itself as a knowledge economy – which aimed to identify key issues in school science, technology and mathematics (stm) education that required attention. the report found that at post-primary level ‘all countries are moving towards more emphasis on practical, student-centred, activelearning models’ (forfás/icsti, 1999, p. 22) and recommended a similar move in ireland. also at this time oecd/pisa had been instituted and there was an awareness among irish policy agents of the ‘very different philosophy’ of realistic maths education (rme) underpinning pisa which it was expected would ‘highlight weaknesses’ (oldham, 2001, p. 276) in irish mathematics education. thus with the knowledge economy as catalyst, evidence of best practice from the benchmarking report, and oecd/pisa as a policy instrument the process of reform of mathematics education policy was undertaken. an empirical bridge: pisa and the educational research centre the educational research centre (erc), on behalf of the department of education and skills (des), implements pisa in ireland (educational research centre, 2015); it fine-tunes and runs the assessment, grades it, provides commentary on it and is commissioned to prepare the official reports. it is supported in its work by a national advisory committee which includes members of the des, the ncca and ‘subject matter experts’ in mathematics, science and reading. following each pisa assessment the erc publishes a series of reports, which coincide with the publication of the oecd reports, and which provide statistical data and comment on the performance of irish students. the erc, as sub-contractor of the state, is never critical of the pisa process or its underlying assumptions. as mentioned earlier pisa aims to provide empirical evidence which will inform policy decisions and the erc as an agent of the state is employed to provide the ‘empirical bridge’ between the pisa assessment and policy. we can trace the construction of this empirical bridge in relation to mathematics education policy in the pisa-related publications of the erc, especially in the erc’s test curriculum rating and comparison exercises. in the first assessment, pisa 2000, ireland achieved a mean score of 502.9, not significantly different from the oecd country average, on the mathematical literacy scale and it ranked 15th of 27 countries (shiel et al., 2001, p. 44). in its report, the erc attributed the average performance in part to ‘substantial differences between what students at junior cycle are taught and what pisa mathematics assesses’ (shiel et al., 2001, p. 158) and proposed, for further consideration, the question of whether the junior certificate should be assessing mathematical skills that are similar to the pisa mathematics assessment. the very act of pairing the two assessments, which take place at roughly the same stage of schooling (age 15 years), contributed to the construction of a new discourse; thus the average performance in pisa provided evidence that the junior certificate programme was in need of revision. given the generally accepted view that assessment determines pedagogy and, to some extent, curriculum, a convergence of assessment could be expected to produce a convergence of classroom practice and syllabus too (torrance, 2009). in the second iteration, pisa 2003, irish students again achieved a mean score which was not significantly different from the oecd country average and was ranked 20th among 40 oecd and partner countries (cosgrove et al., 2004, p. 48). the lack of improvement was noted and attributed, in part, to the hypothesis that ‘the impact of mathematics curriculum in ireland / kirwan 321 curricular change on students’ achievements, if any, is likely to be slow’ (cosgrove et al., 2004, pp. 164-165) an analysis that implies that the erc and its national advisory committee had expected some convergence between the assessment and the revised junior certificate curriculum of 2000. the erc report included a pisa 2000 testcurriculum rating project which compared the ‘intended’ junior certificate examination syllabus and pisa. it analysed the performance of students in pisa and examined ‘the relative strengths and weaknesses displayed … in terms of what they might reasonably be expected to learn in school’ (cosgrove et al., 2004, p. 40). the report observed that a number of the objectives of the revised syllabus could be compared ‘in a general way’ with the pisa mathematics framework, a comparison which depicts the new syllabus as being ‘somewhat similar to the pisa approach to mathematics’ (cosgrove et al., 2004, p. 163). another move in the pisa direction observed in the report concerns the new teaching methodologies, which it says were an attempt to ‘move away from mechanistic approaches towards teaching for understanding, a change that is consonant with the philosophy underlying pisa mathematics’ (cosgrove et al., 2004, p. 164). it is clear that the erc had expected that the 2000 revision of the junior certificate would bring it somewhat closer to pisa. however, by pisa 2006 there was again no improvement in the score4 and while acknowledging the apparent lack of effect of the revised syllabus on the pisa scores a new aspiration was expressed ‘that new developments, such as project maths, may be diverse enough in content and focus to raise the achievement of high performers in mathematics as well as catering to the needs of students at other performance levels’ (eivers et al., 2008, p. 138). thus project maths became, for the erc and its advisors, the apparent route to future success at pisa. let us now consider why the erc may have put its trust in project maths and whether or not that trust was justified. the framework for this analysis will mirror that used by oecd/pisa in its analysis of mathematical literacy and its assessment. mathematical literacy and project maths in the late 1990s pisa’s expert group for mathematics proposed its definition of mathematical literacy an individual’s capacity to identify and understand the role that mathematics plays in the world, to make well-founded mathematical judgements and to engage in mathematics, in ways that meet the needs of that individual’s current and future life as a constructive, concerned and reflective citizen. (oecd, 1999a, p. 41) this definition focuses on the individual and her involvement with mathematics and how her engagement with mathematics is perceived to permeate all aspects of her life. certain aspects of the definition have specific meaning in the pisa context. the term ‘literacy’, for example, indicates ‘the ability to put mathematical knowledge and skill to functional use rather than just to master it within a school curriculum ….. the ability to pose and solve mathematical problems in a variety of contexts’ (oecd, 2002, p. 82). according to de lange, (chair of pisa’s expert group for mathematics which developed the definition of mathematical literacy), the definition addresses the goal of preparing students for society and for future schooling and work and is not concerned with showing students ‘the beauty of the discipline’ (de lange, 2006, p. 21). in this respect project maths adopts the pisa model of mathematical literacy in all but name, aiming to introduce to the school curriculum mathematical knowledge and skill in a more applied format and 4 ireland’s mean score was 501.5, which is not significantly different from the oecd country average. ireland ranked 16th highest among oecd countries, and 22nd among 57 participating countries. (eivers, shiel, & cunninghan, 2008, p. 60) international electronic journal of elementary education vol.8, issue 2, 317-332, 2015 322 abandoning the philosophy which considered mathematics as ‘the queen and the servant of the sciences’ (department of education, 1992; des, 2000). project maths is designed to ensure that current and future students of mathematics at post-primary level will have the opportunity to engage in their studies in a manner which will enhance their understanding of the subject, provide contexts and applications of mathematics that are meaningful and relevant, and enable them to develop problem-solving skills and strategies that will serve them not only in their future study of mathematics but also in their daily lives. (ncca, 2008, p. 5) the move replaces the formal, comparatively abstract and symbolic mathematics with a more practical approach to the subject. in the new programme mathematics is seen as the key to opportunity. no longer simply the language of science, mathematics contributes in direct and fundamental ways to business, finance, health and defence. for students it opens doors to careers….. for nations it provides knowledge to compete in a technological community. participating fully in the world of the future involves tapping into the power of mathematics. (des/ncca, 2010, p. 6) project maths, therefore, emphasises the utility of mathematics, the role it is seen to play in ‘the development of the knowledge society and the culture of enterprise and innovation associated with it’ (des/ncca, 2010, p. 6). de lange identifies pisa’s emphasis on the element of ‘functionality’ and, in discussing countries which take the outcomes of pisa seriously, he claims that ‘they embrace the idea that the output of an educational process should include a certain amount of ‘functionality’, but it is up to the countries to decide how important this aspect is’ (de lange, 2006 n.p.). in project maths, it is argued, that much of what students learn in the new programme will be about putting mathematical knowledge and skill to functional use rather than the application of procedures in ‘a purely mathematical and abstract context’ (cosgrove, oldham, & close, 2005, p. 210). at all levels it is envisaged that mathematics will be taught ‘in contexts that allow learners to see connections within mathematics, between mathematics and other subjects, and between mathematics and its applications to real life’ (des/ncca, 2010, p. 6) and thus mathematics teaching and learning is being directed away from the traditional method with its ‘formal and abstraction-focused approach’ (conway & sloane, 2005, p. 23) towards the more instrumental approach to the subject. while not slavishly following pisa it is a very similar approach. three dimensions of mathematical literacy in order to transform the definition of mathematical literacy into an assessment, the pisa mathematics expert group identified three broad dimensions: processes, content and context (oecd, 2002, p. 82) and outlined the criteria for assessment in each dimension. in doing this it clarified its assessment priorities and provided a framework for policy development. tracing the genetic imprint of pisa will therefore involve an interrogation of the project maths syllabus and assessment material and an analysis of the existence or otherwise of these ‘dimensions’ in the data. process processes or process skills which priorities ‘students’ abilities to analyze, reason and communicate ideas effectively by posing, formulating and solving mathematical problems’ (oecd, 2002, p. 13) are given precedence in mathematical literacy. the skills involved include thinking, argumentation, modelling, problem posing and problem solving, representation, symbolic, technical, communication, and skills in using mathematical tools and aids. pisa does not assess these process skills individually but organizes them into three classes, each of which defines the type of thinking skill required: (i) reproduction, definitions and computations; (ii) connections and integration of problem solving; and (iii) mathematics curriculum in ireland / kirwan 323 mathematisation, mathematical thinking, generalization and insight (oecd, 2002, p. 83). these classes or ‘competency clusters’ are called reproduction, connections and reflection (oecd, 2002, p. 82) respectively. the question to address here is to what extent these competencies are present in project maths syllabi and state examinations and whether or not they appeared in the the pre-2010 syllabi and associated state examinations? 1. process: competency clusters in syllabus learning outcomes. project maths aims to teach mathematics ‘in contexts that allow learners to see connections within mathematics, between mathematics and other subjects, and between mathematics and its applications to real life’ (des/ncca, 2012b, p. 6). the new syllabi, in line with oecd recommendations, are formatted in terms of ‘learning outcomes’ (oecd, 1999b, p. 3) and replace the traditional input/content structure of previous syllabi. each strand of study is delineated by a brief description of the topic to be studied together with details of what students should be able to do. an analysis of learning outcomes in syllabi issued in 2012 (for examination at leaving certificate in 2014 and at junior certificate in 2015) provides some insights into the influence of the pisa conceptual framework on the intended curriculum. the 2012 syllabi were chosen for the analysis as this is the first set of project maths syllabi to include all five strands of study. in the context of pisa, learning outcomes which reflect the characteristics of the reproduction competency cluster (reproduction, computations and definitions), deal with ‘knowledge of facts, representing, recognising equivalents, recalling mathematical objects and properties, performing routine procedures, applying standard algorithms and developing technical skills’ (oecd, 2002, p. 84). these competencies appear in junior and leaving certificate syllabi learning outcomes in the form of requirements to be able to: recall axioms; engage with the concept of a function, domain, co-domain and range; solve first degree equations in one or two variables; use and apply the rules for indices; construct a variety of geometric shapes; recognise that probability is a measure on a scale of 0–1 of how likely an event is to occur; use a calculator to calculate standard deviation; graph functions; perform constructions; perform arithmetic operations; factorise expressions; use trigonometry to calculate the area of a triangle; illustrate complex numbers on an argand diagram; etc. (des/ncca, 2012a, 2012b). the second competency cluster connections (brings together mathematical ideas and procedures to solve problems in familiar contexts), expects students to make ‘connections between the different strands and domains in mathematics, and integrate information in order to solve simple problems…..the problems are often placed within a context, and engage students in mathematical decision making’ (oecd, 2002, p. 84). project maths learning outcomes in this category include: solving problems involving conditional probability in a systematic way; applying knowledge and skills to solve problems in familiar contexts; exploring patterns; analyzing and transferring information into a mathematical form; solving problems involving right-angled triangles, trigonometric ratios, finding profit or loss, income tax and net pay, surface areas, curved surface areas, volume, etc. (des/ncca, 2012a, 2012b). however, the syllabus is structured according to a series of five individual strands (we will discuss this in more detail later) and consequently the making of connections between and across strands and different domains of mathematics is not prescribed by the learning outcomes. nevertheless, elsewhere the syllabus states that ‘where appropriate, connections should be made within and across strands and with other areas of learning’ (des/ncca, 2012a, p. 10). the third competency, reflection (involves mathematical thinking, generalization and insight, and requires students to engage in analysis, to identify the mathematical elements in a situation, and to pose their own problems), asks students ‘to mathematise situations…to recognize and extract the mathematics…use mathematics to solve the international electronic journal of elementary education vol.8, issue 2, 317-332, 2015 324 problem, analyze, interpret, develop…models and strategies,…’ (oecd, 2002, p. 84). again this cluster occurs in the project maths syllabus learning outcomes, at both junior and senior cycles, in particular under the heading of ‘synthesis and problem-solving skills’ which is to be found in each strand of study. within this section students are expected to learn to: explore patterns and formulate conjectures; explain findings; justify conclusions; communicate mathematics verbally and in written form; apply their knowledge and skills to solve problems in familiar and unfamiliar contexts; analyze information presented verbally and translate it into mathematical form; devise, select and use appropriate mathematical models, formulae or techniques to process information and to draw relevant conclusions (des/ncca, 2012a, 2012b). it is clear, therefore, that the pisa framework of competency clusters is present in the learning outcomes assigned to project maths and thus it can be considered part of the ‘intended’ curriculum. this adherence to the pisa competency clusters in learning outcomes translates into a two-part approach to assessment which involves a combination of old and new. 2. process: competency clusters and assessment. an analysis of junior certificate and leaving certificate project maths examination papers reveals a complex, almost schizoid, situation. in the examinations some syllabus items are examined within the context of real-life problems, while the examination of other items relies on the traditional abstract/symbolic question-type. this extraordinary situation, is i contend, linked to the inclusion of pisa and non-pisa items in the syllabi. almost all pisa items are present in project maths and several curricular content areas of the pre-2010 syllabi (not assessed by pisa) have also been retained in the programme: equations, functions, sets, both formal and co-ordinate geometry, trigonometry and property of number (merriman, shiel, cosgrove, & perkins, 2014, p. 75). an inspection of post 2014 examination papers reveals that questions relating to the non-pisa items tend to retain the abstract nature of pre2010 examinations while the assessment of pisa-type items reflects a more pisa-like approach to assessment. in the junior certificate higher level examination papers of 2015, for example, the non-pisa items account for approximately 45% of the recommended time allotment and are examined in a series of non-contextualized questions. table 1. results of an analysis of question types from the 2015 leaving certificate and junior certificate examinations examination examination level abstract/symbolic % ‘real-life’ problems % junior certificate higher 45 55 ordinary 48 52 foundation 45 55 leaving certificate higher 48 52 ordinary 39 61 foundation 3 97 table 1 presents the results of an analysis of question types from the 2015 leaving certificate and junior certificate examinations. the figures (table 1) clearly indicate the divide between the traditional abstract/symbolic approach to examination questions and the real-life problem-solving mathematics curriculum in ireland / kirwan 325 approach. with the exception of leaving certificate foundation level, which is intended for students who may have a limited acquaintance with abstract mathematics (des/ncca, 2012b), there is a near 50/50 division between the examination of mathematics in an abstract form and real-life mathematics. it is difficult to imagine how the pedagogy of active methodologies introduced with project maths will transfer to the classroom within the context of this near 50/50 division. as mentioned earlier assessment tends to determine what is taught. in the early 2000s such was the interest in pisa mathematics that a pisa/junior certificate assessment mapping exercise was carried out by two irish mathematics education scholars (close & oldham, 2005). the findings of this research identified mismatches between skills examined by the junior certificate examination and those of the pisa assessment. the research arose out of the mathematical performance in pisa 2003 of irish 15-year-olds as this had occasioned discussion about the contrast between the type of questions in pisa and those of the junior certificate examination. the authors – close, drawing on his experiences as a member of the mathematics expert group for pisa and oldham, drawing on her experience as former education officer with the ncca and a member of the pisa mathematics forum (close & oldham, 2005, p. 186) took questions from the junior certificate paper of 2003 and mapped these onto the pisa mathematics framework. the results were compared and contrasted. considering that the junior certificate examination had its foundations in new maths and the underlying philosophy of pisa is rme, it is not surprising that the analysis identified many ‘mismatches’ between the two assessments. in particular the junior certificate examinations contained no items on reflection, very few on connections and over 90% on reproduction (close & oldham, 2005, p. 196). thus, according to this analysis, the process of mathematicisation, mathematical thinking, generalization and insight involved in the reflection competency cluster was absent from the examination and the making of connections between the different strands and domains in mathematics and the integration of information while solving simple problems as required in the connections cluster was poorly represented. an analysis of the 2015 junior certificate examination papers reveals a pisa-like trend in many questions, although the traditional comparatively abstract and symbolic nature of previous examination questions is still in evidence. as mentioned above, the approach to the assessment of pisa and non-pisa items differs and consequently the skills examined vary according to the type of question presented. questions relating to non-pisa items focus largely on performing calculations, solving equations and solving routine problems and belong, for the most part, to the reproduction cluster. the real-life problems require students to solve problems using familiar procedures in contexts, to recognise and extract the mathematics in problem situations, to reflect on and analyse situations etc. and in this way the connections and reflection clusters are assessed. all three competency clusters are well represented in the assessment. in relation to the pre-2010 leaving certificate examinations, an analysis of the 1999 higher-level and ordinary-level papers, for example, reveals the absence of the reflection competency and very few items on connections, while the majority of items fall into the reproduction category. at the higher level, apart from probability, questions were presented in an abstract format, while at ordinary level, some real-life problems were posed in relation to money, probability, statistics and linear programming, otherwise the examination dealt with abstract, symbolic mathematics. this situation was repeated with minor variations in all papers until the introduction of project maths. in an effort to aid the implementation of the intended curriculum, in advance of the first project maths leaving certificate examination for the pilot schools, the sec issued a set of leaving certificate sample papers for phase1 of the project together with a set of proposed international electronic journal of elementary education vol.8, issue 2, 317-332, 2015 326 solutions. it subsequently published a report on this trial assessment. the process was intended to measure the effectiveness of the draft sample papers and the marking schemes but the outcomes of the trialing were also to ‘assist with the implementation of the syllabus itself…[and]…to clarify the objectives and learning outcomes of the syllabus for teachers, candidates and other interested parties’ (sec, 2010, p. 8). according to the sec it is the combination of examination paper and marking scheme ‘that determines what competencies are measured and in what proportion’ (sec, 2010, p. 105). thus the examination determines what is taught. this is somewhat ironic considering that teaching to the test was a frequent complaint regarding earlier syllabi (ncca, 2005). on reading the sec report (sec, 2010), the language and underlying skills philosophy of pisa is evident. each suggested solution is preceded by a statement of the main assessment objective(s) whether it is to ‘demonstrate understanding of concepts and connections’ or to ‘execute routine procedures in a mathematical context’, or to ‘apply understanding within a mathematical context’ or to ‘demonstrate understanding of concepts, connections, conditions and implications’, the objective is clearly defined. in the trial examination papers the reproduction cluster presents itself through assessment objectives such as: execute routine procedures in a mathematical context; demonstrate knowledge of notation and terminology; apply routine procedures; demonstrate use of geometrical instruments; demonstrate knowledge of terminology and facts (sec, 2010). assessment of the connections cluster is to be found in questions that require the student to: apply routine procedures (in non-mathematical context); in a nonmathematical context, apply routine procedures (viz. execute trigonometric calculations); in a non-mathematical context, apply routine procedures, interpreting the solutions in the original context; (sec, 2010). and the reflection cluster, which was absent from the previous syllabi, is very much in evidence in questions which require students to: in a mathematical context, apply understanding of concepts and connections, including relevant conditions, implications; demonstrate understanding of concepts, connections, conditions and implications; in a non-mathematical context, apply understanding of concepts and connections, interpreting solutions, conditions and implications in the original context (viz. ‘mathematise’ the presented problem and show how to solve it) (sec, 2010). it seems likely that these competency clusters were deliberately designed to mirror the pisa. however by 2015 the divide between the traditional and the new had crystallised. when all five strands of the project were examined 56% of the leaving certificate higher and ordinary level papers were focused on real-life problems and the remaining 44% of the items examined were situated within the traditional abstract question format. all competency clusters reproduction, connections and reflection were duly represented. at both leaving certificate and junior certificate levels pisa-related material was examined in a pisa-friendly style while material that does not appear on the pisa assessment was examined by traditional abstract/symbolic questions. far from being a radical shift, the examination looks like a compromise. content it is in the area of syllabus content that project maths distinguishes itself from pisa. as we have seen project maths has been influenced by the concept of mathematical literacy and it has incorporated the pisa defined competency clusters in its learning outcomes and assessment objectives. under the influence of pisa the intended curriculum has given priority to interpreting and solving mathematics problems embedded in realistic contexts and this in turn has affected the teaching and learning process where teaching via mathematics curriculum in ireland / kirwan 327 problems is advocated (project maths website, 2011). but the two programmes diverge in their approach to content. the project maths syllabus classifies the mathematics curriculum in five separate content strands. the 2008 pilot phase and the 2010 national roll out of the project involved a phased introduction of these syllabus strands at both junior cycle and senior cycle and it is reasonable to conclude that the decision to opt for strands was at least partly administrative allowing for the introduction of relatively discrete areas of study over time. the five strands are: statistics and probability; geometry and trigonometry; number; algebra; functions (ncca, 2008, p. 2). the practice of classifying mathematics curriculum in content strands is rejected by pisa on the grounds that it compartmentalizes mathematics and does not allow for the complexity of problems, nor does it reflect the complex patterns of the world around us (oecd, 2002, p. 84). de lang argues that it encourages students to view mathematics as a ‘collection of fragmented pieces of factual knowledge’ (de lange, 2006, p. 7). the mathematical content of pisa itself is organized around a phenomenological approach, ‘describing content in relation to a phenomenon and the kinds of problems for which it was created’ (oecd, 2002, p. 84). the phenomena are referred to as ‘overarching concepts’ and they were selected in order to ‘encompass sufficient variety and depth to reveal the essentials of mathematics and… at the same time represent or include the conventional mathematical curricular strands’. the pisa ‘overarching concepts’ are: change and relationships, space and shape, quantity and uncertainty (oecd, 2002, p. 84). the project maths adoption of the content strand structure at second level was intended to provide a link between the primary and secondary school curriculum (ncca, 2008) along the ‘pathways’ which the different topics of mathematics follow as the learner progresses from primary to secondary school (des/ncca, 2011, p. 8). this approach reinforces the compartmentalizing of the subject in spite of assurances by the project maths development team that ‘the various strands of mathematics are inter-related, and therefore can be taught in a more integrated manner’ (ncca, 2008, p. 2). however, the ideal of an integrated approach to the subject succumbs to practicality and the authors of the trial report attempted to justify the proposed sequential introduction of the project maths strands (sec, 2010) being presented as ‘stand-alone component[s] of the course’ (grannell, barry, cronin, holland, & hurley, 2011, p. 5). the decision to introduce strands in a ‘stand-alone’ manner was influenced by factors such as ‘the need to balance an increase in syllabus elements in one strand with a decrease elsewhere, and confining the impact of change at the examinations to one paper in any given year’ (ncca, 2008, p. 2). the effect of the mode of introduction and of the subsequent examination limited the extent of inter-relation in the assessment and hence in the classroom. in spite of the difference in approach to content only a small number of pisa items are not covered by the project maths curriculum, these include 2-d and 3-d rotations of objects and it is argued that project maths students might not be familiar with relating information on a table to information on a map or chart (merriman et al., 2014). as mentioned above, both junior and senior cycle syllabi retain a number of non-pisa items: equations, functions, sets, both formal and co-ordinate geometry, trigonometry and property of numbers (merriman et al., 2014, p. 75). pre-2010 junior cycle syllabi did not include the study of probability, at that time this content was reserved for senior cycle. new syllabi, at all levels, emphasize probability and a des statement relating to project maths stated that ‘the changes other than in probability and statistics are methodological rather than content based’ (des, 2010) although subsequent criticism of project maths would disagree with this statement (grannell et al., 2011; stack & other maths professionals, 2012). international electronic journal of elementary education vol.8, issue 2, 317-332, 2015 328 context the third dimension of pisa mathematical literacy deals with the context within which mathematics is applied. pisa tasks are designed to include a variety of situations such as ‘personal life, work and leisure, the local community, and society’ (oecd, 2002, p. 82). project maths, on the other hand, seeks to teach mathematics in contexts that will allow learners to see connections between mathematics and its applications to real life. in the project maths programme these context areas are reflected in lesson plans, sample questions and in state examination questions. many problems are set within ‘realistic’ situations which involve personal taxes, mobile phone choice, swimming pools, water towers, golf courses, oil spills, password combinations, student heights, health etc. it is difficult to assess how authentic the chosen contexts can be however. while discussing the (un)realistic nature of a particular question from the 2010 leaving certificate examination paper, for example, the university college cork (ucc) interim report on project maths remarked that ‘“[r]eal-world” problems should be formulated so that they are not silly’ (grannell et al., 2011, p. 29). many of the so-called ‘real-world’ problems are outside of the experiences of students and thus the interpretation of questions can cause greater problems than the mathematics required for the solution. questions relating to ‘negative equity’ or to the ‘tolerance of batteries’ (leaving certificate 2012), for example, may be just as confusing for some students as a 2012 leaving certificate question based on a ‘robotic arm’ which was considered biased towards those who study physics or applied maths and which became the subject of a parliamentary question in autumn 2012 (o'connor, 2012). another aspect of ‘real-world’ problems is their literacy demands and in this respect, it is argued that, the ‘presentation of information in the project maths syllabus is more like pisa than in the pre-2010 syllabus’ (merriman et al., 2014, p. 75). both pisa and project maths are criticised for the high level of literacy required by students to successfully engage with ‘real-world’ problems (eivers, 2010; merriman et al., 2014). eivers argues that setting problems in context hampers the validity of the assessment and that for many students their ‘reading skills have an unnecessarily large effect on how well (or poorly) they perform on … mathematics assessments’ (eivers, 2010, p. 100). the problem for project maths, is highlighted in the case of students with reading difficulties or possibly those for whom english or irish are additional languages. in this context in 2012 the national secretary of the irish mathematics teachers association (imta) was critical of the level of difficulty of the language used in a number of leaving certificate questions (o'sullivan, 2012). all of this is despite the fundamental category error of assigning a term like ‘realistic’ to an entirely artificial construct which is only found in the ‘real’ world in its singular form – namely, the paper and pencil, timed examination in an invigilated examination hall. in literature this kind of artificial construct of ‘realism’ is generally known as ‘the realistic convention’, and in terms of these word-based problems, the term is apt. conclusion this article has argued that the irish government in its revision of mathematics education policy was strongly influenced by the conceptual framework of pisa and that concern with the countries ‘average’ placement in the international assessment was instrumental in defining the direction of the revision. it has traced the influence of the pisa concept of mathematical literacy and its instrumental view of mathematics on the project maths rationale. it has shown that the aim of project maths to teach mathematics in contexts that allow students to see connections within mathematics, between mathematics and other subjects, and between mathematics and its application to real life facilitated the mathematics curriculum in ireland / kirwan 329 development of the curriculum within the conceptual framework of pisa. it has traced the inclusion of the pisa competency clusters in both the syllabus learning outcomes and in assessment. the emphasis on problem-solving within real-life contexts is a common feature of both pisa and project maths. thus it is argued that project maths does bear the genetic imprint pisa. however project maths is not a mini-pisa and the two programmes diverge in their approach to content and in their commitment to formal, abstract mathematics. in particular project maths retains the strand/topic structure in the area of content while the assessment of project maths reveals the existence of a curriculum focused in part on a traditional approach to abstract, symbolic mathematics, while at the same time prioritising the interpretation and solving of mathematics problems embedded in realistic contexts. in terms of pedagogy, teaching via problems is advocated by the des despite the fact that almost half of the time/marks allocation on the assessment is for traditional abstract mathematics which teachers have long taught by didactic methods rather than active ones. it is beyond the scope of this paper to speculate on how this will be reflected at classroom level or in future pisa assessments, but suffice it to say that in terms of mathematics education philosophy the compromise is problematical. project maths, according to the minister for education and science, ‘will promote greater maths literacy across the school population, bringing changed emphasis in the maths learned and, in particular, a strong focus on context, application and problem-solving in a general move towards a strengthened emphasis on real-life maths education’ (project maths, 2010). it is notable that the minister here talks about ‘a strengthened emphasis on’ rather than a switch to ‘real-life mathematics’. it could well be argued that this ‘strengthened emphasis’, while not in evidence in curriculum and intended pedagogy is very evident in the assessment. the extent to which the assessment will ultimately govern the pedagogy will be interesting. it remains to be seen whether this experiment in matching a national curriculum and pedagogy to an international assessment will have the desired outcomes of improving the country’s position on the pisa league tables and producing suitably trained human capital which will satisfy the perceived needs of the knowledge economy. however the intensity with which the change was prosecuted by the department of education and skills, and the clarity of its vision with regard to pisa, suggests that government believes there is a great deal at stake. liz kirwan is a researcher affiliated to the school of education, university college cork. her phd thesis presented a genealogy of mathematics education policy in ireland at the beginning of the 21st century. her research interests include mathematics education policy development, the position of mathematics education in the global education policy field, gender in mathematics education, governmentality. international electronic journal of elementary education vol.8, issue 2, 317-332, 2015 330 references ball, s. j. (2009). the governance turn! journal of education policy, 24(5), 537-538. bieber, t., & martens, k. (2011). the oecd pisa study as a soft power in education? lessons from switzerland and the us. european journal of education, 46(1), 101-116. carvalho, l. m. (2012). the fabrications and travels of a knowledge-policy instrument. european educational research journal, 11(2), 172-188. clarke, m. (2012). the (absent) politics of neo-liberal education policy. critical studies in education, 53(3), 297-310. close, s., & oldham, e. (2005). junior cycle mathematics examinations and the pisa mathematics framework. in sean close, therese dooley, & dolores corcoran (eds.), proceedings of first national conference on research on mathematics education (mei 1) mathematics education in ireland: a research perspective (pp. 185-204). dublin: st. patrick’s college of education. conway, p., & sloane, f. (2005). international trends in post-primary mathematics education: perspectives on learning, teaching and assessment. dublin: ncca. cosgrove, j., oldham, e., & close, s. (2005). assessment of mathematics in pisa 2003: achievements of irish 15-year-olds in an international context. paper presented at the proceedings of the first national conference of research in mathematics education, dublin: st. patrick’s college. cosgrove, j., shiel, g., sofroniou, n., zastrutzki, s., & shortt, f. (2004). education for life: the achievements of 15-year-olds in ireland in the second cycle of pisa. retrieved from: http://www.erc.ie/documents/p03education_for_life_main_report.pdf de lange, j. (2006). mathematical literacy for living from oecd-pisa perspective. tsukuba journal of educational study in mathematics,, 25, 13-35; http://www.human.tsukuba.ac.jp/~mathedu/journal/vol25/lange.pdf, accessed 22-12-2011. department of education. (1992). the leaving certificate mathematics syllabus, (higher and ordinary level). dublin: the stationary office. des. (2000). junior certificate mathematics syllabus (higher, ordinary and foundation level). dublin: the stationery office. des (2010). [des reply to imta meeting re: imta policy on project maths]. des/ncca. (2010). leaving certificate mathematics syllabus foundation, ordinary and higher level for examination in 2012 only. retrieved from: http://www.ncca.ie/en/curriculum_and_assessment/postprimary_education/project_maths/syllabuses_and_assessment/lc_maths_for_examination_in_2 012.pdf des/ncca. (2011a). junior certificate mathematics syllabus: foundation, ordinary and higher level: for examination in 2014 only. retrieved from: http://www.ncca.ie/en/curriculum_and_assessment/postprimary_education/project_maths/syllabuses_and_assessment/jc_maths_for_examination_in_2 014.pdf des/ncca. (2011b). leaving certificate mathematics syllabus foundation, ordinary and higher level for examination in 2013 only. retrieved from: http://www.ncca.ie/en/curriculum_and_assessment/postprimary_education/project_maths/syllabuses_and_assessment/leaving_cert_maths_syllabus_f or_examination_in_2013.pdf des/ncca. (2012a). junior certificate mathematics syllabus: foundation, ordinary and higher level: for examination in 2015 only. retrieved from: http://www.ncca.ie/en/curriculum_and_assessment/postprimary_education/project_maths/syllabuses_and_assessment/jc_maths_for_examination_in_2 015.pdf mathematics curriculum in ireland / kirwan 331 des/ncca. (2012b). leaving certificate mathematics syllabus foundation, ordinary and higher level for examination in 2014 only. retrieved from: http://www.ncca.ie/en/curriculum_and_assessment/postprimary_education/project_maths/syllabuses_and_assessment/lc_maths_for_examination_in_2 014.pdf educational research centre. (2015). educational research centre, pisa. retrieved 12 may 2015, 2015, from http://www.erc.ie/?p=7 eivers, e. (2010). pisa: issues in implementation and interpretation. the irish journal of education / iris eireannach an oideachais, 38, 94-118. eivers, e., shiel, g., & cunninghan, r. (2008). ready for tomorrow's world? the competencies of ireland’s 15-year-olds in pisa 2006 main report. retrieved from: http://www.erc.ie/documents/p06ready_for_tomorrows_world_main_report.pdf forfás/icsti. (1999). benchmarking school science, technology and mathematics education in ireland against international good practice. retrieved from: http://www.sciencecouncil.ie/media/icsti990901a_benchmarking_school_education.pdf foucault, m. (1977). discipline and punish: the birth of the prison (a. sheridan, trans. first ed.). london: penguin. grannell, j. j., barry, p. d., cronin, m., holland, f., & hurley, d. (2011). interim report on project maths. retrieved from: http://www.ucc.ie/en/euclid/projectmaths/projectmathsinterimreport_nov2011.pdf grek, s. (2009). governing by numbers: the pisa 'effect' in europe. journal of education policy, 24(1), 23-37. icsti. (1999). benchmarking school science, technology and mathematics education in ireland against international good practice. retrieved from: http://www.sciencecouncil.ie/media/icsti990901a_benchmarking_school_education.pdf merriman, b., shiel, g., cosgrove, j., & perkins, r. (2014). project maths and pisa 2012. dublin: erc. ncca. (2002). mathematics, junior certificate, guidelines for teachers. dublin: the stationery office. ncca. (2008). developing post-primary mathematics education; project maths-an overview. retrieved from: http://www.ncca.ie/uploadedfiles/publications/vacancies/project%20maths%20overview.pdf ncca. (2015). project maths. retrieved 16 may 2015, from http://www.ncca.ie/en/curriculum_and_assessment/post-primary_education/project_maths/ o'connor, m., mitchell,. (2012). school curriculum – question to minister ruairí quinn. dáil debates, 772(2), 18. retrieved from o'sullivan, b. (2012). project maths doesn't quite add up. irish examiner online. retrieved from: http://www.irishexaminer.com/analysis/project-maths-doesnt-quite-add-up-197281.html oecd. (1999a). measuring student knowledge and skills: a new framework for assessment. retrieved from: http://www.oecd.org/dataoecd/45/32/33693997.pdf oecd. (1999b). measuring student knowledge and skills: the pisa 2000 assessment of reading, mathematical and scientific literacy. paris: oecd. oecd. (2002). programme for international student assessment sample tasks from the pisa 2000: assessment of reading, mathematical and scientific literacy. retrieved from: http://www.oecd.org/dataoecd/44/62/33692744.pdf oecd. (2015). programme for international assessment (pisa). retrieved 17 may 2015, from http://www.oecd.org/pisa/aboutpisa/ international electronic journal of elementary education vol.8, issue 2, 317-332, 2015 332 oldham, e. (1991). second level mathematics curricula: the republic of ireland in international perspective. irish educational studies, 10(1), 122-138. oldham, e. (1992). junior cycle mathematics curricula in the republic of ireland 1960–1990 : genesis, exodus and numbers. irish educational studies, 11(1), 134-150. oldham, e. (2001). the culture of mathematics education in the republic of ireland: keeping the faith? irish educational studies, 20(1), 266-277. oldham, e. (2002). the performance of irish students in mathematical literacy in the programme for international student assessment (pisa). the irish journal of education / iris eireannach an oideachais, 33, 31-52. ozga, j. (2012). assessing pisa. european educational research journal, 11(2), 166-171. project maths. (2010). retrieved 18 november 2010, from http://www.projectmaths.ie/october2010/ project maths website. (2011). the purpose of this document. project maths website. retrieved from: ref: http://www.projectmaths.ie/problem-solvingquestions/context_based_problems_sr1_final_sk.pdf?utm_source=project+maths+mailing+list &utm_campaign=084bf88b43-project_maths_news_update20_11_2011&utm_medium=email rizvi, f., & lingard, b. (2010). globalizing education policy. london and new york: routledge. sec. (2010). report on the trialling of leaving certificate sample papers for phase 1 of project maths in the twenty-four initial schools. 111. retrieved from: http://www.examinations.ie/schools/report_on_trial_final.pdf shiel, g., cosgrove, j., sonfroniou, n., & kelly, a. (2001). ready for life? the literacy achievements of irish 15-year olds with comparative international data. retrieved from: http://www.erc.ie/documents/p00ready_for_life_main_report.pdf stack, c., & other maths professionals. (2012). major flaws in project maths. website. retrieved from: http://www.projectmaths.com/index.php/news-comments/ torrance, h. (2009). using assessment in education reform. in h. daniels, h. lauder, & j. porter (eds.), knowledge, values and educational policy, a critical perspective. london, new york: routledge. international electronic journal of elementary education, 2014, 7(1), 13-26. issn:1307-9298 copyright © iejee www.iejee.com how reading volume affects both reading fluency and reading achievement richard l. allington  university of tennessee, knoxville, tn, usa received: 13 october 2014 / revised: 25 october 2014 / accepted: 31 october 2014 abstract long overlooked, reading volume is actually central to the development of reading proficiencies, especially in the development of fluent reading proficiency. generally no one in schools monitors the actual volume of reading that children engage in. we know that the commonly used commercial core reading programs provide only material that requires about 15 minutes of reading activity daily. the remaining 75 minute of reading lessons is filled with many other activities such as completing workbook pages or responding to low-level literal questions about what has been read. studies designed to enhance the volume of reading that children do during their reading lessons demonstrate one way to enhance reading development. repeated readings have been widely used in fostering reading fluency but wide reading options seem to work faster and more broadly in developing reading proficiencies, including oral reading fluency. keywords: volume, fluency, voluntary reading, comprehension, accuracy. introduction fourth-grader abdul is a good reader. few teachers would then be surprised to learn that abdul also reads voluntarily, hooked currently on the diary of a wimpy kid books. in many respects, abdul is a good reader because he reads extensively voluntarily (cipielewski & stanovich, 1992). few teachers would be surprised to learn that abdul is also a fluent oral reader, reading with both accuracy and expression. at the same time, too few teachers realize that it is at least as much the case that his extensive voluntary reading produced his high levels of reading accuracy as well as his ability to read aloud accurately and with expression. abdul, like many effective young readers has never participated in a single lesson designed to foster his fluent reading. he has never engaged in any repeated readings activities. abdul just reads. a lot. and voluntarily. abdul’s development as a reader represents the path followed by many proficient readers, especially students who completed first-grade prior to 2001. that is, before reading fluency was named one of the five scientifically-based pillars of reading development by the national reading panel (2001).  richard l. allington, a209 bec, university of tennessee, knoxville tn 37998, 5184211775, rallingt@utk.edu. international electronic journal of elementary education vol.7, issue 1, 13-26,2014 14 in this article i hope to provide a brief history of reading fluency in american education and then share what we know about the relationship between fluency and reading proficiency broadly considered and reading volume. in truth, this chapter is more about the potentially powerful, but typically overlooked, role of reading volume. the evidence we have is consistent and clear: children who elect to read voluntarily develop all sorts of reading proficiencies, not just the ability to read fluently (mol & bus, 2011). in this chapter, however, i will largely ignore the other proficiencies fostered through extensive voluntary engagement in reading activity and focus on volume of reading and its role in the development of fluent readers. i conclude with strategies for enhancing voluntary reading among elementary school students. the research on the relationship between reading volume and reading fluency. while classroom teachers have paid attention to reading fluency for a long time, researchers largely ignored the development of reading fluency until about 40 years ago when dahl and samuels (1977) published a paper contrasting drill on word recognition in isolation with repeated reading of passages to attain a standard reading rate (100 words per minute). they reported that the repeated reading intervention developed struggling readers’ reading fluency, accuracy, and comprehension far better than the training to rapidly and accurately read words in isolation. shortly thereafter, samuels (1979) published a paper in reading teacher on the repeated reading method. samuels seemed prompted to explore reading fluency primarily as a result of his earlier co-authored paper (laberge & samuels, 1974) that set forth automaticity theory as an explanation of early reading development. basically, this theory argued that automaticity involved developing lower level processes (as in word recognition) to free up attentional space for higher-level processes (comprehension). as sometimes happen in experiments, the dahl and samuels (1977) experiment surprisingly demonstrated that repeated reading worked better than isolated training of word recognition in isolation. their findings have been replicated by other researchers over the years (homan, klesius & hite, 1993; morgan, siderisis & hua, 2012; vadasy, sanders & peyton, 2005). in other words, what has now been repeatedly demonstrated is that working to foster automatic word identification through lessons that feature primarily word level work is simply less effective at developing reading fluency than lessons that engage readers in repeated reading activities. kuhn and stahl (2003) reviewed over 100 research studies on repeated readings but noted that the studies were a mixture of models including many studies with no true control group and most did not compare repeated readings with an alternative intervention. however, in the two studies where a repeated readings model was compared to a control group where students read independently for comparable amounts of time they found no difference in fluency outcomes. overall, they concluded that the repeated reading model improves both fluency and reading achievement. based on the two studies noted above, they also suggested that it may be the increase in the volume of reading that students do when engaged in repeated reading activities that underlies the success observed with the use of repeated readings in developing fluent reading performances. the same year that kuhn and stahl published their review, therrien (2003) provided a meta-analysis of repeated readings studies published since 1979 and found repeated readings to be an effective intervention for improving the reading fluency of both general and special education students. this meta-analysis also indicated that repeated reading with an adult present proved to be more effective than repeated reading interventions where students were engaged with a peer or an audio-tape recording. additionally, therrien how reading volume affects both reading fluency and reading achievement / allington 15 reports that using instructional level texts as opposed to the more difficult grade level texts also produced faster and larger student fluency gains. however, while repeated reading activities are more powerful in fostering fluent reading than are word identification in isolation activities, it also seems that reducing time spent engaging in repeated readings and using that time to engage students in wide reading is an even more powerful option than offering repeated readings activities alone. this is the major finding from a recent series of studies of by kuhn and her colleagues (kuhn, 2005, kuhn, et al, 2006; schwanenflugel, et al, 2006; 2009). in this work they compared use of their wide reading fluency intervention with the traditional repeated reading intervention. much like earlier studies (e.g., homan, et al, 1993) they found that reducing the time spent on repeated readings while extending the time spent reading new texts developed fluency faster and developed both word recognition and comprehension better than a steady diet of repeated readings. reviewing primarily their previous studies, kuhn, schwanenflugel and meisinger (2010, p. 232) argue, "to move beyond this serial processing and toward the autonomous word recognition entailed by fluent reading, learners require the opportunity for extensive practice in the reading of connected text.” in other words, while repeated readings activities typically expand the volume of reading that student do (as compared to the more traditional skills in isolation work provided by worksheets and skills drills), simply expanding not only the volume of reading but also expanding the numbers of texts students read fosters fluency development faster. improving reading fluency by expanding student reading volume is predicted by “instance theory” (logan, 1988). logan explained instance theory in this way: "the theory makes three main assumptions: first, it assumes that encoding into memory is an obligatory, unavoidable consequence of attention. attending to a stimulus is sufficient to commit it to memory. it may be remembered well or poorly, depending on the conditions of attention, but it will be encoded. second, the theory assumes that retrieval from memory is an obligatory, unavoidable consequence of attention. attending to a stimulus is sufficient to retrieve from memory whatever has been associated with it in the past. retrieval may not always be successful, but it occurs nevertheless. encoding and retrieval are linked through attention; the same act of attention that causes encoding also causes retrieval. third, the theory assumes that each encounter with a stimulus is encoded, stored, and retrieved separately. this makes the theory an instance theory..." (p. 493) as children read they encounter words, if these words are correctly pronounced then a useful “instance” has occurred. thus, efforts to expand reading volume need to ensure that students are reading texts with a high level of accuracy. what we’ve learned in the past 25 years is that it takes very few “instances” of correctly pronouncing a word before it becomes readily recognized when next encountered. instance theory underlies the “self-teaching hypothesis” proposed by share (1995; 2004) who has demonstrated that while reading children are actually also acquiring orthographic knowledge of both whole words and word segments. readers use this orthographic knowledge to facilitate pronunciation when they next encounter the same word or an identical word segment occurring in a different word. that is, pronouncing the word segment “ism” in the word racism may assist the reader in pronouncing the word schism that contains the same segment. this sort of self-teaching, which is derived from instance theory, is one mechanism by which reading fluency is achieved. self-teaching is also an important mechanism that supports developing other reading proficiencies, such as vocabulary knowledge (swanborn & deglopper, 1999). international electronic journal of elementary education vol.7, issue 1, 13-26,2014 16 a different role for self-teaching is the development of a core set of words, in skilled readers a huge core of words, that can be pronounced instantly, words that we call “sight words”. the larger the number of words that can be instantly recognized is in large part what separates skilled readers from developing (or emergent) readers. the ability to recognize many words with little conscious effort also underlies the ability to read aloud with fluency. shany and biemiller (1995) provide one example where self-teaching seems to have occurred. they studied the effects of teacher-assisted reading and tape-assisted reading on reading achievement. the study consisted of three groups: one control group and two experimental groups. one experimental group received 30 minutes of extra reading practice with adult assistance (pronouncing any mispronounced words) while the other experimental group received 30 minutes of extra reading practice with audio-taped recordings of the texts to assist the reading. students in both experimental groups read more books in and out of the classroom than the control group. most subjects "read" through 2.5 years worth of basal stories in 64 days (or 32 hours) of practice! treatment students read 5 to 10 times as many words as the control group students during this 16 week study. (p. 390) shany and biemiller (1995) evaluated different aspects of reading achievement, comparing the two experimental groups to each other as well as to the control group. they found that students in both treatment groups scored significantly higher in reading comprehension, listening comprehension, and reading speed and accuracy, than the control group that completed less reading activity. comparing the treatments, the tape-assisted group scored significantly better in listening comprehension. there were similar gains in reading comprehension, reading speed and accuracy between the two treatments and these gains were higher than those obtained by the control group students. neither treatment improved word identification in isolation, nor decoding proficiency on the woodcock reading mastery test. the authors’, nonetheless, concluded that, "increased reading experience led to increased reading competence." (p. 392) in this study, then, simply expanding the volume of reading, with or without teacher feedback, resulted in improved fluency (as measured by reading rate and reading accuracy) and improved reading comprehension. in other words, the groups that completed the greater volume of reading activity demonstrated a larger gain in reading achievement than the control group students. the potential role of reading volume in daily classroom reading lessons was demonstrated in a large-scale observational study conducted by foorman, schatschneider, eakins , fletcher, moats and francis (2006). they reported that the key factor of the reading instruction offered by over 100 observed 1st and 2nd grade teachers was the time that they allocated to text reading. key because it was this measure of reading volume during reading instruction that explained any variance observed on any of the outcome measures including word recognition, decoding, and reading comprehension. none of other time factors, including time spent on phonemic awareness, word recognition or decoding were related to reading growth. these findings suggest that teachers should design their lessons such that student reading volume is expanded, perhaps by reducing the time planned for other, not very useful, activities that too often replace wider reading. the outcomes from these studies noted above should not be unexpected. torgeson and hudson (2006) reviewed several studies, each which demonstrated that neither improving recognition of words of in isolation nor improving decoding proficiencies improved either reading fluency or comprehension. in other words, reading fluency and reading comprehension develop largely separate from word identification and decoding. in the case of struggling readers, too many have huge deficits in reading volume and therefore huge how reading volume affects both reading fluency and reading achievement / allington 17 deficits in the number of words they can recognize automatically, when compared to their achieving peers. as torgeson and hudson (2006) contend, "the most important factor appears to involve difficulties in making up for the huge deficits in accurate reading practice the older struggling readers have accumulated by the time they reach later elementary school... one of the major results of this lack of reading practice is a severe limitation in the number of words the children with reading disabilities can recognize automatically, or at a single glance... such 'catching up' would seem to require an extensive period of time in which the reading practice of the previously disabled children was actually greater than that of their peers.” (p. 148) if educators hope to improve either the oral reading fluency or the reading comprehension of struggling readers then expanding reading volume, it seems, must necessarily be considered. considered as in evaluating the reading volume of every struggling reader as a first task to complete prior to attempting to design an intervention to address the student’s reading difficulties. an unfortunate characteristic of current models for diagnosing the difficulties some children exhibit with reading acquisition is almost total neglect of any consideration that reading volume deficits are likely a more critical factor than knowledge of the sounds linked to vowel digraphs. while diagnosticians and school psychologists routinely evaluate struggling readers proficiencies with decoding words in isolation and their proficiency with various decoding subcomponents, i have yet to find a single school psychologist who attempted to track and estimate the daily reading volume of students with reading difficulties that they are evaluating. thus, reading volume deficits are largely overlooked when explanations of reading difficulties (or fluency problems) are offered and overlooked in designing intervention lessons to remediate the reading difficulty. reading volume is typically not addressed in individual education plans (iep) developed for pupils with disabilities even though some 80 percent of these students exhibit reading difficulties. thus, we have a series of research reports noting that pupils served by special education programs read less than do general education students (allington & mcgill-franzen, 1989; vaughn, moody & schumm, 1998; ysseldyke, algozzine, shinn & mcgue, 1982; ysseldyke, o’sullivan, thurlow & christenson, 1989) and that struggling readers of all stripes read less during general education classroom reading lessons than do achieving readers (allington, 1983; 1984; hiebert, 1983). outside of daily reading lessons students have other opportunities to expand their reading volume. lewis and samuels (2005) conducted a meta-analysis of 49 studies of providing students with independent reading time during the school day. they concluded that, "no study reported significant negative results; in no instance did allowing students time for independent reading result in a decrease in reading achievement." (p. 13) the overall effect size for the eight true experiments was d=0.42 indicating a moderate and statistically significant effect for volume of reading, they also conducted an analysis of 43 studies that were insufficient for including in the meta-analysis. there were 108 student samples in these 43 studies. of these 108 samples, 85 of the samples were students who improved their reading achievement after participating in some form of an independent reading activity. in fourteen samples there were reported no positive effects on reading achievement, and nine reported negative effects on reading achievement. all of the studies reporting no effects or negative effects on reading achievement were done with older students enrolled in middle or secondary schools. topping, samuels and paul (2007) provide other necessary aspects to consider when attempting to expand the reading volume of students. their analysis of the records of some international electronic journal of elementary education vol.7, issue 1, 13-26,2014 18 45,600 students (primarily k-6 students) drew from the national database compiled by the accelerated reader firm. they report that until quite good reading comprehension (at least 80% comprehension) was achieved the added engagement in reading added little, if any, growth. as topping, et al (2007) note: "the current study suggested that simple information-processing models of reading practice were inadequate. volume of practice is only one relevant variable, and not all practice is the same. pure quality of independent reading practice and classroom placement were as important as quantity of reading practice. theoretical models need to take account of three variables not one, and distinguish between affordances and the extent to which they are actively utilized." (p. 262) topping and colleagues (2007) may have provided us with a basis for explaining why the research on expanding reading activity may seem inconsistent. none of the experimental studies of extensive reading that are available attempted to control for 1) the level of accuracy that was achieved while reading, 2) the level of comprehension of the material read, 3) the variety of texts that are available to subjects, 4) the role of self-selection of texts to be read, or 5) the classroom context of students who participated in the studies. each of these five factors, however, do seem related to the outcomes observed. so we have a research basis for assuming that expanding reading activity will improve reading achievement and reading fluency as well. the repeated readings model is likely to expand students’ opportunity to read and this may be the primary reason for its observed success in developing fluency. simply expanding the opportunities to read seems to generally produce improved reading fluency and reading comprehension (krashen, 2011). thus, perhaps, repeated readings lessons are not actually necessary or can be useful when used for only a short period of time. why many children never acquire fluent reading proficiencies and what to do about it. while the restricted reading volume of struggling readers, when compared to their higher achieving peers, has a strong research base as an important factor in the development, or the lack of development, of reading fluency, there is also evidence that differences in the reading instructional environment, beyond differences in reading volume, may also contribute to dysfluent reading behavior. for instance, many struggling readers read aloud word-by-word with little phrasing or intonation. this sort of dysfluent reading may be the result of being given a text that was simply too difficult given their level of reading development. fluent reading only occurs when oral reading accuracy is high. on the other hand, many struggling readers still read word-by-word even when given a text that they can read quite accurately. these readers seem to have habituated reading as a word-by-word reading performance. thirty-five years ago i published a paper (allington, 1980) documenting the differences observed during oral reading segments of reading lessons in the primary grades. using audio-tapes of the oral reading segments of the reading lessons primary grade teachers provided, i noted that when working with the struggling readers in the classroom (as contrasted with working with the achieving readers), the teachers were more likely to: 1) interrupt the oral reading of struggling readers, 2) interrupt struggling readers more quickly, and 3) after interrupting offer different verbal responses to struggling readers and achieving readers. how reading volume affects both reading fluency and reading achievement / allington 19 these differences were actually quite striking with almost every miscue made by struggling readers resulting in an immediate teacher interruption while many miscues made by achieving readers produced no response from the teacher. when teachers responded to achieving reader miscues they typically targeted sense-making or simply rereading the sentence. teacher responses to struggling readers typically targeted letters or sounds and rarely targeted sense-making. perhaps, i argued, these differences in teacher responses to miscues occurred because the point at which the teacher interrupted the two groups readers (achieving and struggling) differed. for achieving readers the most common point of teacher interruption, when an interruption was observed, came at the end of the sentence that was being read when the miscue occurred. for struggling readers the most common point of interruption was the utterance of an incorrect word or letter sound. hoffman, et al (1984) later reported that immediate interruptions had a detrimental impact on students’ reading performances when compared to other, more delayed interruption options. i have argued elsewhere (allington, 2009) that the common pattern both hoffman and his colleagues (1984) and i observed, interrupting struggling readers immediately when they miscue, creates both passive and non-reflective readers as well as word-by-word readers. i suggest that the continued use of such interruptive practice will stymie all attempts to produce reading fluency. creating a non-interruptive reading environment. what we are attempting to produce is active and reflective silent readers that is, readers who are engaged with the story and who notice when they miscue and then attempt to self-correct their miscue. but an immediate teacher interruption after an oral reading miscue undermines both of these goals. interruptions always interfere with reading engagement and prompts to “sound it out”, to “look at the first letter”, or asking “what is the sound of the vowel” take attention away from making sense of what was read. perhaps a steady diet of immediate interruptions and letter and sound focused prompts actually foster the non-reflective and word-by-word reading so commonly observed with struggling readers. it is with these struggling readers who read word-by-word even when they are reading accurately that repeated readings can be an effective solution. perhaps this is because in most cases the repeated readings are done without a teacher interrupting to “correct” each miscue. without teacher interruptions students read along with greater fluency. there is no need to read slowly and to look up at the teacher when you encounter a word that is unknown. assuming the text is being read with a high level of accuracy, it also means that more instances of correct word identification are accumulating. every instance of correct pronunciation leads to another trace on the reader’s brain that will make the response to the next encounter of that word more likely a correct response. the point is this, if we want to foster fluent reading then we need to create an instructional environment where fluent reading is fostered not suppressed. shifting away from immediately interrupting students when they miscue on a word and moving towards a delayed response that focuses on making sense rather than on surface level characteristics of the misread word will both foster the development of fluent and reflective reading. adopting what i have dubbed the pause-prompt-praise (p-p-p) interaction pattern while listening to students reading aloud is one strategy for becoming a more positive influence on students struggling with fluency. in the p-p-p pattern the teacher waits until the end of the sentence when a student is reading aloud and misreads a word. when the student has reached the end of the sentence, the teacher simply asks. “does that sentence that you just read make sense to you?” or, “did that sentence sound right to you?” the goal is to stimulate self-regulation – the ability to monitor one’s own reading. self-monitoring is central to the international electronic journal of elementary education vol.7, issue 1, 13-26,2014 20 development of fluent reading and self-monitoring is central to self-correcting responses when oral reading miscues occur (clay, 1969). breakout box pause – wait until the reader had finished reading the sentence before you interrupt and call attention to the miscue. prompt – the key prompt you want to make is to draw attention to making sense while reading. praise -two possibilities here—praise making sense or praise the effort to make sense. if you want to foster better use of available decoding knowledge, fine, but not in the middle of an oral reading segment. note the miscue and after the reading segment is completed you can discuss the appropriate decoding strategy the child might have used. many struggling readers do better with decoding in isolation than decoding words while reading. enhancing reading volume by expanding access to texts. once you have created a noninterruptive classroom reading environment you can focus on developing a classroom where all students can locate books they really want to read and can read with a high level of accuracy, say with 98% words correctly pronounced or higher (allington, mccuiston & billen, in press). this typically means you will need to develop a classroom library of books that provide texts across the range of reading levels and interests of students in your classroom. when considering the range of difficulty of the texts you will need in your classroom library remember that, as hargis (2006) demonstrated so powerfully, that in second-grade you can expect to have some children still reading at the very beginning reading levels (e.g., primer, first reader) and some children who can read fourthand fifth-grade texts. by fourthgrade this gap between your best and worst readers widens even further with some children reading at the first-grade level and others at the ninth-grade level! 5 -----------------------------( )----------------------- 4 --------------------------( )------------------------- ------------ 3---------------------( )---------------------------------------------- 2----------------. ( )---------------------------------- 1 ----( ) ----------------------------- 1 2 3 4 5 6 7 8 reading grade equivalent on the left side of the graph is the student current grade placement level. across the bottom are the grade level equivalencies. the arrowhead on the left indicates the lowest scoring children and the range of scores for the lowest 25% of students. the arrowhead on the right indicates the reading level of the top scoring students and the length of the arrow indicates the range of reading proficiency of the top 25% of students. the area between the brackets is the performance of the middle 50% of students. ________________________________________________________________ figure 1. range of reading levels found typically in american elementary classrooms (developed from the data in hargis, 2006) how reading volume affects both reading fluency and reading achievement / allington 21 as illustrated in figure 1 the range of reading proficiency widens as children go through the elementary school year. the data in the figure showing the range of reading proficiency at each grade level is a good guide as you develop your classroom library. the breadth of proficiency levels at each grade is why you should plan on acquiring 1,000 individual titles for your classroom library. classroom libraries provide children with easy access to a range of books that have been selected at appropriate levels of difficulty. classroom access to books is especially important in schools where many children live in poverty. classroom access is important because so many poor children own not a single book, much less have a home library such as the ones you can find in many middle class homes. the number of books in the home is a powerful and significant predictor of children’s reading achievement (schubert & becker, 2010), even when family income, parental education, language used in the home and other factors are controlled. in a 27 nation international study with over 70, 000 cases evans, kelly, sikora and treiman (2010) report that the number of books in the home, after controlling for ses, father’s occupation, and parental education they reported that the effect of home access to books was about the same as parental education, twice as large as father’s occupation, and stronger than family ses. it is children from low-income families that routinely lack access to books. they rarely have home libraries of books. they live in neighborhoods where few books are available, either to purchase or to check out of a community library. worse still, in the schools they attend both the school library and the classroom libraries have far fewer books than are found in middle class schools and libraries (neuman & celano, 2012; pribesh, gavigan & dickinson, 2011)). the differences in the availability of children’s books are striking. neuman and celano (2012) report that there were 358 books for sale at the four stores that carried children’s books in the high-poverty neighborhood they studied. at the same time, in a nearby middleclass community there were 16,453 children’s books available for purchase. of course, these communities differed not just on average family income but also in the numbers of books available for purchase from merchants who sold children’s books. school libraries in these two communities – one poor and the other not poor – followed the same pattern. there were 26 books per child available in middle-class school libraries but half that number available per child (13) in the school libraries located in high-poverty communities. pribesh and colleagues (2011) extend this finding and note that schools attended by children from higher-income families purchase more than twice as many books for the school libraries as do schools enrolling mostly children from low-income families. access to books is, of course, linked to voluntary reading activity (mcquillan & au, 2001). and, no matter how you look at the issue, poor children have substantially more limited access to books than do middle-class children.. but when you live in a “book desert,” as do too many children from low-income families, one should not expect that these children will engage in much voluntary reading. increasing children’s access to books has been shown to have dramatically positive effects on reading growth and achievement (lindsay, 2013). yet, even with this body of research establishing that the children from low-income families have restricted access to books and that altering the situation so that ease of access to books is improved for lowincome children improves their reading achievement we largely ignore the data and attribute the limited proficiencies in reading among poor children to other factors and then focus on those other factors when designing our interventions! international electronic journal of elementary education vol.7, issue 1, 13-26,2014 22 conclusion given the research evidence linking volume of reading to reading achievement and oral reading fluency it seems surprising that american commercial core reading programs only provide roughly 15 minutes of daily reading activity (brenner & hiebert, 2010). that means that in too many classroom children have 75 minutes daily to listen to the teacher or to complete low-level worksheets instead of actually reading. given the findings of foorman, and her colleagues (2006) that the sole aspect of reading lesson design that was related to reading achievement was the volume of reading done during the lessons, it is undoubtedly time to reconsider the use of such programs as a central characteristic of american reading lessons. finally, given that the latest survey of adult reading habits (national endowment for the arts, 2007) reports that young american adults (ages 18-24) read less than any other age group and read less today than ever before, it seems that a substantive effort to promote greater voluntary reading, both in and out of school, is needed. we know much about reading instruction that fosters fluency and comprehension. the design of our reading programs and reading lessons must begin to reflect what we know. • • • richard l. allington is professor of literacy studies at the university of tennessee and is a pastpresident of the international reading association and the literacy research association. he received the outstanding dissertation award and the william s. gray citation of merit from ira for his contributions to the profession. dick has been twice co-recipient of the albert j. harris award from ira in recognition of his research contributing to the understanding of reading and learning disabilities, and has been elected to the reading hall of fame. dick currently serves on several editorial boards including reading research quarterly, the journal of educational psychology, remedial and special education, language arts and the journal of disabilities policy. he is author of several books, including what really matters for struggling readers (pearsonallynbacon), the handbook of reading disability research (routledge) co-edited with anne mcgill-franzen and most recently, also with mcgill-franzen, summer reading: closing the rich/poor reading achievement gap (teachers college press) and with michael pressley, reading instruction that works: the case for balanced teaching (guilford). references allington, r. l. (1980). teacher interruption behaviors during primary grade oral reading. journal of educational psychology, 72, 371-377. allington, r. l. (1983). the reading instruction provided readers of differing abilities. elementary school journal, 83, 548-559. allington, r. l. (1984). content coverage and contextual reading in reading groups. journal of reading behavior, 16(1), 85-96. allington, r. l. (2009). what really matters in response to intervention: research-based designs. boston: allynbacon. how reading volume affects both reading fluency and reading achievement / allington 23 allington, r. l., mccuiston, k., & billen, m. (in press). what research says about text complexity and learning to read. reading teacher. allington, r. l., & mcgill-franzen, m. (1989). school response to reading failure: chapter 1 and special education students in grades 2, 4, & 8. elementary school journal, 89(5), 529-542. brenner, d., & hiebert, e. h. (2010). if i follow the teachers' editions, isn't that enough? analyzing reading volume in six core reading programs. elementary school journal, 110(3), 347-363. cipielewski, j., & stanovich, k. e. (1992). predicting growth in reading ability from children's exposure to print. journal of experimental child psychology, 54(1), 74-89. clay, m. m. (1969). reading errors and self-correction behaviour. british journal of educational psychology, 37(1), 47-56. dahl, p. r., & samuels, s. j. (1977). an experimental program for teaching high-speed word recognition and comprehension skills. in j. button, t. lovitt & t. rowland (eds.), communications research in learning disabilities and mental retardation. (pp. 33-65). baltimore: university park press. evans, m. d. r., kelley, j., sikora, j., & treiman, d. j. (2010). family scholarly culture and educational success: books and schooling in 27 nations. research in social stratification and mobility, 28(2), 171-197. foorman, b. r., schatschneider, c., eakins, m. n., fletcher, j. m., moats, l., & francis, d. j. (2006). the impact of instructional practices in grades 1 and 2 on reading and spelling achievement in high poverty schools. contemporary educational psychology, 31(1), 1-29. hargis, c. (2006). setting standards: an exercise in futility? phi delta kappan, 87(5), 393-395. hiebert, e. h. (1983). an examination of ability grouping for reading instruction. reading research quarterly, 18, 231-255. hoffman, j. v., & clements, r. (1984). reading miscues and teacher verbal feedback. elementary school journal, 84(4), 423-439. homan, s., klesius, p., & hite, s. (1993). effects of repeated readings and non-repetitive strategies on students' fluency and comprehension. journal of educational research, 87(1), 94-99. krashen, s. (2011). free voluntary reading. santa barbara, ca: libraries unlimited. kuhn, m. r. (2005). a comparative study of small group fluency instruction. reading psychology, 26(2), 127-146. kuhn, m. r., schwanenflugel, p., morris, r. d., morrow, l. m., woo, d., meisinger, b., et al. (2006). teaching children to become fluent and automatic readers. journal of literacy research 38(4), 357388. kuhn, m. r., schwanenflugel, p. j., & meisinger, e. b. (2010). aligning theory and assessment of reading fluency: automaticity, prosody, and definitions of fluency. reading research quarterly, 45(2), 230251. kuhn, m. r., & stahl, s. a. (2003). fluency: a review of developmental and remedial practices. journal of educational psychology, 95(1), 3-21. laberge, d., & samuels, s. j. (1974). toward a theory of automatic information processing in reading. cognitive psychology, 6, 293-323. lewis, m., & samuels, s. j. (2005). read more, read better? a meta-analysis of the literature on the relationship between exposure to reading and reading achievement. university of minnesota. lindsay, j. j. (2013). impacts of interventions that increase children’s access to print material. in r. l. allington & a. mcgill-franzen (eds.), summer reading: closing the rich/poor reading achievement gap. (pp. 20-38). new york: teachers college press. international electronic journal of elementary education vol.7, issue 1, 13-26,2014 24 logan, g. d. (1988). toward an instance theory of automatization. psychological review, 95(4), 492527. mcquillan, j., & au, j. (2001). the effect of print access on reading frequency. reading psychology, 22(3), 225-248. mol, s. e., & bus, a. g. (2011). to read or not to read: a meta-analysis of print exposure from infancy to early adulthood. psychological bulletin, 137(2), 267-296. morgan, p. l., sideridis, g., & hua, y. (2012). initial and over-time effects of fluency interventions for students with or at risk for disabilities. journal of special education, 46(2), 94-116. national endowment for the arts (2007). to read or not to read: a question of national consequence. washington, dc: office of research and analysis. national reading panel (2000). teaching children to read: an evidence-based assessment of the scientific research literature on reading and its implications for reading instruction. downloaded from: http://www.nichd.nih.gov/research/supported/pages/nrp.aspx/ neuman, s. b., & celano, d. c. (2012). giving our children a fighting chance: poverty, illiteracy, and the development of information capital. new york: teachers college press. pribesh, s., gavigan, k., & dickinson, g. (2011). the access gap: poverty and characteristics of school library media centers. library quarterly, 81, 143-160. rasinski, t., rikli, a., & johnston, s. (2009). reading fluency: more than automaticity? more than a concern for the primary grades? literacy research and instruction, 48(4), 350-361. samuels, s. j. (1979). the method of repeated reading. reading teacher, 32, 403-408. schreiber, p. a. (1991). understanding prosody's role in reading acquisition. theory into practice, 30(3), 158-164. schubert, f., & becker, r. (2010). social inequality of reading literacy: a longitudinal analysis with crosssectional data of pirls 2001 and pisa 2000 with the pairwise matching procedure. research in social stratification and mobility, 29(1), 109-133. schwanenflugel, p. j., kuhn, m. r., morris, r. d., morrow, l. m., meisinger, e. b., woo, d. g., et al. (2009). insights into fluency instruction: shortand long-term effects of two reading programs. literacy research and instruction, 48(4), 318-336. schwanenflugel, p. j., meisinger, e. b., wisenbaker, j. m., kuhn, m. r., strauss, g. p., & morris, r. d. (2006). becoming a fluent and automatic reader in the early elementary school years. reading research quarterly, 41(4), 496-523. shany, m. t., & biemiller, a. (1995). assisted reading practice: effects on performance for poor readers in grades 3 and 4. reading research quarterly, 30(3), 382-395. share, d. l. (1995). phonological recoding and self-teaching: sine qua non of reading acquisition. cognition, 55(2),151-218. share, d. l. (2004). orthographic learning at a glance: on the time course and development onset of self-teaching. journal of experimental child psychology, 87(4), 267-298. swanborn, m. s. l., & deglopper, k. (1999). incidental word learning while reading: a meta-analysis. review of educational research, 69(3), 261-286. therrien, w. j. (2003). fluency and comprehension gains as a result of repeated reading: a metaanalysis. remedial and special education, 25(4), 252-261. topping, k. j., samuels, s. j., & paul, t. (2007). does practice make perfect? independent reading quantity, quality and student achievement. learning and instruction, 17, 253-264. how reading volume affects both reading fluency and reading achievement / allington 25 torgesen, j. k., & hudson, r. f. (2006). reading fluency: critical issues for struggling readers. in s. j. samuels & a. e. farstrup (eds.), what research has to say about fluency instruction. (pp. 130-158). newark, de: international reading association. vadasy, p. f., sanders, e. a., & peyton, j. a. (2005). relative effectiveness of reading practice or wordlevel instruction in supplemental tutoring: how text matters. journal of learning disabilities, 38(4), 364-380. vaughn, s., moody, s. w., & schumm, j. s. (1998). broken promises: reading instruction in the resource room. exceptional children, 64, 211-225. ysseldyke, j. e., algozzine, r., shinn, m. r., & mcgue, m. (1982). similarities and differences between low achievers and students classified as learning disabled. journal of special education, 16(1), 7385. ysseldyke, j. e., o'sullivan, p. j., thurlow, m. l., & christenson, s. l. (1989). qualitative differences in reading and math instruction received by handicapped students. remedial and special education, 10(1), 14-20. international electronic journal of elementary education vol.7, issue 1, 13-26,2014 26 www.iejee.com this page is intentionally left blank international electronic journal of elementary education, 2015, 7(3), 319-332. issn:1307-9298 copyright © iejee www.iejee.com pre-school students’ informal acquisitions regarding the concepts of point and straight line keziban orbay  amasya university, turkey mehmet hikmet develi̇ amasya university, turkey received: 25 february 2015 / revised: 5 april 2015 / accepted: 18 may 2015 abstract this study aimed to investigate the informal cognitive structures regarding “point” and “straight line” -two basic and undefined terms of geometryin children registered in preschool – the previous step before in-class formal education process. the study was conducted with the participation of 50 children enrolled in nursery, kindergarten and preschools of a total of five educational institutions -three public and two privatein a city which is in the middle of the turkey. the qualitative research model was utilized in the study since observing, analyzing and assessing children’s intuitive thinking and informal knowledge construction process would be difficult and good results would not be obtained via quantitative research methods. data were collected through clinical interview technique. results show that children, in general, possess major and to a large extent correct acquisitions that would be the basis of subsequent formal concept development process in children. keywords: qualitative research, clinical interview, informal acquisition, point, straight line. introduction in mathematics teaching, it is known that children’s intuitive learning especially in preschool and first years of primary school provide an important foundation for future learning. common opinion of many researchers (ausubel, gagne, piaget and others) who generated the theoretical background related to this issue is: “when suitable learning environments are created, it is easier for the cognitive acquisitions (cognitive structures) obtained through children’s intuitions to construct and transfer knowledge. piaget argues that intuitive thinking starts at the beginning of year four (ülgen, 1999). it is well known that modern mathematics teaching approaches are usually shaped based on this fundamental view.  keziban orbay, amasya university, faculty of education, department of elementary education, amasya, phone: (+90358)2526230, e-mail: keziban.orbay@amasya.edu.tr http://www.iejee.com/ international electronic journal of elementary education vol.7, issue 3, 319-332,2015 320 piaget stated that children construct knowledge themselves and this process called “adaptation” includes various sub-processes (assimilation, accommodation and equilibration) (altun, 2010). in his studies related to concept formation, piaget emphasized that analysis of the existing informal acquisitions should precede the formal formation process of a concept and stated that children’s formal acquisitions are built upon these informal acquisitions (altun, 2010; günçe, 1973). piaget also remarked that children construct knowledge on their own based on their own anatomical structures and through interactions with the environment and that especially language development and concept development go hand in hand (ülgen, 2001). based on constructivist approach, current study aimed to observe the existing informal cognitive structures in children regarding the concepts of point and straight line -the two basic and undefined concepts of geometryduring the preschool period which is provided immediately before in-class formal education process and to discuss and interpret the contributions of this structure to formal education and training process. according to piaget and various clinical psychologists, it is rather hard and sometimes impossible to reconstruct misconceptions in children’s cognitions (ülgen, 2001). children may not be able to ensure equilibrium between misconceptions and the correct constructs presented in the classroom and may face dual concept formation (one created by the children and one presented to the children), misconceptions and incomprehensibility in the future (ülgen, 2001). therefore, the current study hopes to observe possible misconceptions in children as well. it is necessary to point out here that this study does not intend to comprehend how cognitive constructs related to point and straight line concepts are generated in children. the study focuses on and is limited to observing and assessing the existing constructs. any scientific research –regardless of its field– is conducted with the help of a specific method or more than one method that complements one another. various classifications are possible for different types of scientific research. one of these classifications distinguishes types of research as “descriptive”, “relational” and “experimental” (karakaya, 2011). two major research methods in the field of mathematics teaching are “quantitative research” and “qualitative research” methods. quantitative research is generally based on numerical data about mathematical knowledge and skills and is used to determine level/degree. on the other hand, the qualitative research method is used to observe the existing states in individuals and obtain and interpret general impressions that are not based on numerical data (cemaloglu, 2011). qualitative research model current study is a descriptive qualitative research and information obtained from the related literature regarding the appearance, development, principles and techniques (alternative methods) of qualitative research method and suggested action research types are provided below in a partly chronological manner. the research is carried out in accordance with this model. paul ernest defines qualitative research method as an important method whose developmental process is ongoing, uses the term “model” in the place of “method” and states that qualitative research method was first used in social research. he also points that qualitative research was first shaped in mathematics education research by erlwanger (ernest, 1998). ernest addresses the developmental process of qualitative research model as composed of three periods: “rationalist period”, “modern period” and “post-modern pre-school students’ informal acquisitions / orbay & develi 321 period”. he also argues that rationalist period started with descartes and the modern period commenced with piaget (ernest, 1998). piaget started his qualitative research in 1920’s (ginsburg, 1981). piaget employed a clinical interview technique in his research to comprehend small children’s cognitive constructs of and how knowledge is constructed in the minds of children. for long years, piaget used standard tests that utilized this technique but after the 1950’s, he tried to integrate some flexibility in the technique (ernest, 1998; ginsburg, 1981). starting with 1970’s, this model and post-modern research that highly contributes to the development of epistemology are more prominent. erlwanger, rorty, gardes, gardner, zoslovsky, ginsburg, ernst von glasersfeld, erickson, croswell, goldin, silverman and kilpatrick can be cited among the prominent researchers in the post-modern era. some of these studies focused on “psychology of learning” whereas others focused on “problemsolving” (baki, karataş & güven, 2002; ekiz, 2004; ernest, 1998; ginsburg, 1981). clinical interview technique the clinical interview is regarded as the most suitable technique to observe and interpret existing acquisitions of small children and suggest future measures. however, it is stated that this technique is rather difficult and risky; it requires diligence in terms of reliability and validity and in order to increase validity and reliability, it is necessary to employ the technique by freeing it from obligatory standards and to administer it in a flexible manner in the form of semi-structured activities (çepni, 2007; ekiz, 2004; ginsburg, 1981). therefore, the sequence of experiences for the activities should be the same for each interviewer but questions and stimulants should be systematic and somewhat impromptu according to the atmosphere, children’s desire for synergy and the developed empathy and dialogue. success mostly comes from researcher’s ability to empathize and his/her research experiences (ekiz, 2004; ernest, 1998).activity instructions and observation forms should be prepared beforehand for clinical interviews and sequential experiences should be realized by “working together” with children and should be immediately recorded. according to the above information, this type of activity can be cited as “structured fieldwork” (cemaloğlu, 2004). the items below should be taken into consideration during the interviews in the light of the relevant literature:  doing interviews while seated at a table may not provide best results. therefore, children’s natural environments (floor, play corner, etc.) should be preferred. children may present some cognitive skills in their natural environments while they may not be able to display them in artificial environments (such as interview rooms) (ginsburg, 1981).  interviews should be carried out in accordance with the anatomical structure of the children and their current affective states (excitement, use of the left hand, style of using materials etc.) (ernest, 1998; goldin, 1998).  it is possible for children to provide unexpected, surprising and interesting answers during interviews. in such cases, it is necessary to focus on the answer during the interview of in the upcoming interviews and impromptu questions should be carried in an order of increasing accuracy to comprehend the actual cognitive constructs (bacanak, 2008). international electronic journal of elementary education vol.7, issue 3, 319-332,2015 322  children should be given ample time to give their answers and to present their answers by doing whereas no interventions should be provided other than small tips (if necessary).  children should feel free to ask questions when they do not understand the questions and when they need clarification or explanations.  when the children are unwilling to work together, when they feel disinterested in any part of the interview or when they lose motivation, it is not wise to commence or continue the interview.  when a question is unanswered, the interviewer can move on to next question provided that willingness and motivation still exist (ginsburg, 1981). method based on the rationale presented in introduction, it was decided to undertake the study in the form of one-on-one interviews with the children in an appropriate environment and to use written observation forms only with the concern that voice recording or video recording may distract 4-6 year-olds although voice or video recording is generally suggested (bacanak, 2008; clarke, 1998; ekiz, 2004). creating the hypothesis gestalt theory which was started at the beginning of 1990’s by wertheimer, kafka and köhler; and was evolved after the years of 1990’s especially in america and has still been evolving is a teaching and learning theory that was formed against behaviorist’s warning response theory. this theory is also against constructivist theory’s in-depth analysis that reduces to elements of mind (senemoglu, 2002; yıldırım, 2008; schunk, 2009). gestalts advocate that introspection method of structuralists is a suitable method for examine learning case however, it is used wrong. they assert that it is better to handle mental experiences organized as a whole instead of analyzing segmentally (senemoglu, 2002). as discussed in the introductory part, the observation of a configured shape as intuitively is targeted, not the way of how the concept of points and lines is shaped in children. it is understood that it is configured the point as a small circular track and the line as a flat and solid line. in addition, this situation can be observed and analyzed at a satisfactory level with using proper materials. it was supposed that “children between the ages of 4–6 may have informally perceived the point as a very small circular trace and the straight line as a straight and unbroken line”. based on this hypothesis, materials to work on point and straight line concepts and “instructions” and observation forms” for interviews were created. materials for working on the concept of point: blank papers, papers presenting single colored very small points (zero-dimensional), as a distractor somewhat larger points (small filled circles), small scale circles, filled or empty triangles, alike squares and very short lines. for working on the concept of straight line: blank and lined (squared) white papers, papers presenting black colored lines, curves, broken lines, disconnected lines etc., strings, rubbers and short bent plastic rods (3–4 cm). pre-school students’ informal acquisitions / orbay & develi 323 instructions sequential experiences designed for the concept of point: • the child is provided with a blank paper and pencil and asked to form points on the paper. the child is asked to compare actual formations by drawing attention to possible wrong formations (which one is better? which one looks more like a point? is it like the one you make with your finger? etc.) • the paper that includes points and distractors are placed on the table and the child is asked to point to the points. possible wrong selections and correct selections are compared and questions are directed about the differences between them (does it look like the one you drew with the pencil? what is the difference? can you try one more time? etc.) • the researcher draws a simple shape on a blank paper (triangle, circle, square etc.) and the child is asked to create a similar one by using points. • the child is congratulated at the end of the activity. the child is expected to answer questions such as “yes, so what is a point?”, “what does it look like?” and to point to the points. • the researcher records his/her observations and the dialogs for each sequential experience in the observation form with care and diligence. sequential experiences designed for the concept of straight line: • a similar discussion platform is created by mentioning the term “straight line”. • the child is provided with a blank paper and pencil and asked to form/draw several straight lines on the paper. the child is asked to compare actual formations by drawing attention to possible wrong formations (do you think that part of the line is fine? can you see the difference between them? how would you draw the best one if you did it again? shall we do it again? etc.). also, the reactions are recorded. • the child is provided with a lined (squared) blank paper and asked to draw several straight lines. whether the child takes the lines on the paper as a reference is noted and recorded. • a white colored paper with straight lines, curves, broken and disconnected lines is presented to the child and he7she is asked to select the correct ones among the lines. a dialog similar to the one experienced for the concept of points is generated for possible wrong selections. if all selections of the child are correct, he/she is asked to make comparison with a sample that is wrong (for instance, why did not you choose this one? what would you say if i selected this one? etc.). based on the answers, question-answer session continues. • the child is provided with a piece of string and asked to form a line using the string. he/she is asked to do the same with the rubber. whether the child uses both hands and whether the materials are stretched is observed for both materials. if the child is not stretching the materials sufficiently, he/she is asked to do so and asked questions about the difference between both cases. the researcher may intervene when the string is stretched and asks the difference between the two conditions (what happened now? what should we have done? what should i do? what happens if i do this? etc.). then the researcher presents 6-7 rods with some curved parts and asks the student to use three rods to make a straight line by placing them side by international electronic journal of elementary education vol.7, issue 3, 319-332,2015 324 side. questions are asked about possible wrong selections (is this rod ok? is it fine now? what should we do? etc.) • finally, last ideas are collected with questions such as “yes, so what is a straight line?” and “how do we make a straight line?” if necessary, actions are used to present the ideas. • observations and impressions during the sequential experiences are recorded in the observation form in the same way it is handled for the point concept. testing the hypothesis based on a consensus with their teachers, 10 children (one female and one male student from each school) between 50-70 month chronological age were selected from the nursery, kindergarten and pre-school classes of the five schools. the schools were contacted beforehand to obtain necessary permits for the study. two researchers visited the identified schools. the study was conducted with the selected students in a separate location (in a separate corner of the class) away from the rest of the classroom and necessary notes were taken. based on the impressions obtained during the hypothesis testing phase, required changes in the instructions for the general implementation, the manner of getting together with the children at the beginning of the study and the necessary actions to determine effective readiness (suppress excitement, increase curiosity, generate willingness etc.) were identified. since satisfying levels of empathy were established with the children during test hypothesis phase, researchers felt that the test was successful and experienced selfesteem for the actual implementation. it was also believed that experienced gained in a similar qualitative research (develi & orbay, 2002) would support the implementation. establishing the actual working group 20 children who completed year 4 and 20 children who completed year 5 were selected based on teacher views from the identified schools by taking their chronological age into consideration and ensuring balanced gender distribution. in each school, all the children with prior nursery school experience were included in the working group. instructions for the activities were reorganized in line with the observations obtained during the hypothesis testing phase. materials were improved and finalized for the implementation. results the implementation was carried out in 10 working days in the identified schools. two researchers worked in separate environments by dividing the number of children among themselves. insufficient motivation, problems in the flow of sequential experiences due to various reasons, negative reactions (sudden silences, shrugging etc.) and unwillingness were observed just a few times (in 4 children). interviews were stopped with these children and other children were included in their places. almost all the children started the interviews with high motivation at first maybe due to the promised reward. all children behaviors during interviews, their surprising comments, interesting dialogs that took place and their drawings were collected diligently for assessment. 36 of the 40 children that participated in the implementation provided fun and challenging, surprising and interesting dialogs and displayed amazing actions. in order to clarify the data analysis, some of these dialogs were given as examples below and the first example was detailed. pre-school students’ informal acquisitions / orbay & develi 325 example 1 researcher: k.o. child friend: z. b. g. k.o.: welcome, my friend (they shake hands). my name is k.o. can i learn your name? z.b.g.: z.b.g. k.o.: z.b., now we are going to undertake a very entertaining activity with you. i believe we will be successful. you know there is a reward at the end! z.b.g.: ok! k.o.: z.b., can you tell me what a point is? z.b.g.: circle k.o.: ok, can you also tell me how a point looks like? for instance, if i ask you to show me with your finger! z.b.g.: the child taps the low table with the tip of is/her index finger: tap! tap! k.o.: well done! congratulations! now can you make a few points on this paper with this pencil? z.b.g.: the child is carefully making points by using the tip of the pencil (figure 1). figure 1. marking points k.o.: nice job! congratulations! now i will give you a paper. there are some marks on it. can you show me which of these marks are points? it is sufficient to point with your finger! z.b.g.: ok! (the child points to all objects shaped like the following “. , ., o”.) k.o.: “if i told you to select only one of them”, which one would you choose as the point? z.b.g.: the child selects the “.”. k.o.: why did not you select “o”? z.b.g.: because it is blank inside. k.o.: then why did not you select “.”? z.b.g.: it is very big! k.o.: we are getting closer to the end of the point work, z.b.! now i will draw a figure for you. can you do the same for me with points, i mean by using points? z.b.g.: i can! k.o.: here is a figure for you. do it and we will see! international electronic journal of elementary education vol.7, issue 3, 319-332,2015 326 z.b.g.: hmm, this is a triangle! it is easy! (the child finishes figure 2 in a short time) figure 2. drawing triangles by using points k.o.: great work! so, what is the point? how is it then? z.b.g.: just like that! (the child shows the shapes on the paper other than “o” and touches them with the tip of her finger) k.o.: applauds from me! congratulations! you are very successful! now let’s rest for a while. we will undertake another small activity shortly. it won’t take a lot of your time! your reward is waiting for you! (they rest) k.o.: now let’s move on to the straight line! are you ready? z.b.g.: yes. k.o.: z.b., what is a straight line? z.b.g.: the child does not answer. he/she shrugs. k.o.: ok. z.b., what is a line then? how is it made? z.b.g.: hmm, that! the child draws a line on the floor that looks close to a straight line. k.o.: dear z.b., how is a very straight line then? z.b.g.: the child draws a line with her finger more carefully. k.o.: dear z.b., the last line you drew, the one that is “very straight”, is called a “straight line”. z.b.g.: you mean a road! k.o.: very true! bravo! now, draw a few straight line son this paper with the pencil! z.b.g.: ooo it is very easy! (the child draws the vertical line provided below, figure 3). k.o.: now let’s make another one like this (the researcher roughly points to the horizontal line). z.b.g.: the child draws the line on the paper (provided above) (he/she does not use the lines on the lined paper as a reference, figure 4). pre-school students’ informal acquisitions / orbay & develi 327 k.o.: bravo! bravo! now i will show you another paper that looks like the previous one. i will ask you to select the “straight lines” from the figures. please pay attention dear z.b.! (the child mostly pointed to the straight lines however he/she also marked those) figure 3. vertical line figure 4. horizontal line k.o.: ok, so which ones are correct? these or (the child does not answer, purses lips!) k.o.: dear z.b., are there any differences between these two? pay attention! z.b.g.: has a hole! k.o.: then? z.b.g.: the child hesitantly points to k.o.: why? z.b.g.: because, there shouldn’t be any holes. k.o.: hımm! bravo! z.b., now i will give you a piece of string. can you make a straight line for me using this? z.b.g.: i can! it is easy! (the child holds the stretched string parallel to the floor with her hands.) k.o.: nice! can you also do the same with the rubber? z.b.g.: yes, it is easy! (the child stretches the rubber.) k.o.: we are almost done! one last experiment! (the researcher takes out 6-7 plastic rods some of which are curved). come on! i want you to select 3 of these and connect them like this (displays with both fingers) to make a straight line. you have only three options! pay attention, good luck. z.b.g.: (the child makes the selections) k.o.: don’t you think there is something wrong here? what do you say? international electronic journal of elementary education vol.7, issue 3, 319-332,2015 328 z.b.g.: ha, yeees, the one in the middle does not fit! k.o.: ok. now, select something else instead of it. z.b.g.: the child makes the correct selections and places them where they belong. k.o.: good job! so, what is a straight line then? z.b.g.: it is a very straight line, like this (shows with hands)! k.o.: you are a very successful child. thanks for working with me! would you like a chocolate miss z.b.? (they share laughter, researcher pats her head and they say goodbye) example 2 researcher: h.d. child friend: d.m. h.d.: can you make a shape that is similar to the one i will draw by using points? d.m.: yes! like that. (figure5) figure 5. shape h.d.: why did not you select this? (pointing to ) d.m.: because it is wavy. example 3 k.o.: why did you select this? (pointing to ) z.i̇.: because they are the most beautiful ones! k.o.: why? z.i̇.: looks like a necklace! example 4 h.d.: why did not you select this? (pointing to ) y.ö.: zigzags won’t do! example 5 h.d.: what is a straight line? pre-school students’ informal acquisitions / orbay & develi 329 k.d.: it means it is not wrong! (confuses the terms since turkish word for straight line also means true/correct) h.d.: i did not mean that. how do we draw a straight line? k.d.: ooo, you mean this? just like that (draws with finger). example 6 h.d.: the researcher holds the pencil to the stretched string and forms a bulge and asks, “so, it is ok now?” ç.ö.: nooo! h.d.: why not? ç.ö.: because it cannot have a peak! data analysis as the interviews of the study were conducted as open-ended questions, analyze of the data was executed responsively to the extent allowed by the literature (cemaloğlu; 2011; çepni, 2007). as was expected, it was observed that it required expertise to continue the implementation, from appropriate dialogues, suitably generate the question chain, comprehend what the children mean and make sense of children’s gestures and facial expressions. it was observed that children left some questions unanswered although in our opinion they looked easy to understand and answer or they provided insufficient or incorrect answers to them whereas they were able to answer questions that required higher level competence with unexpected level of correctness and meaning. for instance, it was observed that the child who drew the straight line as the curve was able to select the correct options from among the distracters and another child who selected the curve instead of the straight line was able to identify the fact that the string would be taut while making a line and implement the action as well. although the first step in concept development process is “informal recognition”, the sequential experiences activities we designed purposefully aimed to collect children’s views about point and the straight line, which can be regarded as the informal definition step. this was designed for two purposes. firstly, it was aimed to observe the consistency between children’s ideas and impressions about the concepts of point and straight line and the mechanical formation and use of those concepts during the activities in implementation and secondly, it was aimed to compare and interpret answers to “so what is a point?” and “so, what is a straight line?” provided by the children at the end of activity and their behaviors during the process. as a result of the activities designed with those purposes, it was observed that the majority of childrenother than a few exceptionswas able to define the concepts correctly by pointing to the concepts instead of talking about them and they were also able to relate correct views when they were asked although their ideas did not fully cover the topic. children, in general, perceive the point as a very small circle object. however, sometimes they can select or perceive formations that are larger or full in the inside as points as well. this may be related to their idea of making the point more observable rather than lack of perception or misconceptions. it was observed that children did not identify geometric figures that are full in the inside, that are very small or empty in the inside or that have corners as points. also when children with previous nursery school experiences and children who have learned the figures of triangles, rectangles, squares and circles in preschool classes until the implementation date were asked why they did international electronic journal of elementary education vol.7, issue 3, 319-332,2015 330 not select these figures as points, their answers included statements such as “because it is a triangle…etc” which showed that perception of the concept of point as an object with no dimension (with zero dimensions). children were able to display expected behaviors when they were asked to form simple shapes using points with appropriate tips (form the shape by using points). this competence shaped the opinion that the children were ready to use points as basic geometrical instruments. although there were no significant differences between children who completed year 4 and 5 in terms of recognition and use of points, it was observed that children who completed year 5 needed fewer tips and researcher support. it was seen that the majority of children were not able to answer the question “in your opinion, what is a straight line?”. probably due to the fact that this concept was not introduced as a geometrical thought during class activities at the time of the implementation. but they were able to provide expected answers to the question “what is a line?” even though they mostly pointed to lines while replying. it was identified in the interviews held with the teachers of the children that the children that participated in the implementation were able to recognize and use concepts such as “line” and “rod” during class. this information led us believe that the children actually perceive the straight line as continuous, unbroken and unbent straight line and that their hesitations at the beginning of the activity resulted from lack of familiarity with the words used to describe the concept, not with the concept itself. the majority of children were observed to be able to select the correct option from among the distractors. however, a small minority of children selected the shapes that looked nice to them. when those children were asked to compare their selections with the actual answer, they did not select the shapes that they pointed at first. when they were asked why they changed their minds, they started with their own words that they later realized the discontinuous nature of the shape (“it is broken”, “there can’t be hopes in it” etc.). these statements led the researchers believe that misperceptions were somewhat psychological and sometimes they resulted from carelessness. the children were successful when they were given enough tips (such as use both hands, connect by bringing both ends together etc.) during formation of the straight line with materials (string, rubber, rods). they generally drew the straight line as vertical and this may be related to the implementation of drawing number 1 as a vertical line during preschool classes while learning number 1. children used the line of lined paper during this action. however, when horizontal lines were pointed and they were asked to “draw another one like that”, they were somewhat less skillful and did not use the lines of the paper as reference. this finding points the fact that the concept of lines is perceived intuitively, but intuitive competence was not developed at the point. broken, curved or disconnected lines were not credited with the following reasons: “this has zigzags, this won’t do”, “it is like the sea”, “it is wavy”, “it spreads out from the borders”, “it is disconnected, it has holes”, “it is curvy”, “it is like a mountain (hill)”, “it goes to the side”, “it is skewed”. these statements show that the majority of the children had a cognitive competence to identify the correct shape from among the distractors. results and recommendations study results generated the view that children between the ages of 50-70 months had important and mostly correct informal acquisitions about the concepts of point and straight lines. these informal acquisitions will be the basis of future formal concept development. we believe that similar studies that will be held in similar environments will result in similar findings as well. significant findings were not reached during the study about how the children acquired these perceptions. it is known that, regardless of the concept, it is the hardest part of studies to comprehend and interpret informal acquisitions of children (ernest, 1998). therefore, as mentioned in the introduction, the study did not intend to focus on this area. pre-school students’ informal acquisitions / orbay & develi 331 we believe that the study provides a good example to preschool teachers who undertake semi-formal training activities and especially to first grade teachers about the importance of children’s informal acquisitions and the need to establish formal training on this basis. especially while starting to teach similar concepts; it is suggested for preschool and classroom teachers to carefully use the question-answer technique similar to the one used in the study and to frequently engage in dialogues with students about children’s informal acquisitions and to direct their teaching based on the impressions gained from these interviews and dialogues. it is imperative to achieve high quality in preschool education which is becoming widespread in our country. assessment of the semi-informal acquisition process efficiently will be the precondition of minimizing possible future misconceptions, ambiguities and formation of double concepts. acknowledgements we would like to thank the preschool teachers who provided the researchers with information about the students and made important contributions to the study with their views during the study. • • • references altun, m. (2010). eğitim fakülteleri ve sınıf öğretmenleri için matematik öğretimi, aktüel alfa yayınevi, bursa. bacanak, a. (2008). fen ve teknoloji dersi performans değerlendirme formlarına yönelik oluşturulan web tabanlı programın etkinliğinin araştırılması, yayımlanmamış doktora tezi, trabzon. baki, a., karataş, i̇., güven, b. (2002). klinik mülakat ile problem çözme becerilerinin değerlendirilmesi, v. ulusal fen bilimleri ve matematik eğitimi kongresi, 1043–1049, ankara. cemaloğlu, n. (2011). veri toplama teknikleri nicel-nitel, bilimsel araştırma yöntemleri, 131–164, ed. a. tanrıöğen, anı yayınları, ankara. çepni s. (2007). araştırma ve proje çalışmalarına giriş, üçyol kültür merkezi yayınları, trabzon. clarke, d.j. (1998). studying the classroom negotiation of meaning: complementary accounts methodology, monograph, national council of teachers of mathematics, 9, 98–111, usa. develi, m.h., orbay, k. (2002). i̇şlem öncesi dönem çocuklarında sayı kavramının gelişimi üzerine, v. ulusal fen bilimleri ve matematik eğitimi kongresi, 969–974, ankara. ekiz, d. (2004). eğitim dünyasının nitel araştırma paradigmasıyla i̇ncelenmesi: doğal ya da yapay, www.tebd.edu.tr/arsiv/2004-cilt2/sayı-4/415-439pdf., ankara. ernest, p. (1998). the epistemological basis of qualitative research in mathematics education: a postmodern perspective, qualitative research methods mathematics education, monograph, national council of teachers of mathematics 9 22–39, usa. international electronic journal of elementary education vol.7, issue 3, 319-332,2015 332 ginsburg, h.p. (1981). the clinical i̇nterview in psychological research on mathematical thinking: aims, rationales, techniques, for the learning of mathematics, 3, 4–11. goldin, g. a. (1998). observing mathematical problem solving through task-based i̇nterviews, qualitative research methods education, monograph, national council of teachers of mathematics, 9, 40–62, usa. günçe, g. (1973). çocuk zihin gelişimi piaget kuramına toplu bakış, baylan matbaası, ankara. karakaya, i̇. (2011). bilimsel araştırma yöntemleri, bilimsel araştırma yöntemleri, 55–84, ed.a. tanrıöğen, anı yayınları, ankara. schunk, d., h. (2009). öğrenme teorileri eğitimsel bir bakış, nobel yayın dağıtım, ankara senemoğlu, n. (2002). gelişim öğrenme ve öğretim kuramından uygulamaya, gazi kitapevi, ankara. ülgen, g. (1999). (c.m. charles’den çeviri). öğretmenler i̇çin piaget i̇lkeleri, anı yayınları, ankara. ülgen, g. (2001). kavram geliştirme kuramlar ve uygulamalar (3. baskı), pegem yayınları, ankara. international electronic journal of elementary education, 2015, 7(3), 441-450. issn:1307-9298 copyright © iejee www.iejee.com impacting children’s health and academic performance through comprehensive school physical activity programming timothy a. brusseau university of utah, usa james c. hannon university of utah, usa received: 20 december 2014 / revised: 13 may 2015 / accepted: 19 may 2015 abstract physical activity is associated with numerous academic and health benefits. furthermore, schools have been identified as an ideal location to promote physical activity as most youth attend school regularly from ages 5-18. unfortunately, in an effort to increase academic learning time, schools have been eliminating traditional activity opportunities including physical education and recess. to combat physical inactivity in you, numerous organizations are promoting a comprehensive school physical activity program to encourage academic achievement and overall health. comprehensive school physical activity programs include five components and should be centered around 1) quality physical education, 2) physical activity before and after school, 3) physical activity during school (both recess and classroom activity), 4) staff involvement, and 5) family and community engagement. keywords: physical education, recess, youth, cspap. introduction physical activity has been associated with increases in school performance including concentration, memory, and classroom behavior (i.e. strong et al., 2005). specifically, elementary school aged girls performed better in math and reading when they had additional physical activity time (carlson et al., 2008). furthermore, sallis and colleagues (1999) have stated that increases in school pa opportunities do not hinder academic performance; in fact they suggest the inverse may be true. unfortunately, today’s children not accumulating recommended levels of physical activity (i.e. brusseau, tudor-locke, & kulinna, 2013). physical inactivity is associated with numerous health risks, including heart disease, cancer, diabetes, hypertension, as well as anxiety and depression (kohl & cook, 2013). lee and colleagues (2012) have  timothy a brusseau, university of utah, department of exercise and sport science, 250 s 1850 e rm 205 salt lake city ut 84112 usa. phone: 1-801-809-4400, mail: tim.brusseau@utah.edu http://www.iejee.com/ international electronic journal of elementary education vol.7, issue 3, 441-450,2015 442 suggested that the health burden of physical inactivity approaches that of smoking and obesity. the centers for disease control and prevention (cdc; 2012) has reported that less than half of youth are accumulating the recommended 60 minutes of activity each day. the prevalence of physical inactivity among school aged children has contributed to the substantial rise in overweight and obesity (tremblay & willms, 2003). obesity has increased in the us dramatically over the past ten years with approximately 32% of girls and 37% of boys classified as overweight or obese (ogden et al., 2006). the obesity epidemic is affecting children of all ages including young children and adolescents and when obesity occurs during adolescents, it tends to persist into adulthood (deckelbaum & williams, 2001). physical activity (pa) during childhood has shown to be effective in preventing health problems later in life (strong et al., 2005). pa is associated with strong bones and muscles, decreases in the likelihood of obesity, type 2 diabetes, and heart disease and it promotes positive mental health (usdhhs, 2008). to decrease the risk of overweight and obesity, it is recommended that children engage in moderate to vigorous physical activity (mvpa) for at least 60 minutes each day (usdhhs, 2008) or at least 12,000 steps/day (colley, janssen, & tremblay, 2012). pa peaks at about age 12 (tudor-locke et al., 2011) making the elementary school the ideal time to provide pa opportunity and training. due to the accessibility of children, schools are an ideal setting for promoting pa (cdc, 2011). most children are in school for 30-35 hours per week. however, the opportunity for children to be physically active during school has decreased due to many factors including an increase in vehicle transportation to school, environmental factors, school policy, and reducing time children spend in physical education (pe) and at recess (naspe, 2006). the more access children have to physical activity the more active they will be at school (brusseau & kulinna, 2015). recently, the cdc (2013) and shape america (naspe, 2008) have suggested that one solution to the lack of childhood physical activity in comprehensive school physical activity programming (cspap). a cspap is a multi-component approach by which schools use all available opportunities for students to be physically active, meet the nation recommendation of 60 minutes of pa per day, and develop the knowledge, skills, and confidence to be physically active for a lifetime (cdc, 2013). cspap has five components (see figure 1) including quality physical education, physical activity during the school day (e.g. recess and classroom pa), pa before and after school, staff involvement, and family and community engagement. there are two main goals of cspap: (1) to provide a variety of school-based physical activities to enable all students to participate in 60 minutes of moderate-to-vigorous pa each day and (2) to provide coordination among the cspap components to maximize understandings, application, and practice of the knowledge and skills learned in physical education so that all student will be fully physically educated and well-equipped for lifetime of pa (cdc, 2013; naspe, 2008). below is a detailed description of each component and strategies for implementation. quality physical education physical education is an academic subject that is the foundation for the cspap (cdc, 2013). quality pe should be guided by national pe/pa standards, be student-centered and developmentally appropriate, have a core focus on physical activity and motor skill development, teach management skills and promote self-discipline, include all students, emphasize proper learning over outcome, promote lifetime personal wellness, and teach responsibility and cooperation and promote diversity (darst, pangrazi, brusseau, & erwin, 2015). the two major outcomes of physical education should be physical activity and health. furthermore, children should be active at least 50% of class time (cdc, 2013). impacting children’s health and academic performance / brusseau & hannon 443 quality physical education provides the necessary skills and may encourage young people to be active adults (darst et al., 2015). figure 1. cspap (cdc, 2013) at the elementary school level, all students should be required to take pe and have a minimum of 150 minutes per week. pe classes should be in line with the class sizes of other academic subjects, should be taught by a trained specialist and should have adequate equipment and facilities (cdc, 2013). quality physical education should also include regular assessment and pa should not be used as punishment. quality physical education has shown to contribute up to 25% of daily physical activity (brusseau, kulinna, tudor-locke, & ferry, 2013) and youth are significantly more active on days that they have physical education class (brusseau, kulinna, tudor-locke, van der mars, & darst, 2011). physical activity before and after school before and after school programs provide opportunities to promote physical activity by providing structured and unstructured physical activity opportunities and teaching youth the skills needed for a lifetime of activity (trost, rosenkranz, & dzewaltowski, 2008). the cdc (2011) and usdhhs (2012) suggest that before/after school programs have the ability to: 1) practice what they have learned in physical education, 2) work toward the nationally recommended 60 minutes of daily physical activity, 3) become more adequately prepared for learning, 4) engage in safe, social, and supervised activities, and 4) identify activities they enjoy and might engage in long term. numerous scholars have identified the ability of before and after school programming to increase youth physical activity (i.e. trost et al., 2008) and decrease overweight (salcedo aguilar et al., 2010). international electronic journal of elementary education vol.7, issue 3, 441-450,2015 444 the cdc (2013) in their cspap recommendations state that “beforeand after-school physical activity programs offer students an opportunity to be physically active instead of waiting in a sedentary setting for the school day to begin or end. these programs may include a walking and biking to school program (i.e. walking school bus), clubs and intramural programs (i.e., programs that are developmentally appropriate and give an opportunity for all students to participate), informal free play on school grounds, and integrating physically active homework during out of school hours. finally, beforeand after-school physical activity programs can be coordinated with community-based organizations (e.g., ymcas, community parks and recreation) and delivered in school settings (cdc, 2013). physical activity during school in addition to physical education, schools can offer physical activity in a variety of settings during the school day. the main ways students can participate in physical activity during the school day are recess, and physical activity integrated into lessons or classroom activity breaks. these opportunities can be offered to all grade levels. schools can facilitate increased physical activity during the school day by encouraging students to be active; providing students with space, facilities, equipment and supplies that make participating in activity appealing; and providing organized times and structured physical activities for interested students (cdc, 2013). recess. recess offers an excellent opportunity for children to engage in free play or semi-structured physical activity during the school day, and allows them to apply skills learned in pe. recess should not, however, replace physical education or be used to meet time requirements set forth in pe policies (cdc, 2013). recess has seen a reduction by more than 20% of school districts in order to allocate more time for english and math (lee, burgeson, fulton, & spain, 2007) and this trend has continued over the past decade. participation in recess is associated with academic benefits, such as improving attentiveness, concentration, behavior, and time on-task in the classroom (i.e. pellegrini & bjorklund, 2010) and also provides a unique contribution to a child’s creative, social, and emotional development (ramstetter, murray, & garner, 2010). strategies for implementing recess in elementary schools include: providing age-appropriate equipment for students, having adult recess supervisors encourage students to be physically active, and providing semi-structured activity that involves activity stations (e.g., jump rope, four square, hopscotch stations) (cdc, 2013). simple modifications including semi-structure and equipment has shown to significantly increase youth physical activity at recess (i.e. larson, brusseau, chase, heinneman, & hannon, 2014) as has adding playground markings (esaclante, garcia-hermoso, backx, & saaverda, 2014). physical activity integrated into classroom lessons. integrating physical activity within classrooms as part of planned lessons that teach academic subjects through movement can increase students’ overall physical activity and improve time-on-task and attentiveness (mahar et al., 2006; donnelly & lambourne, 2011; erwin, beighle morgan, & noland, 2011; goh et al., 2014). physical activity can be integrated into academic subjects for all grade levels, not just elementary school grades. this type of physical activity helps establish an active school environment, and enhance students’ learning experiences. examples of evaluated programs or interventions that have shown improvements in students’ physical activity levels include the north carolina energizers (www.eatsmartmovemorenc.com/energizers/elementary.html) and take 10! (www.take10.net/). a specific example of a take 10! lesson might include completing imaginary jump roping while counting using odd numbers on every jump or completing addition or subtraction problems based on the number of jumps a student might complete. impacting children’s health and academic performance / brusseau & hannon 445 physical activity breaks in the classroom. physical activity breaks in the academic classroom allow students to take a mental and physical break from current academic tasks. these breaks can occur at any time during the school day, last from 5–30 minutes, and occur all at one time or several times during the school day. bershwinger and brusseau (2013) found that 10 minute activity breaks can lead to an increase of 1000 steps/day. chaddock, hillman, buck, and cohen (2011) found that even a short break from focused concentration allows the brain to consolidate information for better retention and retrieval of memory. studies (cdc, 2010) have found that offering physical activity breaks during standard classroom instruction may have favorable associations with some indicators of cognitive functioning (e.g., attention/concentration); academic behaviors (e.g., classroom conduct); and/or academic achievement (e.g., test scores). examples of physical activity breaks in the classroom include: • stretching or relaxation break. • walking around the classroom or hallway. • jumping with an invisible jump rope. • doing squats, push-up, or sit-ups. • passing a ball around the classroom staff involvement school employees play an integral role in a school’s cspap (cdc, 2013). school employee wellness programs improve staff health, increase physical activity levels, and are cost effective (osilla et al., 2012). when school staff commits to good health practices, they are positive role models for students, and may show increased support for student participation in physical activity (cullen et al., 1999). support for school employee wellness and leadership training contribute to the overall culture of physical activity at a school. teachers and other school staff members can integrate physical activity into classroom academic instruction and breaks, and support recess, intramurals, and other physical activity offerings. a simple program to encourage faculty to actively engage in physical activity with their students or in their classrooms is the gimme5 initiative where many schools will reward teachers for every week they implement five cspap components with their students they get entered into a lottery or have access to external rewards like free weekly membership at a local fitness center. additionally, school employees can be positive role models for students by demonstrating active lifestyle choices in and out of school (cdc, 2013). it is also important to integrate staff wellness activates into a cspap program. these might include a staff wellness room, cooking classes, or community opportunities (i.e. gym passes) for faculty and staff. family and community engagement family and community engagement in school-based physical activity programs provides numerous benefits (cdc, 2012). research shows that youth participation in physical activity is influenced by participation and support of parents and siblings (lee et al., 2010). when families are active together, they spend additional time together and experience health benefits (lee et al., 2010). parents, guardians, or other family members can support a cspap by participating in evening or weekend special events, or by serving as physical education or physical activity volunteers. physical activity homework from the classroom or physical education teacher might require the family to be active as part of the students at home assignments (williams & hannon, 2013). an example might include requiring the family to participate in a variety of activities at home and calculate their heart rates during each activity to determine which are light, moderate, or vigorous. international electronic journal of elementary education vol.7, issue 3, 441-450,2015 446 community involvement allows maximum use of school and community resources and creates a connection between school and community-based physical activity opportunities. community organizations might provide programs before or after school or establish joint-use or shared-use agreements with schools (cdc, 2013). community events including walkathons or zumba decathlons encourage everyone to be active and at the same time raising money for local charities. the importance of coordination of the cspap the cdc (2013) suggests that in order to maximize physical activity opportunities in schools they need to be coordinated, well planned, and thoughtfully executed and evaluated, thus creating a culture of physical activity that is integrated throughout the school environment and reaches beyond the school and into the community. a school that establishes student health as a priority will form a cspap team and develop a comprehensive physical activity plan that includes all of the components described in the preceding sections (cdc, 2013). a cspap reflects the social, emotional, and cultural needs of students, their families, and the broader community, thereby establishing a strong social and culturally supportive environment for students, families, and communities to engage in physical activity (cdc, 2013). strong support from school administration and staff involvement in the cspap are important to school program success. the physical educator is ideally positioned to address issues of physical inactivity during the school day, as they understands the school environment, parents, the community, correlates of physical activity, and unique characteristics and needs of the school culture (cdc, 2013). from this perspective, the physical education teacher is ideally situated to lead the development and implementation of the cspap, with strong support from other staff, volunteers, and teachers (castelli & beighle, 2007). in addition, classroom teachers and school staff play a vital role in promoting the health of their students by integrating physical activity opportunities throughout the school day (pangrazi, beighle, & pangrazi, 2009) and serving as positive role models while supporting student participation in physical activity (naspe, 2008). when coordinated approaches are implemented they have begun to illustrate positive impacts on children’s physical activity (i.e. burns, brusseau, & hannon, 2015; kulinna, brusseau, cothran, & tudor-locke, 2012) the following sections define and describe steps to develop, implement, and evaluate a cspap (cdc, 2013). 1. establish a team or committee and designate a physical activity leader. 2. conduct an assessment of existing physical activity opportunities. 3. create a vision statement, goals, and objectives for your cspap. 4. identify the outcomes or specific changes that will be direct results of program implementation. 5. identify and plan the activities for your cspap. 6. implement your cspap. 7. evaluate your cspap. the cdc (2013) also recommends creative ways to schedule physical activity throughout the school day:  adding 5 more minutes to recess time.  integrating physical activity into academic lessons at least once per day.  adding physical activity clubs during times that students arrive early at school in the mornings, or depart late after school. impacting children’s health and academic performance / brusseau & hannon 447  hosting a morning movement activity for each grade level in the school, prior to the start of the school day.  developing intramural sport programs and physical activity clubs. conclusion cspap provide a school and community program geared to maximizing physical activity in children. increases in physical activity are associated with improved academic performance and overall health. with a little planning and commitment from all school personnel, cspap has great potential to make improvements at very little cost. • • • references bershwinger, t, & brusseau, t. a. (2013). the impact of classroom activity breaks on the school-day physical activity of rural children. international journal of exercise science, 6, 6. brusseau, t. a., kulinna, p. h., (2015). an examination of four school physical activity models on children’s step counts and mvpa. research quarterly for exercise and sport, 86 (1), 89-93. brusseau, t. a., kulinna, p. h., tudor-locke, c., van der mars, h., & darst, p. w. (2011). children’s step counts on weekend, physical education, and non-physical education days. journal of human kinetics, 27, 123–134. brusseau, t. a., kulinna, p. h., tudor-locke, c., & ferry, m. (2013). daily physical activity patterns of children living in an american indian community. journal of physical activity and health, 10, 48-53. brusseau, t. a., tudor-locke, c., & kulinna, p. h. (2013). are children meeting any of the suggested step recommendations? biomedical human kinetics, 5, 11-16. burns, r. d., brusseau, t. a., & hannon, j. c. (2015). effect of a comprehensive school physical activity program on school day step counts in children. journal of physical activity & health. epub ahead of print doi: http://dx.doi.org/10.1123/jpah.2014-0578 carlson, s. a., fulton, j. e., lee, s. m., maynard, l. m., brown, d. r., kohl iii, h. w., & dietz, w. h. (2008). physical education and academic achievement in elementary school: data from the early childhood longitudinal study. american journal of public health, 98(4), 721. castelli, d. m., & beighle, a. (2007). the physical education teacher as school activity director. journal of physical education, recreation & dance, 78(5), 25-28. centers for disease control and prevention (2013). comprehensive school physical activity programs: a guide for schools. atlanta, ga: u.s. department of health and human services. centers for disease control and prevention (2012). parent engagement: strategies for involving parents in school health. atlanta, ga: u.s. department of health and human services; 2012. international electronic journal of elementary education vol.7, issue 3, 441-450,2015 448 centers for disease control and prevention (2011). school health guidelines to promote healthy eating and physical activity. mmwr, 60, 28–33. chaddock, l., hillman, c. h., buck, s. m., & cohen, n. j. (2011). aerobic fitness and executive control of relational memory in preadolescent children. medicine & science in sports and exercise, 43(2), 344-9. colley, r. c., janssen, i., & tremblay, m. s. (2012). daily step target to measure adherence to physical activity guidelines in children. medicine & science in sports and exercise, 44(5), 977-982. cullen, k. w., baranowski, t., baranowski, j., hebert, d., demoor, c., hearn, m. d., & resnicow, k. (1999). influence of school organizational characteristics on the outcomes of a school health promotion program. journal of school health, 69(9), 376-380. darst p. w., pangrazi, r. p., brusseau, t. a., & erwin, h. (2015). dynamic physical education for secondary school students (8th ed.). benjamin cummings: san francisco. deckelbaum, r. j., & williams, c. l. (2001). childhood obesity: the health issue. obesity research, 9(s11), 239s-243s. donnelly, j. e., & lambourne, k. (2011). classroom-based physical activity, cognition, and academic achievement. preventive medicine, 52, s36-s42. erwin, h. e., beighle, a., morgan, c. f., & noland, m. (2011). effect of a low‐cost, teacher‐directed classroom intervention on elementary students' physical activity. journal of school health, 81(8), 455-461. escalante, y., garcía-hermoso, a., backx, k., & saavedra, j. m. (2014). playground designs to increase physical activity levels during school recess a systematic review. health education & behavior, 41(2), 138-144. goh, t. l., hannon, j. c., brusseau, t. a., webster, c., podlog, l., & newton, m. (2014). effects of a classroom based physical activity program on children’s physical activity levels. journal of teaching in physical education, 33, 558-572. kohl iii, h. w., & cook, h. d. (eds.). (2013). educating the student body: taking physical activity and physical education to school. national academies press. kulinna, p. h., brusseau, t. a., cothran, d. j., & tudor-locke, c. (2012). changing school physical activity: an examination of individually school designed programs. journal of teaching physical education, 31, 113-130. larson, j., brusseau, t. a., chase, b., heinneman, a., & hannon, j. c. (2014). youth physical activity and enjoyment during semi-structured versus unstructured school recess. open journal or preventive medicine, 4, 7. lee, s. m., burgeson, c. r., fulton, j. e., & spain, c. g. (2007). physical education and physical activity: results from the school health policies and programs study 2006. journal of school health, 77(8), 435-463. lee, s. m., nihiser, a., strouse, d., das, b., michael, s., & huhman, m. (2010). correlates of children and parents being physically active together. journal of physical activity and health, 7, 776783. impacting children’s health and academic performance / brusseau & hannon 449 lee, i. m., shiroma, e. j., lobelo, f., puska, p., blair, s. n., & katzmarzyk, p. t. (2012). effect of physical inactivity on major non-communicable diseases worldwide: an analysis of burden of disease and life expectancy. the lancet, 380(9838), 219-229. mahar, m. t., murphy, s. k., rowe, d. a., golden, j., shields, a. t., & raedeke, t. d. (2006). effects of a classroom-based program on physical activity and on-task behavior. medicine and science in sports and exercise, 38(12), 2086. national association for sport and physical education and american heart association (2006). 2006 shape of the nation report: status of physical education in the usa. reston, va: national association for sport and physical education. national association for sport and physical education (2008). comprehensive school physical activity programs. reston, va: national association for sport and physical education. ogden, c. l., carroll, m. d., curtin, l. r., mcdowell, m. a., tabak, c. j., & flegal, k. m. (2006). prevalence of overweight and obesity in the united states, 1999-2004. journal of the american medical association, 295(13), 1549-1555. osilla, k. c., van busum, k., schnyer, c., larkin, j. w., eibner, c., & mattke, s. (2012). systematic review of the impact of worksite wellness programs. the american journal of managed care, 18(2), e68-81. pangrazi, r. p., beighle, a., & pangrazi, d. (2009). promoting physical activity and health in the classroom. pearson education, inc. san francisco: ca. pellegrini, a. d., & bjorklund, d. f. (1997). the role of recess in children's cognitive performance. educational psychologist, 32(1), 35-40. ramstetter, c. l., murray, r., & garner, a. s. (2010). the crucial role of recess in schools. journal of school health, 80(11), 517-526. salcedo aguilar, f., martínez-vizcaíno, v., sánchez lópez, m., solera martínez, m., franquelo gutiérrez, r., serrano martínez, s., ... & rodríguez-artalejo, f. (2010). impact of an afterschool physical activity program on obesity in children. the journal of pediatrics, 157(1), 3642. sallis, j. f., mckenzie, t. l., kolody, b., lewis, m., marshall, s., & rosengard, p. (1999). effects of health-related physical education on academic achievement: project spark. research quarterly for exercise and sport, 70(2), 127-134. strong, w. b., malina, r. m., blimkie, c. j., daniels, s. r., dishman, r. k., gutin, b., ... & trudeau, f. (2005). evidence based physical activity for school-age youth. the journal of pediatrics, 146(6), 732-737. tremblay, m. s., & willms, j. d. (2003). is the canadian childhood obesity epidemic related to physical inactivity?. international journal of obesity, 27(9), 1100-1105. trost, s. g., rosenkranz, r. r., & dzewaltowski, d. (2008). physical activity levels among children attending after-school programs. medicine & science in sports and exercise, 40, 622-629. international electronic journal of elementary education vol.7, issue 3, 441-450,2015 450 tudor-locke, c., craig, c. l., beets, m. w., sarahjane, b., cardon, g. m., duncan, s….& blair, s. n. (2011). how many steps/day are enough? for children and adolescents. international journal of behavioral nutrition and physical activity, 8, 78. u.s. department of health and human services (2012). physical activity guidelines for americans midcourse report: strategies to increase physical activity among youth. washington, dc: u.s. department of health and human services. u.s. department of health and human services (2008). physical activity guidelines advisory committee report. washington, dc: u.s. department of health and human services. williams, s. m., & hannon, j. c. (2013). physical education homework that involves the family. strategies, 26(3), 3-8. international electronic journal of elementary education june 2023, volume 15, issue 5, 371-382 371 © 2023 published by kura education & publishing. this is an open access article under the cc byncnd license. (https://creativecommons. org/licenses/by/4.0/) copyright © www.iejee.com issn: 1307-9298 teacher professional development for stem integration in elementary/ primary schools: a systematic review tugba boz* abstract introduction there has been a recent increase in studies devoted to integrated science, technology, and engineering mathematics (stem) education. however, as the field is emerging, there has not been a large amount of scholarship on how to best provide teacher professional development (pd) on integrated stem education. on the other hand, well-grounded research is available in the field of effective teacher professional development. therefore, in this study, a systematic review of empirical studies on training elementary/primary school teachers in integrated stem education was undertaken. using an adapted version of lawless and pellegrino's analytical framework (2007), the emerging commonalities among eleven studies in conjunction with the literature on effective teacher professional development were analyzed and discussed. this study aims at connecting current stem-integrated pd activities with the literature on effective pd and laying the groundwork for future professional development programs and evaluations in integrating stem subject areas in elementary or primary schools. in recent years, there has been an increase in studies dedicated to exploring the conceptual, theoretical, and practical implications of integrated stem education which refers to “the approach to teaching the stem content of two or more stem domains, bound by stem practices within an authentic context for the purpose of connecting these subjects to enhance student learning” (kelly & knowles, 2016, p.3). the literature (e.g., english, 2016; nadelson & seifert, 2017; pearson, 2017) argues that integrated stem education increases students' interest in and engagement with stem fields and better prepares students for the future stem workforce in which disciplines are interrelated and integrated. this attention to integrating stem subject areas has brought about the development and implementation of integrated stem programs in real classrooms. engineering design projects or the development of robotics projects in science and mathematics classrooms are a few examples. as there is more emphasis on initiating integrated stem keywords: teacher professional development, integrated stem, review of literature, elementary teachers received : 27 october 2022 revised : 5 april 2023 accepted : 17 june 2023 doi : 10.26822/iejee.2023.306 * correspondance details: tugba boz, department of educational theory and practice, university of georgia, athens ga, united states. e-mail: tugba.boz@uga.edu orcid: https://orcid.org/0000-0002-8020-127x 372 june 2023, volume 15, issue 5, 371-382 education at early ages (e.g., cunningham, 2018), programs have been designed and implemented to train elementary school teachers to implement integrated stem education in their classrooms. however, since the field is emerging, there has been a limited amount of research on teacher professional development in integrated stem education. on the other hand, the literature provides well-grounded and comprehensive answers on the essential characteristics of effective teacher professional development (pd) and their effects on teacher beliefs and classroom practices. therefore, this study begins with an in-depth literature review on the key characteristics of effective teacher professional development (pd). the review draws on highly influential and frequently cited studies, authored by the leading figures in the field systematic review is then presented to identify the current empirical studies on professional development activities in integrated stem education. the discussion focuses on the connection between the current stem-integrated professional development activities and the literature on effective pd characteristics. this inquiry is aimed to lay the groundwork for future professional development programs and evaluations in stem integration since, as desimone and garet (2015) argued, specific inquiries in teacher pd research are necessary to "translate the broad features into specific, effective activities in varying contexts" (p. 260). furthermore, this study aims to provide a framework that outlines the effective components of professional development in the integration of stem subjects and presents guidelines for program/project developers and policymakers in their consideration of teacher professional development in stem integration. overview of teacher professional development definition and history professional development (pd) is an activity that aims to improve paid staff members' performance (little, 1987). it is broadly defined as teachers' professional growth resulting from their formal or informal experiences, such as attending workshops, professional meetings, mentoring, reading teacher magazines, and watching documentaries (villegasreimers, 2003; oecd, 2019). some scholars prefer using the term professional learning instead of professional development to emphasize "the nature and ownership of teacher learning" and position teachers as "the key decision makers about what matters for their growth" (smith, 2017, p.2). according to borko (2004), teacher learning can occur in various contexts, such as brief hallway conversations or counseling a troubled child after school. his broad definition of pd recognizes that professional development can occur in multiple settings and ways." it is worth noting that teacher professional development or learning has not always been defined and considered in such a broad way or as an ongoing process. in the first half of the 20th century, research on teacher professional development began with large survey studies (cochran-smith, 2009). starting in the 1970s, a new approach started to govern teacher education. according to cochran-smith and demers (2008), this new model was "more constructivist than transmission-oriented—the recognition that both prospective and experienced teachers (like all learners) brought prior knowledge and experience to all new learning situations, which are social and contextually specific" (p.1011). therefore, in this new model, teacher education was not a one-time event but an ongoing and complex process which "was marked by an equally important set of factors" embedded in teachers' everyday practices and interactions (avalos, 2011, p.17). today, it is widely accepted that teacher professional development plays a significant role in enhancing the quality of school, influencing teachers' beliefs and practices (desimone, 2009, 2011; garet et al., 2001; villegas-reimers, 2003), and improving student achievement (desimone et al., 2005). effectively designed and implemented pd programs can help teachers in their day-to-day work (fullan & miles, 1992). therefore, pd is not only a government or statemandated mass learning practice but is also highly desired by teachers. characteristics of effective professional development the literature proposes well-grounded recommendations for designing and delivering effective teacher pd. one such recommendation is to focus on active learning in pd, as research indicates thar active learning is more likely to result in intended teacher knowledge and skills (bates & morgan, 2018; garet et al., 2001; penuel et al., 2007). in other words, pd should offer teachers many opportunities to observe the modeling of intended changes in instructional practices, incorporate lesson planning and enactments, and practice and reflect on them. (birman et al., 2000; darling-hammond et al., 2017; darling-hammond & richardson, 2009; darlinghammond & mclaughlin, 1995; desimone & pak, 2017; desimone, 2009; desimone et al., 2013; knapp, 2003; popova et al., 2018). additionally, pd should encourage teachers to discuss their work and their students' work, in order to foster collaboration and shared learning (desimone et al., 2002). second, pd should focus on student learning-oriented teaching practices and activities (guskey, 2000; darling-hammond & richardson, 2009; desimone et al., 2013; knapp, 2003): as widely discussed in the literature, the main objective behind pd is to improve student learning. guskey (2000) emphasizes that all pd 373 teacher professional development for stem integration in elementary/primary schools: a systematic review / boz efforts should primarily focus on learning and learners, and that even in planning pd, educators should first decide on the specific student learning outcomes they expect to achieve (guskey, 2014). knapp (2003) claimed that professional learning experiences should concentrate on classroom teaching and the evidence of student learning (e.g., gupta & lee, 2020). additionally, analyzing student work can be incorporated into pd and used to develop a shared understanding among teachers of what constitutes good work, common misconceptions students may have, and which instructional strategies are effective (darling-hammond & richardson, 2009). third, pd should have a specific subject focus (desimone et al., 2013; garet et al., 2001; villagasreimeners, 2013): popova et al. (2018) have indicated that having a specific subject focus is positively associated with student learning. villagas-reimeners (2013) has indicated that a specific subject matter is important to choose for pd because the content, design, and implementation of pd vary according to the subject matter. for example, villagas-reimeners found that concentrated time for pd is more effective for mathematics, while distributed time is more effective for science. fourth, pd should be a collaborative and collegial act (darling-hammond & richardson, 2009; desimone et al., 2013; garet et al., 2001; knapp, 2003): collaborative and collegial learning environments create communities of practice and help improve/increase the change process beyond individual classrooms. however, for such collaborative interactions to emerge, professional development leaders should ensure a safe place to nurture trust and critical dialogue (borko, 2004). fifth, teacher learning in pd should be a coherent part of and seamlessly linked to the curriculum, assessment, and standards in place (darling-hammond & richardson, 2009; desimone, 2009): desimone and garet (2015) emphasized the importance of linking professional development (pd) explicitly to curriculum and classroom realities, as this connection can improve the success of pd initiatives. additionally, pd programs should be designed to ensure that there are no disparities between what teachers learn and what they can implement in their classrooms (desimone, 2009). to achieve this goal, pd should be connected to other changes that are already in place within schools (e.g., garet et al., 2001). by making these connections, teachers can more easily integrate new knowledge and skills into their classroom practices. finally, pd should be sustained over time through on-demand in-site expert support and follow-ups (darling-hammond et al., 2017): while there is no consensus on the duration of pd required to lead to positive outcomes, some scholars suggest that 80 hours (darling-hammond & richardson, 2009), 20 hours (desimone, 2009), or even 8 hours (parsad, 2001) can yield improvements in teaching in real-world classrooms. it is worth noting that the effectiveness of pd is also influenced by the content, format, and quality of the activities offered to teachers (oecd, 2019). therefore, pd programs should offer teachers a variety of activities that allow for active learning, such as observation of effective teaching practices, lesson planning, and opportunities for reflection and collaboration with peers. as darling-hammond (2009) claimed, the academic community is on its way toward reaching a consensus on the content, design, and implementation of effective professional development. however, contextual and background factors should also be considered when designing and delivering pd programs (e.g., altun et al., 2021). research suggests that teacher motivation plays an important role in the success of pd initiatives (guskey, 2002; shin & jun, 2019). the content of pd also influences the impact of pds on teacher change. according to desimone and garet (2015), it is easier for teachers to change procedural classroom behavior than to acquire inquiry-oriented instructional techniques. boyd et al. (2009) claimed that professional development that focuses on practical aspects of teaching is more likely to produce positive effects on students than professional development that primarily emphasizes teacher behaviors. it seems that the existing body of literature on teacher professional development is still growing. still, available literature pinpoints the ways in which pd evaluations can be undertaken and pd effectiveness can be determined. evaluating the effectiveness of professional development as argued by knapp (2003), "the most immediate target of professional development is pro-learning" (p.112). therefore, pd evaluations aim to explore the "changes in the thinking, knowledge, skills, and application that form practicing teachers' or administrators' repertoire" p.112). historically, many early pd evaluations consisted of self-reported changes in the participating teachers' behavior and knowledge at the end of a workshop or series (desimone, 2011). some scholars (e.g., guskey & sparks, 1991; smylie, 1989) subsequently made this criticism and called for more in-depth studies on the subject. today, scholars often suggest that rigorous evaluations should be done during the process of planning and during the process of delivery through formative and summative assessments. for example, guskey (2014) listed the following questions to be considered in planning a pd: "is this experience or activity leading to the intended results? is it better than what was done in the past? is it better than another, competing activity? is it worth the costs?" (p.1219). he 374 june 2023, volume 15, issue 5, 371-382 also added that "answers to these [the following] questions require more than a statement of findings. they demand an appraisal of quality and judgments of value, based on the best evidence available" (p.1219). additionally, more rigorous evidence is expected from researchers or pd practitioners about evaluation. it is recommended that these evaluations should be rigorously handled, including the use and administration of many tools (desimone, 2009, 2011; guskey, 2014). depending on the goal of pd, questionnaires, structured interviews, lesson plans, “oral or written personal reflections, or examinations of participants’ journals and portfolios…direct observations, either by trained observers or using digital recorders” are recommended in the literature to be employed in evaluations (guskey, 2014, p.1227). furthermore, more agents are expected to be involved, such as teachers, administrators, parents, and/or students (guskey, 2014). furthermore, student learning is recommended as a priority in pd evaluations. however, student learning outcomes are not only confined to increases in test scores or cognitive outcomes, but affective and behavioral outcomes should also be considered (guskey, 2002), such as "their perceptions of teachers, fellow students, and themselves; their sense of self-efficacy and their confidence in new learning situations can be especially informative" (guskey, 2014, p.1228). purpose of the study after elaborating on the elements of effective professional development and evaluation, the author now turns to what is considered effective in pd planning, implementation, and evaluation in stem integration. for this particular study, a broad and widely-agreed upon the conceptualization of integrated stem education was adopted: stem integration is the intentional and explicit connection of stem disciplines—science, mathematics, engineering, and technology (english, 2016; kelly & knowles, 2016; national academy of sciences, 2014; pearson, 2017). under this concept of integrated stem education, the following research questions were posed: 1. what is the current state of teacher pd in integrated stem education in elementary schools? 2. what connections are present between the current state of pd in integrated stem education and the literature on effective teacher pd in terms of types, design and evaluation of pd effectiveness? 3. what implications/recommendations can be offered to improve the current state of pd in integrated stem education for elementary teachers based on the recommendations offered by the literature on effective teacher pd? methods screening and eligibility process of the review the main databases in educational research (eric, web of science, psychinfo, academic search complete) were used to find the empirical articles published in peer-reviewed journals published between 1999 to 2022. empirical was defined as having qualitative and quantitative data collection and analysis. generalists: elementary classroom teachers (k-5) and primary school teachers (k-6) were targeted. using the boolean operators, the following keywords and a combination of keywords were used: (stem integration* or interdisciplinary stem) and (professional development or continuing training) and (elementary teachers* or primary teachers*). based on this initial query, 94 articles were retrieved after removing duplicates. for this study, abstracts were examined based on the following criteria: c1. integration was aimed extensively and intentionally in pd, and pd was the reviewed study's focus. c2. more than one subject area was explicitly targeted and was used as integral to the activities rather than peripheral. for example, technology or engineering alone was not considered integrated. c3. only in-service elementary or primary school teachers were targeted. c4. the pd aimed to change teacher knowledge, skills, or behaviors. at the end of this review process, 11 studies were included in this current review (see table 1 below). figure 1. identification of the reviewed studies (page et al., 2021) 375 teacher professional development for stem integration in elementary/primary schools: a systematic review / boz analytical framework and analysis lawless and pellegrino (2007) suggested a schema to review technology tpd programs. in this schema, they recommended evaluating pd programs or research studies based on: three critical dimensions … [which are] [a] type of professional development, which includes issues of delivery, duration, and content… [b] the unit (or units) of analysis that serves as the focus of any research/ evaluation of the outcomes and efficacy of that program … [c] the nature of the research/evaluation study design and method, above and beyond the issues of a unit of analysis and measures. (p. 582) in this study, the schema proposed by lawless and pellegrino was partially adapted. the investigation focused on (a) the types of professional development, including duration, content, and delivery, (b) the unit of analysis or the outcomes evaluated in the professional development, such as student, program, or teacher outcomes, and (c) the designs and methods employed to research the effectiveness of professional development in integrated stem education. in order to ensure the study's rigor and repeatability (xiao & watson, 2019), the author utilized the prisma framework for finding and selecting the reviewed studies and strictly followed the schema by lawless and pellegrino. thematic analysis (joffe, 2011) was employed to identify the main themes of the schema in the 11 reviewed studies, and the similar and different themes for each component of the schema were then reported. the following section discusses the similarities and the differences among the 11 studies in terms of their type, unit of analysis, and design of professional development (pd) programs. then, the connections between these stem-integrated pd programs and the literature on effective pd characteristics are explored. finally, the implications are presented. table 1 lists of empirical studies on stem integrated professional development in k-5. study duration & delivery participants design & methods baxter et al., 2014 a two-day workshop in the summer, a six-day workshop during the school year, and two classroom observation sessions for ten contact days 44 k-5 classroom teachers field notes of workshops, an online survey of teacher confidence in integrated mathematics and science, self-reported responses to changes in practice boice et al., 2021 five-week summer pd, ongoing support during the school year, 17 stem and art teachers from nine elementary schools surveys focus groups, written reflections cook et al., 2020 two-year pd program that included approximately 130 hours of pd during the school year. 25 classroom teachers lesson plans, reflections guzey et al., 2016 a three-week summer institute and year-long support from a coach through monthly meetings 48 k-8 teachers from three large school districts the researchers developed a stem integration assessment tool havice et al., 2018 two-day pd 33 k-5 teachers pre-post survey mcfadden and roehrig, 2017 three-week intensive summer pd, developing stem-integrated curricular units in teams, pilot teaching over the university-based summer camp and in their classrooms over the following academic year four elementary teachers in one group and three elementary teachers in the other group participant reflections, field notes, recordings of pd days, interviews, team conversations, curriculum development artifacts parker et al., 2015 two-week summer workshop, application during 4-week summer school in 22 schools six k-5 teachers from each of the 22 participant schools focus group interviews and teacher surveys over the academic year rich et al., 2017a year-long series of weekly pd followed by classroom practices (year 1 of a longitudinal study) 27 teachers of k-6 teacher self-efficacy and beliefs survey for integrating computing and engineering, interviews and documented observations rich et al., 2018a weekly pd meetings for 45 minutes with a pd researcher on friday afternoons over an academic year (year 2 of a longitudinal study) 17 elementary teachers interviews at the end of the lesson study/application, pd practitioners’ fieldnotes sias et al., 2017 one-week-long pd 39 teachers of the third to fifth grades all 39 lesson plans were coded/analyzed by the researchers developing rubric. wentworth and monroe, 2011 series of pd over two years (details are not specified) 38 in-service elementary teachers ten lesson plans (along with enactments) produced by 38 teachers in groups of three to four; coded by two mathematics instructors based on the four criteria outlined at the beginning of the given pd notes. a = as these two papers report on the same pd, they will be considered together in reporting their findings. 376 june 2023, volume 15, issue 5, 371-382 findings and discussion in this section, the similarities and the differences among the 11 studies regarding the type, unit of analysis, and the design features of their teacher professional development (pd) programs. for example, under the “type” of pd theme, the reviewed studies were categorized and presented based on the delivery medium employed, the duration of the pd offered, and the content (curriculum). the “unit of analysis” theme revealed the tools employed in the reviewed studies to evaluate the effectiveness of pd programs. the connections between these stemintegrated pd programs and the literature on effective pd traits were presented for each theme (type, design, and unit of analysis). type of pd: delivery, duration, and content reviewed studies regarding the delivery and duration, the majority of the reviewed pd studies were delivered during the summer months and consisted of workshops lasting from two days to five weeks. in addition, the majority of pd providers included year-long support for teachers to ensure ongoing engagement with the pd content and implementation in their classrooms. for example, baxter et al. (2014) provided a two-day summer pd and workshops for six days during the school year. in addition, their participants engaged in two classroom observations for ten contact days over the year. guzey et al. (2016) offered a three-weeklong summer pd activity and continued to support teachers through monthly meetings throughout the year. rich et al. (2017; 2018) did not hold a summer academy but provided teachers with weekly 45-minute workshops throughout the school year. two reviewed programs (mcfadden & roehrig, 2017; parker et al., 2015) employed oneto five-week-long workshops followed by classroom implementation during the academic year. only one of the reviewed pd programs (havice et al., 2018) offered a two-day workshop without any follow-up activity, and they discussed that their two-day-long pd was successful as there were successful changes in their participants’ understanding of integrated stem education. regarding the content and curriculum, all the reviewed pd programs aimed to address the existing gradelevel curriculum and standards. this aspect of stemintegrated pd literature agrees with the literature (e.g., desimone, 2009), as pds should be naturally linked to the curriculum and standards in place. in addition, as offered by the literature on effective pd, some pd programs included community-building activities such as field trips and family stem activities. some studies emphasized particular stem domains to be integrated. for example, mcfadden & roehrig (2017) focused on the integration of elementary engineering into content areas and standards in place (math, science, and technology). two of the pd programs were geared toward integrating two domains, mathematics and science (baxter et al., 2014), or technology and mathematics (wentworth & monroe, 2011). in the pd workshops by rich et al. (2017), teachers were introduced to the integration of engineering and computing units, and in their 2018's study, their participant teachers worked on incorporating engineering or computing into their teaching through lesson study. parker et al. (2015) focused on fully integrating science with reading, math, and engineering. all studies encouraged integration to align with the existing grade-level curriculum and standards. the majority of the reviewed pd programs consisted of introducing example units and lessons to their participant teachers, such as elementary is engineering units, researchers-developed integrated stem lesson plans (mcfadden & roehrig, 2017; parker et al., 2015; rich et al., 2017; wentworth & monroe, 2011). in boice et al.'s study (2021), teachers felt more comfortable surrounded by support staff. when they worked with innovators outside the field of education and collaborated to develop integrated stem lessons, they were inspired to prepare creative integrated stem lesson plans. three of the reviewed pd programs (boice et al., 2021; sias et al., 2017) ran special events such as field trips to local companies or engineering design facilities, engineering panels, stem-related conferences, and family science activities. overall, the reviewed programs provided some details about the content of the pd activities. however, the majority of the reviewed pd programs did not provide sufficient information about the aspects that could make critical differences in the implementation and consequences of pd programs, such as pd instructors' background, available resources (material and staff capacity), presence/types of instruction and activities during pd. connections with the literature on effective pd the reviewed studies align with the literature on effective pd that recommends pd to be run over the course of time with an on-site support (e.g., darlinghammond & richardson, 2009; penuel et al., 2007). however, providing encompassing pd programs is related to the availability of human, physical and financial resources, and as observed in havece et al.’s study (2018), short workshops might still enhance teachers’ understanding of stem integration and support teachers to become comfortable with integrated stem curriculum. all the reviewed pd programs aimed to address the existing grade-level curriculum and standards. this 377 teacher professional development for stem integration in elementary/primary schools: a systematic review / boz aspect of stem-integrated pd literature agrees with the literature (e.g., desimone, 2009), as pds should be naturally linked to the curriculum and standards in place. in addition, as offered by the literature on effective pd, some pd programs included communitybuilding activities such as field trips and family stem activities. the reviewed studies did not emphasize some design features of effective pd. for example, the following concerns were not located in the reviewed studies: how did the content present in pd? what types of inquiry prompts and tasks were presented? what was the level of modeling provided? what kind of opportunities were teachers granted for reflection on their work or their students' work? in particular, the reviewed studies do not explicitly discuss whether and how they employed active learning in their pd setting, an element frequently acknowledged as crucial for effective pd (bergh et al., 2014). unit of analysis reviewed studies researchers utilized a variety of tools to evaluate the effectiveness of their pd programs, including analysis of lesson plans created by participant teachers, prepost surveys, field notes, interviews, and teacher debriefs. however, only one study focused on evaluating program outcomes (parker et al., 2015), while the others evaluated teacher outcomes, such as changes in beliefs, understandings, self-reported knowledge, perceptions, confidence, self-efficacy, and self-reported levels of practice/implementation. none of the reviewed studies collected student data to assess changes in knowledge, skills, or attitudes resulting from their teachers' participation in integrated stem programs. only baxter et al. (2014) included student work in their pd to allow participant teachers to reflect on students' understanding of integrated science and mathematics concepts, but it was unclear whether participant teachers evaluated their students' work. additionally, none of the studies included external evaluators to assess the effectiveness of their pd practices. connections with the literature on effective pd the pd literature encourages the use of multiple tools and methods to evaluate changes in teachers' knowledge, skills, attitudes, classroom teaching, and student learning. the majority of the reviewed pd studies employed various data collection methods, such as field notes and teacher debriefs. however, as recommended in the literature on effective pd, rigorous methods are needed to examine the direct influence of pd on real-life classroom instruction and learning (e.g., guskey, 2014). none of the reviewed studies examined student learning, and it is unclear whether the integrated stem activities continued to be taught by teachers after support from the research team was discontinued. furthermore, the long-term effects of pd on teacher practice were not clearly addressed in the studies. design reviewed studies the majority of the studies discussed the importance of teacher collaboration in stem-integrated pd. it was argued that having a supportive communal learning environment in the pd setting helped teachers develop confidence and comfort in implementing the targeted curriculum. boice et al. (2021) argued that collaborative lesson planning was critical to ensure equal stem integration and create a sense of community among teachers. parker et al. (2015) emphasized that teachers benefitted from talking about their successes or challenges with the implementation of stem integration with their colleagues who taught the same grade. the assessment was considered to be a challenging aspect of integrated stem education. guzey et al. (2016) recommended to support teachers in assessing student learning in integrated stem lessons. cook et al. (2021) claimed that in implementing integrated stem lessons, their participant teachers used assessments that depended on one stem subject. they recommended performance-based assessment types (in pd and classroom) that “include student choice on the product they produce or the process through which they showcase their content understanding” (p. 206). studies focused on the elements of pd that help participants establish links between pd and their teaching and student learning. for example, noticing positive student reactions/responses to integrated stem materials and lessons increased teacher willingness to dive deeper into stem integration (boice et al., 2021; cook et al., 2020; rich et al., 2017, 2018). boice et al. (2021) proposed to present teachers with many hands-on activities to first-hand experience integrated stem learning and even use these activities in their teaching of integrated stem lessons. upon their assessment of the lesson plans, wentworth and monroe also (2011) recommended that teachers can develop more complex integrated lesson plans if pd addresses how and where teachers can find and examine integrated stem lesson examples. the reviewed studies included pd features that supported their participants’ learning and teaching of integrated stem education. modeling the stemintegrated curriculum was discussed in the reviewed 378 june 2023, volume 15, issue 5, 371-382 articles as necessary for teachers to become familiar with the new content and pedagogy of integrated stem education (parker et al., 2015; sias et al., 2016; wentworth & monroe, 2011). the authors recommended that the pd practitioners need to model the skills and practices that teachers are expected to demonstrate in pd, and teachers should be clear about what skills and knowledge teachers will gain at the end of pd. sias et al. (2017) even claimed, “without experience or models of the implementation of the educational innovations, teachers are less likely to conceive of how they might integrate the innovations in their lessons” (p. 235). studies discussed the importance of examining and presenting reflexive connections across stem subject areas in pd settings (e.g., baxter et al., 2014; cook et al., 2020). they also suggested that pd that focuses on the natural relationship between mathematics and science will enhance teacher confidence and practice. guzey et al. (2016) indicated that science teachers need more subject-matter knowledge to teach mathematics effectively, and teachers might mainly focus on engineering and science integration. therefore, it is necessary for pd providers to provide opportunities for teachers to have the chance to increase their understanding of all the subject areas that will be integrated. wentworth and monroe (2011) similarly emphasized that pd that integrates mathematics and technology should focus on the use of technology in an integral way that “allows candidates to think more deeply about the mathematics in ways they could not without the technology” (p. 271). some contextual factors were proposed to consider in organizing and implementing pds. the provision of necessary physical resources (i.e., supplies, materials for engineering lessons, printouts, and teacher and student workbooks) enabled teachers to incorporate computing and engineering in their classes (e.g., rich et al., 2017, 2018). boice et al. (2021) indicated that "ongoing financial and material support allowed teachers to engage students in new and otherwise impossible ways" (p. 17). being provided with support staff (researchers, pd practitioners, or coaches) or having administration/ peer support at school also assisted teachers in teaching integrated stem (boice et al., 2021; rich et al., 2018). time constraints in developing integrated lessons, unanticipated changes during trainings (participant turnovers, changing expectations), and lack of shared vision among the actors of pd implementation inhibited teachers’ efforts to integrate stem in implementation (guzey et al., 2016; rich et al., 2018). parker et al. (2015) discussed that their participant teachers expected coaches to have a strong content area and district-level contextual knowledge. in implementing an integrated stem lesson, teachers surely needed to believe that integration of stem lessons would help them reach their instructional goals and state standards and that the provided/ presented materials would align well with their classroom realities. two studies concluded that teachers were not ready to think in nontraditional ways as integrated curriculum and teaching just started to be included in elementary schools (mcfadden & roehrig, 2017; sias et al., 2017). in addition, working in large schools was correlated with increased engineering integration activity per teacher. only one study (baxter et al., 2014) included information about the nature of integration for a successful pd: opportunities of infusion and transfer in connecting mathematics and science. connections with the literature on effective pd some implications and recommendations offered in the reviewed studies agreed with the literature on effective pd. for example, modeling the stemintegrated curriculum was discussed as crucial in the reviewed studies. collaborative practices and peer/ administration support were also discussed as critical enablers of stem-integrated curriculum and lesson implementation in most of the reviewed studies. the pd literature also discusses the importance of collaborative and collegial learning environments as conducive to creating communities of practice and increasing the change process beyond individual classrooms (e.g., bates & morgan, 2018). the literature on effective pd recommends that pd should have a specific subject focus (e.g., desimone et al., 2013). some studies focused on the integration of certain stem subject areas, such as mathematics and science, and those studies suggested that it would be important to focus on the natural connections across stem subject areas so that elementary/primary teachers increase their cross-curricular subject matter knowledge. some studies focused on stem without any concerns to increase teachers' knowledge of particular subjects (e.g., boice et al., 2020). from the reviewed studies, it is not possible to conclude whether focusing on individual subject areas or stem overall leads to different results in teacher knowledge and skills in integrated stem education. cochran-smith and demers (2018) also underline the importance of "the recognition that both prospective and experienced teachers (like all learners) brought prior knowledge and experience to all new learning situations, which are social and contextually specific" 379 teacher professional development for stem integration in elementary/primary schools: a systematic review / boz (p.1011). therefore, context plays an important role in pd practitioners/researchers' successes and challenges in the field. however, only a few studies reviewed (i.e., parker et al., 2015) included detailed information regarding teacher, school, and district backgrounds. in alignment with the literature on effective pd, the reviewed studies showed that examining intended change in student work in pd and positive student reactions/responses to integrated stem materials and lessons increased teacher willingness and motivation to engage with stem integration in classrooms (e.g., rich et al., 2017, 2018). some practices that were raised in the reviewed studies were interesting. the assessment was one area that was discussed as important to focus on in integrated stem pd. the reviewed studies recommended that teachers are challenged with developing assessment practices that will equally assess student learning in more than one subject area in integrated stem lessons, and they should be supported in terms of how to assess student learning in integrated stem classes. human and material resources made the difference in the way teachers employed integrated stem education in their practice. the provision of materials, supplies and support from the school community (i.e., administrators) enabled teachers to incorporate intended pd knowledge and skills in their practice. in particular, it requires more time for teachers to plan and develop integrated stem lessons; therefore, time constraints can significantly influence teachers' efforts to teach integrated stem lessons. implications and conclusion this systematic review identified several key implications for improving professional development (pd) practices and advancing research on integrated stem education. first, it is important to recognize that some teachers may lack confidence or expertise in certain subject areas, which can hinder their ability to create balanced and effective integrated stem lessons. to address this issue, pd providers can offer targeted workshops to enhance subject matter knowledge and encourage collaborative planning with content experts. additionally, pd developers should model active learning strategies and provide teachers with a variety of examples of effective integrated stem curriculum for primary/elementary classrooms, including lessons, videos, and student artifacts. it is also crucial to ensure ongoing support and highlight successful practices for teachers implementing integrated stem activities. furthermore, pd programs should align with school or district-level knowledge and provide opportunities for teachers to receive continuous support from administrators and parents. field trips to local businesses and community-wide seminars and workshops can also be organized to establish a shared vision and sense of community around integrated stem education. pd providers should also emphasize assessment practices that equally assess student learning and understanding across each subject area integrated into stem lessons. finally, it is important to support teachers in planning and implementing integrated stem lessons. one way to achieve this is by involving support staff from integrated disciplines such as engineering, who can assist teachers in developing creative ideas for lesson planning and delivery. to further advance research on effective stem-integrated pd, future studies should explicitly describe the pd process, including details about teacher, instructor, school, and community backgrounds, available resources, and types of instruction and tasks. despite the insights gained from this systematic review, there are several limitations to consider. the review was based on a relatively small sample of 11 studies; therefore, more research is needed to establish consensus on effective components of stemintegrated pd. nevertheless, this review provides a solid foundation for future pd development and evaluation in stem integration and can inform the development of more comprehensive research designs to explore the important characteristics of effective teacher professional development in integrated stem education. references references marked with an asterisk indicate studies included in the systematic review altun, s., yabas, d., & bal-nayman, h. (2021). teachers’ experiences on instructional design based professional development: a narrative inquiry. international electronic journal of elementary education, 14(1), 35–50. avalos, b. (2011). teacher professional development in teaching and teacher education over ten years. teaching and teacher education, 27(1), 10–20. bates. c. c., & morgan, d. n. (2018). seven elements of effective professional development. the reading teacher, 71(5), 623–626. https:// doi:10.1002/trtr.1674 *baxter, j. a., ruzicka, a., beghetto, r. a., & livelybrooks, d. (2014). professional development strategically connecting mathematics and science: the impact on teachers' confidence and practice. school science and mathematics, 114(3), 102– 113. https://doi:10.1111/ssm.12060 380 june 2023, volume 15, issue 5, 371-382 van den bergh, l., ros, a., & beijaard, d. (2014). improving teacher feedback during active learning: effects of a professional development program. american educational research journal, 51(4), 772–809. https://doi. org/10.3102/0002831214531322 birman, b., desimone, l., porter, a., & garet, m. (2000). designing professional development that works. educational leadership,57(8), 28–33. borko, h. (2004). professional development and teacher learning: mapping the terrain. educational researcher, 33(8), 3–15. https:// doi:10.3102/0013189x033008003 *boice, k. l., jackson, j. j., alemdar, m., rao, a. e., grossman, s., & usselam, m. (2021). supporting teachers on their steam journey: a collaborative steam teacher training program. education sciences, 11, 105. https://doi.org/10.3390/ educsci11030105 cochran-smith, m., & demers, k.e. (2008). how do we know what we know? research and teacher education. in m. cochran-smith, s. feimannemser, & d. j. mcintyre (eds.), handbook of research on teacher education: enduring questions in changing contexts (3rd ed., pp. 1009–1016). routledge. cochran-smith, m., & zeichner, k. (2005). studying teacher education: the report of the aera panel on research and teacher education. lawrence erlbaum associates, inc. *cook, k., bush, s., cox jr, r., & edelen, d. (2020). development of elementary teachers' science, technology, engineering, arts, and mathematics planning practices. school science and mathematics, 120(4), 197–208. https://doi:10.1111/ ssm.12400 darling-hammond, l. (2000). how teacher education matters. journal of teacher education, 51(3), 166–173. https://doi:10.1177/002248710005100300 2 darling-hammond, l. (2006). constructing 21stcentury teacher education. journal of teacher education, 57(3), 300–314. https:// doi:10.1177/0022487105285962 darling-hammond, l., & richardson, n. (2009). research review/teacher learning: what matters? educational leadership, 66(5), 46–53. darling-hammond, l., hyler, m. e., & gardner, m. (2017). effective teacher professional development. learning policy institute. desimone, l. (2009). improving impact studies of teachers' professional development: toward better conceptualizations and measures. educational researcher, 38(3), 181–199. https:// doi:10.3102/0013189x08331140 desimone, l. m., & garet, m. s. (2015). best practices in teachers’ professional development in the united states. psychology, society, & education, 7(3), 252–263. https://doi:10.25115/psye.v7i3.515 desimone, l.m., & pak, k. (2017). instructional coaching as high-quality professional development. theory into practice, 56(1), 3–12. desimone, l., porter, a.c., garet, m.s., & yoon, k. s., & birman, b. f. (2002). effects of professional development on teachers' instruction: results from a three-year longitudinal study. educational evaluation and policy analysis, 24(2), 81–112. https://doi:10.3102/01623737024002081 desimone, l., smith, t.m., & phillips, k. (2013). linking student achievement growth to professional development participation and changes in instruction: a longitudinal study elementary students and teachers in title i schools. teachers college record, 115(5), 1–46. desimone, l. m., smith, t. m., hayes, s., & frisvold, d. (2005). beyond accountability and average math scores: relating multiple state education policy attributes to changes in student achievement in procedural knowledge, conceptual understanding and problem solving in mathematics. educational measurement: issues and practice, 24(a), 5–18. english, l. (2016). stem education k-12: perspectives on integration. international journal of stem education, 3(1), 1–8. https://doi:10.1186/s40594016-0036-1 fullan, m., & milles, m. (1992). getting reform right: what works and what doesn’t. phi delta kappan, 73, 744–752. garet, m., porter, a., desimone, l., birman, b., & yoon, k. (2001). what makes professional development effective? american educational research journal, 38(4), 915–945. https:// doi:10.3102/00028312038004915 gupta, a., & guang-lea, lee. (2020). the effects of a site-based teacher professional development program on student learning. international electronic journal of elementary education, 12(5), 417–428. https://doi.org/10.26822/ iejee.2020562132 381 teacher professional development for stem integration in elementary/primary schools: a systematic review / boz guskey, t. r. (2000). evaluating professional development. thousand oaks, ca: corwin. guskey, t. r. (2002). professional development and teacher change. teachers and teaching, 8(3), 381–391. https://doi:10.1080/135406002100000512 guskey, t. r. (2014). evaluating professional learning. in billett, s., harteis, c., & gruber, h. (eds.). international handbook of research in professional and practice-based learning. (pp. 1215–1235). springer. *guzey, s. s., moore, t. j., & harwell, m. (2016). building up stem: an analysis of teacher-developed engineering design-based stem integration curricular materials. journal of pre-college engineering education research (j-peer), 6(1), 11–29. *havice, w., havice, p., waugaman, c., & walker, k. (2018). evaluating the effectiveness of integrative stem education: teacher and administrator professional development. journal of technology education, 29(2), 73–90. joffe, h. (2011). thematic analysis. in d. harper & a. r. thompson (eds.), qualitative methods in mental health and psychotherapy: a guide for students and practitioners (pp. 209-223). wiley. knapp, m.s. (2003). professional development as a policy pathway. review of research in education, 27, 109–157. kelley, t., & knowles, j. (2016). a conceptual framework for integrated stem education. international journal of stem education, 3(1), 1–11. https:// doi:10.1186/s40594-016-0046-z lawless, k.a., & pellegrino, j.w. (2007). professional development in integrating technology into teaching and learning: knowns, unknowns, and ways to pursue better questions and answers. review of educational research, 77(4), 575–614. https://doi:10.3102/0034654307309921 little, j. w. (1987). teachers as colleagues. in v. richardson-koehler (ed.), educators' handbook: a research perspective (pp. 491–518). longman. *mcfadden, j., & roehrig, g. (2017). exploring teacher design team endeavors while creating an elementary-focused stem-integrated curriculum. international journal of stem education, 4(1), 1–22. https://doi:10.1186/s40594017-0084-1 nadelson, l. s., & seifert, a. l. (2017). integrated stem defined: contexts, challenges, and the future. the journal of educational research, 110(3), 221–223. https://doi:10.1080/00220671.2017.12897 75 oecd (2019). talis 2018 results (volume i): teachers and school leaders as lifelong learners. oecd publishing. https://doi:10.1787/1d0bc92a-en page, m. j., mckenzie, j. e., bossuyt, p. m., boutron, i., hoffmann, t.c., & mulrow, c. d., et al. (2021). the prisma 2020 statement: an updated guideline for reporting systematic reviews. bmj, 372(71). https://doi:10.1136/bmj.n71 *parker, c., abel, y., &amp; denisova, e. (2015). urban elementary stem initiative. school science and mathematics, 115(6), 292–301. https://doi:10.1111/ssm.12133 parsad, b., lewis, l., farris, e., & greene, b. (2001). teacher preparation and professional development (nces 2001-088). u.s. department of education, national center for education statistics. pearson, g. (2017). national academies piece on integrated stem. the journal of educational research, 110(3), 224–226. https://doi:10.1080/00 220671.2017.1289781 penuel, w. r., fishman, b. j., & yamaguchi, r. (2007). what makes professional development effective? strategies that foster curriculum implementation. american educational research journal, 44(4), 921–958. https:// doi:10.3102/0002831207308221 popova, a., evans, d., breeding, m. e., & arancibia, v. (2018). teacher professional development around the world: the gap between evidence and practice (wps 8572). world bank group. http://documents.worldbank.org /curated/ en/349051535637296801/teacher-professionaldevelopment-around-the-world-the-gapbetween-evidence-and-practice *rich, p. j., jones, b., belikov, o., yoshikawa, e., & perkins, m. (2017). computing and engineering in elementary school: the effect of year-long training on elementary teacher self-efficacy and beliefs about teaching computing and engineering. international journal of computer science education in schools, 1(1), 1–20. https:// files.eric.ed.gov/fulltext/ed572692.pdf 382 june 2023, volume 15, issue 5, 371-382 *rich, p.j., belikov, o., yoshikawa, e., & perkins, m. (2018). enablers and inhibitors to integrating computing and engineering lessons in elementary education. journal oftechnology and teacher education, 26(3), 437–469. https:// www.learntechlib.org/primary/p/181979/ *sias, c. m., nadelson, l. s., juth, s. m., & seifert, a. l. (2017). the best laid plans: educational innovation in elementary teacher generated integrated stem lesson plans. the journal of educational research, 110(3), 227–238. https://do i:10.1080/00220671.2016.1253539 smith, k. (2017). teachers as self-directed learners: active positioning through professional learning. springer. https://doi:10.1007/978-981-10-3587-6 villegas-reimers, e. (2003). teacher professional development: an international review of the literature. unesco international institute for educational planning. xiao, y., & watson, m. (2019). guidance on conducting a systematic literature review. journal of planning education and research, 39(1), 93–112. https://doi.org/10.1177/0739456x17723971 shin, y.-s., & jun, j.s. (2019). the hierarchical effects of individual and organizational variables on elementary school teachers’ lifelong learning competence. international electronic journal of elementary education, 12(2), 205–212. https:// doi.org/10.26822/iejee.2019257668 *wentworth, n., & monroe, e. e. (2011). inquiry-based lessons that integrate technology: their development and evaluation in elementary mathematics teacher education. computers in the schools, 28(4), 263–277. https://doi:10.1080/07 380569.2011.620938 microsoft word 4iejee_6_1_magnusson international electronic journal of elementary education, 2013, 6(1), 61-82. issn:1307-9298 copyright © iejee www.iejee.com paradigms in swedish as a second language – curricula for primary school and secondary school in swedish as a second language ulrika magnusson ∗∗∗∗ stockholm university, sweden received: 5 april 2013 / revised: 28 june 2013 / accepted: 11 october 2013 abstract this article analyzes and compares the curricula of swedish and swedish as a second language for primary and secondary school. the school subject of swedish as a second language is young, and its ideological foundation has not been debated to any large extent, in contrast to swedish. this article analyzes the curricula of both subjects in terms of “paradigms”, i.e. beliefs and conceptions on a school subject, and the appraisal system developed within the framework of systemic functional linguistics. in comparison, the curriculum of swedish as a second language turns out to be more oriented towards skills and communicative paradigms, at the expense of paradigms related to personal growth, literature or bildung. also, the curriculum seems to have weak connections to research on second language development or education. the article also gives an overview of the swedish school system with special focus on education for immigrants and multilingual students. keywords: curriculum analysis, swedish as a second language, second language development, second language teaching, appraisal introduction throughout the swedish school system there are two subjects of swedish, swedish (sw) and swedish as a second language (sws), the latter of which is intended for students with swedish as a second language. sw is one of the oldest subjects in the swedish school system and has a long history of ideological discussion and debate on its identity. on the other hand, sws is a young subject, established in 1995, and, also, a subject that has been repeatedly questioned and criticized and that has met organizational challenges. there is an obvious need for a survey of the ideological and pedagogical background of sws. this article aims to chart the aims and ideas behind the school subject of sws through text analysis of the current curricula of sws and ∗ ulrika magnusson, institutionen för språkdidaktik, stockholm university, 106 91 stockholm. phone: 08-12 07 67 38 e-mail: ulrika.magnusson@isd.su.se international electronic journal of elementary education vol. compare these to the curricula of the subject of swedish. the texts are related to trends and policies regarding immigration and multilingualism in sweden. the article opens with a brief introduction to the swedish educational system. focus is put on education for multil theoretical framework of the text analysis is given: first ideological and epistemic concepts from curricular analysis, and, second, the analytical framework of (martin & rose, 2010) for a brief review of relevant earlier research on curricula in swedish is also given. in the results section, the curricula for sw and sws in primary and secondary school are compared and analyzed in education. the results are discussed in relation to earlier research on curricula and on learning and literacy in a second language. organization of education for immigrants and multilingual students the swedish school system in brief. years. normally, a child enters compulsory education the year s/he turns seven and continues up till the age of 15. the majority of schools are public, but there are also private schools run by foundations or corporations. all schools are free to choose a pedagogical profile, e.g. clil, focus on a certain language, waldorf etc. children younger than seven years are offered pre and pre-school class from the age of six. upper secondary education follows the nine years of compulsory schooling. there are 18 national programmes, which comprise three years. six of these are preparatory for tertiary education (e.g. the social science programme, the nat programme) and twelve are vocationally oriented (e.g. the hotel and restaurant programme, the industry programme, the health care programme). students in the national programmes who meet the standards of the education, fulfil entry requirements for tertiary education. the vocational programmes do not generally give qualification to tertiary education. an overview of the swedish education system is given in figure 1. figure 1. international electronic journal of elementary education vol.6, issue 1, 61 62 o the curricula of the subject of swedish. the texts are related to trends and policies regarding immigration and multilingualism in sweden. the article opens with a brief introduction to the swedish educational system. focus is put on education for multilingual students and adult immigrants. sub theoretical framework of the text analysis is given: first ideological and epistemic concepts from curricular analysis, and, second, the analytical framework of (martin & rose, 2010) for analysis of evaluation and the texts’ degree of dialogicity. a brief review of relevant earlier research on curricula in swedish is also given. in the results section, the curricula for sw and sws in primary and secondary school are compared and analyzed in the light of paradigms and conceptions of mother tongue education. the results are discussed in relation to earlier research on curricula and on learning and literacy in a second language. organization of education for immigrants and multilingual students the swedish school system in brief. in sweden, compulsory education comprises nine years. normally, a child enters compulsory education the year s/he turns seven and continues up till the age of 15. the majority of schools are public, but there are also rivate schools run by foundations or corporations. all schools are free to choose a pedagogical profile, e.g. clil, focus on a certain language, waldorf etc. children younger than seven years are offered pre-school from their second year ss from the age of six. upper secondary education follows the nine years of compulsory schooling. there are 18 national programmes, which comprise three years. six of these are preparatory for tertiary education (e.g. the social science programme, the nat programme) and twelve are vocationally oriented (e.g. the hotel and restaurant programme, the industry programme, the health care programme). students in the national programmes who meet the standards of the education, fulfil entry s for tertiary education. the vocational programmes do not generally give qualification to tertiary education. an overview of the swedish education system is figure 1. the swedish education system 61-82, 2013 o the curricula of the subject of swedish. the texts are related to trends and policies regarding immigration and multilingualism in sweden. the article opens with a brief introduction to the swedish educational system. focus ingual students and adult immigrants. subsequently, the theoretical framework of the text analysis is given: first ideological and epistemic concepts from curricular analysis, and, second, the analytical framework of appraisal nalysis of evaluation and the texts’ degree of dialogicity. a brief review of relevant earlier research on curricula in swedish is also given. in the results section, the curricula for sw and sws in primary and secondary school are and conceptions of mother tongue education. the results are discussed in relation to earlier research on curricula and on in sweden, compulsory education comprises nine years. normally, a child enters compulsory education the year s/he turns seven and continues up till the age of 15. the majority of schools are public, but there are also rivate schools run by foundations or corporations. all schools are free to choose a pedagogical profile, e.g. clil, focus on a certain language, waldorf etc. school from their second year upper secondary education follows the nine years of compulsory schooling. there are 18 national programmes, which comprise three years. six of these are preparatory for tertiary education (e.g. the social science programme, the natural science programme) and twelve are vocationally oriented (e.g. the hotel and restaurant programme, the industry programme, the health care programme). students in the national programmes who meet the standards of the education, fulfil entry s for tertiary education. the vocational programmes do not generally give qualification to tertiary education. an overview of the swedish education system is paradigms in swedish as a second language – curricula for primary school and secondary school in swedish as a second language / magnusson 63 five introductory programmes are aimed at students who have not reached the required standards in compulsory school and to help them qualify for a national programme. they are strongly individualized, e.g. lacking a common overall structure. although it is not compulsory, the large majority of students attend upper secondary school, but there are differences between groups of students in terms of whether they are eligible for upper secondary education or not. approximately 85–88 % of all students finishing 9th grade in 2012 were eligible for one or more of the preparatory programmes or vocationally-oriented programmes (skolverket, 2013b). however, only 69–74 % of immigrant students (sw. “elever med utländsk bakgrund”) were eligible for a preparatory or vocationally-oriented programme; among the students that had immigrated after 2003, only 47 % were eligible for such programmes. an important factor is the education of the parents, as students who had immigrated after the age of seven and whose parents had a low education were to a lower extent eligible for national vocationally-oriented programmes than students who had immigrated after the age of seven but whose parents had post-secondary education (skolverket, 2013, p. 33). according to taguma, kim, brink and telteman (2010), differences found between swedish and immigrant students (the denomination used in the report) in primary education are accentuated in upper secondary education. management and regulatory system. in the beginning of the 1990’s, the swedish school system underwent several far-reaching reforms. the responsibility for schools was taken over by the municipalities, which were from then on employers of the teachers and owners of the schools. in addition, municipalities became responsible for the pedagogical content, for the economy and allocation of means and for students’ goal attainment. since the reforms of the 1990’s, the educational system has been managed by objectives. the general goals and objectives of the education are decided at a national level, by the government, the parliament and authorities. the municipalities or school owners are however free to choose how to reach these goals. this implies that methods and, to a large extent, content are determined by local authorities and schools. thereby, schools are assumed to be able to adapt the pedagogy and educational content to local circumstances and conditions. at a national level, two national governmental authorities administrate the educational system. the swedish schools inspectorate scrutinizes all schools in the country, assesses applications to run an independent school and settles in cases of complaint from students or parents. the swedish national agency for education supports, supervises and evaluates public and independent schools. the same agency also formulates the curricula and syllabi, and thereby goals and requirements. education for immigrants and multilingual students the school subject of sws is part of a strong “infrastructure” of education for l2learners and multilingual persons. apart from sws, there is mother tongue education for children year k–12 and swedish for immigrants (sfi), which serves adult immigrants. sw and sws are of equal merit. each has its own curriculum and grading criteria, and both are offered from grade one to twelve. in upper secondary school, both prepare for tertiary education, and give the same qualifications. in compulsory school, the principal of a local school decides which students will study sws, on the basis of the child’s knowledge of and competence in swedish. in upper secondary school, the student decides if s/he will study sws or sw. international electronic journal of elementary education vol.6, issue 1, 61-82, 2013 64 in curricula and regulations, the term mother tongue1 is reserved for the special l1 subject aimed at multilingual students, e.g. kurdish, arabic, somali, english, croatian or serbian. the subject is to be offered multilingual students whose parent(s) has/have a first language other than swedish, if the child uses the language on a daily basis at home or has basic knowledge of the language. in 2011, only 57 % of students entitled to mother tongue tuition actually studied it (skolverket, 2011). the third part of the educational infrastructure for multilingual students and immigrants is swedish for immigrants (sfi) that serves adult immigrants. every person older than 16 years who lives in sweden and lacks the language knowledge and competence that sfi gives is entitled to education within sfi. the municipality is responsible for offering sfi, through public or independent schools. sfi consists of four courses, after which the student can study in upper secondary education. implementation of the school subject of swedish as a second language even though the regulatory and legislative framework is strong from an international perspective, with an independent l2 subject, a mother tongue subject and special support for immigrant children, deficits in implementation have been reported repeatedly. concerning sws, the subject has been evaluated in several publications by governmental authorities, which depict poor implementation of the subject, poor observance of the regulations, poor assessment, and a high share of unqualified teachers. often, sws is not offered as an independent subject equivalent to sw, as it should be according to the compulsory school ordinance, it has a low status among students, parents and teachers, and in many schools sws is a subject for lowperforming students and not for l2-students (msu, 2004; skolverket, 2004; 2005; 2008; utbildningsoch kulturdepartementet, 2006). also, according to skolverket (2008) l2-students in sws come from families with lower education and weaker position on the labor market than l2-students in sw. in a similar vein, the mother tongue subject has met organizational challenges. for example, it is often offered off schedule, the teachers have unfavorable terms of employment, and few of entitled students study the subject (57 % in 2011). the reasons for the poor implementation of sws can perhaps partly be traced to its short history, i.e. it has not yet been consolidated. hyltenstam and milani (2012) however also emphasize a conflict between the rhetorical confession of the value of plurilingualism on the one hand and the monolinguistic norms that predominate in society as well as in education on the other. for example, the creation of the mother tongue subject was, according to hyltenstam and milani, a “vicarious marker” for pluralistic ideology. the consequences and needs were however not sufficiently carefully analyzed, e.g. regarding the need for qualified teachers, teacher education, terms of employment, collaboration with the school at large. lindberg (2009, p.18), in turn, characterizes the creation of sws as a manifestation of “strategic essentialism”. the creation of one subject for all multilingual students without differentiation between beginners and advanced l2-users, or students with different educational background and literacy implied that a varied and heterogeneous student group was treated as homogenous. also, the division of swedish into two school subjects may have emphasized the exclusion of multilingual students from the swedish norm. the history and prerequisites of sw and sws are diametrically opposed: sw has a solid position in school as one of the first school subjects in the system, whereas sws was established only in 1995. both subjects have been the objects of debate, but in 1 in the paper mother tongue is used to designate the school subject, in line with the swedish regulations. in the international reserach cited (sawyer & van de ven 2007) on paradigms, mother tongue does however refer to the national language of a country, in sweden’s case swedish. paradigms in swedish as a second language – curricula for primary school and secondary school in swedish as a second language / magnusson 65 different respects. in relation to sw, debates have concerned the content and aims of the subject, how it ought to relate to the experiences of the students, how literature or grammar ought to be studied and taught or how language development is best supported. as regards sws, on the other hand, the debate has to a large extent concerned the very existence of the subject itself (for a survey, see axelsson & magnusson, 2012; hyltenstam & milani, 2012). the advocates of sws bring out the pedagogic necessity of the subject and the fact that l2-students will need special arrangements to develop language and content knowledge. the opponents stress a separating practice built-in in the subject, when l1 and l2 students are placed in different study groups. a complicating factor is that both opponents and advocates of the subject use the poor implementation of sws as an argument for defending their own opinion. the opponents (fridlund, 2011) observe that the organization of sws is insufficient, that the assessment of students is gratuitous, and that sws stands out as a subject not for l2 learners but for low-performing students. the advocates, instead, notice – for the same reasons – that sws has not yet been implemented according to the intentions. its effects on students’ learning cannot, consequently, be evaluated. when discussing sws, it is also important to remember the composition of students in sw. sw serves a heterogeneous group of l1 and l2 students. partly, this is probably a consequence of the lack of assessment instruments mentioned above, but certainly also to be expected, considering the heterogeneity of multilingual students on the whole as regards language background (cf. fraurud & boyd, 2006). aims of the study as for the school subject of swedish, it has long been discussed in terms of paradigms, conceptions of the subject, in line with an international research tradition on mother tongue education by which the swedish debate is heavily influenced. hellberg (2002) investigated voices in the sw curriculum of 2000, and hellberg (2008) studied paradigms and divergent influences in sw curricula from 1962–2000, and found different, often conflicting positions. the present paper aims to contribute to the debate on the school subject of swedish as a second language by investigating the rhetoric surrounding and construing sws in terms of paradigms. the focus is on the curriculum of sws for primary and secondary school from 2011. the curricula will be analyzed in terms of appraisal (cf. methods), to capture the degree of monologicity/dialogicity of the texts, and paradigms; the latter, as defined by hellberg (2002; 2008), sawyer and van de ven (2007) and van de ven (2004) will be briefly introduced below. the focus is on the curriculum of sws, and to a lesser extent on the curriculum of sw. the curriculum of mother tongue is treated briefly, as this subject is an integral part of multilingual education and thus a complement for multilingual students. it is however less emphasized in the paper, as it does not hold an equally strong position in school as sw and sws, that address the language of instruction and are the largest subjects in school in terms of hours studied. further, only marginally more than half of students entitled to mother tongue tuition actually study it (cf. skolverket, 2011). paradigms of mother tongue education van de ven (2004) and sawyer and van de ven (2007) are two of several theorists who have studied mother tongue education in different countries of the west as it has developed during the 19th and the 20th centuries, and who characterize different stages in terms of paradigms. a paradigm is defined as a “basic set of beliefs that guides international electronic journal of elementary education vol.6, issue 1, 61-82, 2013 66 action” (sawyer & van de ven, 2007, p. 8) and “a certain value orientation on education, with strong implications for content, teaching-learning activities and the legitimacy of mother tongue education” (2007, p. 9). sawyer and van de ven (2007, pp. 11ff.) describe four paradigms. the academic paradigm originates in the 19th century and is discipline based with strong academic traditions. it favors high literature and the study of grammar and literary standards, the national language and canon. the developmental paradigm, that developed in the early 20th century, is childcentered and influenced by reform pedagogy. education has a personal orientation and should promote children’s language development. writing instruction aims at individual expression “in one’s own and ‘authentic’ language” and reading serves “personal development” (2007, pp. 11ff.). the paradigm arose in times of meritocratic ideals, and conceives of mother tongue education as serving social progress. the communicative paradigm developed according to sawyer and van de ven after a reaction against the developmental paradigm during the 20th century, when concerns were raised about standards. the communicative paradigm is emancipatory and “society centered” and aims at social equality. it aims both at children’s development of communicative competence, for them to function in society, and at insights in society by means of language analysis. meta-reflections on language are central. reading and writing are developed in real life situations, arranged by the teacher, and students write a broad range of genres. the utilitarian paradigm is related to the communicative paradigm, as both paradigms aim at raising schooling standards, and developed as an answer to complex societies’ increasing need of “well-educated citizens” (sawyer & van de ven, 2007, p. 13). sawyer and van de ven link it to a stronger interest in tests and examinations. ‘communication’ is defined in a more narrow sense than in earlier paradigms, and students “should be educated for a future contribution to the development of society” (2007, p. 13). hellberg (2008, p. 8), who uses the concept of paradigm in his analysis of sw curricula, refers to ball’s, kenny’s and gardiner’s (1990) distinction between “english as skills”, “english as the great literary tradition”, “progressive english/english as personal growth” and “english as critical literacy”. these overlap with the paradigms as described by sawyer and van de ven above. in the present paper personal growth is considered to correspond to the developmental paradigm, and the great literary tradition to the academic paradigm. english as critical literacy is considered a component of the communicative paradigm, whereas english as skills is considered a component of the utilitarian paradigm. in the swedish debate, malmgren’s (1996) definitions of three conceptions of the school subject of swedish have been influential: “swedish as a skills subject”, “swedish as a subject of bildung” and “swedish as an experience pedagogical subject”. these are highly influenced by the english concepts. in the experience pedagogical subject, importance is attached to the students’ experiences and questions, and through literature, they are to get to know the life and ideas of others. the conceptions and their reciprocal correspondences are presented in table 1. paradigms in swedish as a second language – curricula for primary school and secondary school in swedish as a second language / magnusson 67 table 1. paradigms in mother tongue tuition sawyer & van de ven (2007) ball et al. (1990) malmgren (1996) academic paradigm english as great literary tradition swedish as a bildung subject developmental paradigm english as personal growth swedish as an experience pedagogical subject communicative paradigm english as critical literacy – a component of the communicative paradigm utilitarian paradigm english as skills – a component of the utilitarian paradigm swedish as skills – a component of the utilitarian paradigm the paradigms succeed each other historically but without replacing one another. they are assumed to exist today as competing currents in mother tongue education. according to hellberg (2008, p. 10), several researchers consider the paradigms mutually exclusive, whereas others see them as supplementary. depending on point of view, curricula are perceived either as loci of conflict between divergent interests or as harmonizing different but not necessarily conflicting goals, which together constitute a “best path”. paradigms and voices in curricula of the school subject of swedish 1962–2000 hellberg’s (2008) analysis of the curricula of swedish 1962–2000 in terms of paradigms is one of the most extensive swedish in-depth studies based in the international tradition. hellberg assumed that curricula are the result of power struggles in society and conflicts between different fractions (2008, p. 84) and shows that these conflicts are made visible in some curricula but are hided in others. the voice of the curriculum itself is called the “ego”. the voice of the other is studied as occurrence of conjunctions signaling for example objection, negation, obligation or causality, i.e. signs of argumentation or debate. according to hellberg the sw curricula of 1962, 1969 and 1988 are characterized by dialogicity. the main dialogue is held between a skills voice on the one hand – representing the other – and a voice emphasizing content, the student’s own interest, experience pedagogy and personal growth on the other, which represents the ego of the curricula. for example, in the 1988 curriculum, literature is discussed as a means of understanding others and should therefore “dominate the teaching of swedish”2 (2008, p. 25). the study of literature should not be confined to the history of literature or canon, as the paradigm of literary tradition would claim, but recognize other, more popular genres and be chosen out of consideration of the students’ experiences and conditions. even though less dialogic, the 1980 curriculum too advocates experience pedagogy and personal growth, at the expense of skills training and literary tradition. in 1994, a new voice enters the scene, which hellberg calls “cross-curricular”. it emphasizes the importance of language for all school subjects and is according to 2 all translations into english are mine. international electronic journal of elementary education vol.6, issue 1, 61-82, 2013 68 hellberg a kind of skills voice. ego does however not want to confine language to these functions but also stresses language for identity, relations, thinking and creativity and “life and achievements”. the opinions of the other is not easily distinguishable, according to hellberg, as ego embraces several (implicit) paradigms. in passages, the skills paradigm is the other, but the skills paradigm is also embraced by the ego, e.g. when ego argues in favor of the study of grammar or text types as a means for language development. the vagueness of the 1994 curriculum leads hellberg to the conclusion that the text strives for establishing harmony between paradigms that are actually conflicting. the 2000 curriculum also blends voices of skills in the form of cross-curricular perspectives and personal growth. in summary, hellberg (2008) shows that paradigms and ideals compete for space in the curricula, with varying success. for example, the paradigm of personal growth has an important position, but also the paradigm of literary history and the idea of bildung emerge sometimes, and in some curricula, the skills paradigm is given a more prominent voice, but blended with and not easily disentangled from other paradigms. in particular, the curricula of 1962, 1994 and 2000 advocate meaningfulness and consideration of students’ interests and wishes as opposed to the skills paradigm. hellberg also shows that the earlier curricula of 1962 and 1969 are dialogic, allowing space to different, overtly contradicting voices, particularly the skills paradigm on the one hand and, on the other hand, a weak version of personal growth blended with a modified skills training voice which admits the benefit of meaningfulness and anchorage in students’ interests. the succeeding curricula of 1994 and 2000 hide conflict and strive towards harmonization in the sense that conflict is, in many respects, not acknowledged. another way of putting it is that these later curricula are monologic to a large extent but with interruptions, without placing the paradigms in order of preference or letting them argue overtly, and thus make the arguments explicit. hellberg (2008, p. 34) even calls the uncovered tendencies “the decline of a genre”, with defective coherence and turbid distribution of the subject matter. “consensus as a result of power struggle” seems to lead to a text that is hard to understand and interpret. in an earlier analysis of the curriculum of 2000, hellberg (2002) uses the concept of voices, instead of paradigm, which relates to “values and interests existing before, outside and after” the curriculum. contradictions are found also in this analysis, for example between mother tongue/cultural identity vs. pluralism/diversity: cultural identity: the history and cultural identity of a nation are harboured in a language diversity: [language] reflects the wealth of cultures that enriches and molds a society other contradicting voices are, among others, individual vs. norm, and practice vs. grammar. the voice of the individual states that language and literature are of great importance to identity and meet a need for self-expression. the norms voice, on the other hand, represents the norms of spoken and written language which are imperative irrespective of the opinions of the individual. in a similar vein, the voice of practice, in turn, states that students, by using language in meaningful contexts, learn to manage situations that raise different linguistic demands. the voice of grammar, on the other hand, states that knowledge of structure and language development deepens the understanding of register variability. it stands in contrast to the practice voice, which advocates doing in favor of knowing, not only for language development but also for language structure. paradigms in swedish as a second language – curricula for primary school and secondary school in swedish as a second language / magnusson 69 previous research on sw the literature on the sw subject is rich in comparison to the few reports that exist on sws. here only occasional studies are mentioned that are relevant for the present paper. there are no empirical studies of sw from l2 perspectives. malmgren’s (1992) and ask’s (2005) results on sw tuition in different upper secondary programs rather relate to social class. even though neither malmgren (1992) nor ask (2005) explicitly investigate paradigms in their empirical studies of the realization of sws in upper secondary school, their studies show differences between programs that can be interpreted in these terms. malmgren (1992) studied the educational practice and the “literary socialization” in swedish in two year vocational programs and in three year preparatory social and natural science programs. the students in the two year vocational programs had working class backgrounds, consumed popular culture and many were reluctant to embrace literature and arts, whereas they tended to accept a “useful” skills concept. the interests and literary repertoires of the middle class students in the three year preparatory programs were to a higher degree in accordance with the ideals of school. teachers at the general social and natural science programs had an ideal of bildung, high literature and linguistic norms, and they taught a subject inspired by the academic disciplines of literature and linguistics. in contrast, the teachers at the two year vocational programs had a more pragmatic, flexible attitude towards the subject content and were to a lower extent upholders of ideals of bildung. malmgren (1992, pp. 325f.) points out that this latter attitude corresponded more to the curriculum in force, but she also finds a cleavage between an academic focus on subject matter in the preparatory programs and a trivial “needs-and-interest school subject” in the vocational programs. these different conceptions of the school subject may, according to malmgren, reproduce or reinforce social differences between working class and middle class students. ask’s (2005) results, in a study of writing in upper secondary school, are in line with those of malmgren in the sense that differences were found between vocational and preparatory programs. students in vocational programs wrote practical texts whereas students in preparatory programs were trained in academic writing. the sw curricula of 2011 that are analyzed in the present paper, were studied by lundström, manderstedt and palo (2011) and liberg, wiksten folkeryd and af geijerstam (2012). lundström et al. studied the formulations on literature in the curriculum and found that the 2011 curriculum of sw for primary and lower secondary school, in comparison to earlier curricula, is characterized by a focus on measurable skills. democratic values are however weaker, according to the authors. liberg et al. also discussed the 2011 curriculum of sw for primary school and found a strong focus on language and a weaker focus on literature than in the earlier curriculum of 1994. liberg et al. also observed that language is focused more in terms of formal aspects than critical literacy. methods in the present paper, the curricula of 2011 of swedish as a second language and swedish for primary and secondary school are analyzed in terms of paradigms; the curriculum of the mother tongue subject is also briefly studied. the paradigms are those of sawyer and van de ven (2007) and hellberg (2008). hellberg’s (2002) term voice is also used, when a phenomenon is not covered by a paradigm, for example the voices of norm or diversity. international electronic journal of elementary education vol.6, issue 1, 61-82, 2013 70 in order to capture the multivocal stance of the curricula, if any, these are analyzed in terms of appraisal (martin & rose, 2010). appraisal, developed within the framework of systemic-functional linguistics, is resources for expressing evaluation, i.e. attitudes that are negotiated in a text (2010, p. 25). the central resources of appraisal are three kinds of attitude: affect, expressing feelings such as happy, overjoyed, sad, despairing; judgment, for evaluating people, e.g. sensible, shrewd, kind-hearted; and appreciation, for evaluating all kinds of things – artifacts, nature, art, architecture, relations, ideas – e.g. a beautiful relationship, a good piece of workmanship, a bright idea, a nice jacket. the resources can be expressed in different grammatical niches. among the resources of the appraisal systems which are brought to the fore in the present analysis is also the intertextual element source, i.e. from whom an evaluation comes, e.g. he says that6, he has been regarded as..., it may be deemed6 source is an important means for dialogicity but also for modality, as quotation of the voices of others is an effective means of strengthening one’s own line of argument. modality and concession also belong to the system of appraisal, and, alongside source, represent ways of introducing voices in a text. this is the case in the following excerpts from the curriculum of 1980, from hellberg (2008, p. 25; modals in italics): “in every choice of literature, the teacher must be careful about the demands put on texts. a teacher must consider the students’ conditions of life, experiences and knowledge when choosing literature. reading must give possibilities of recognition and identification to the class, group and individual students, but also new experiences, in which emotions as well as the intellect are challenged.” the modals must are signs of a debate – or even conflict – between the ego of the curriculum and opposing voices – presumably advocates of other motives for choosing literature, for example from a canon. the concessive conjunction but, in turn, also marks an opposition, or at least the existence of complementary experiences from reading literature: the known vs. the unknown. together, the resources of evaluation, modality, concession and source capture the degree of monologue/dialogue of a text, its monogloss or heterogloss (martin & rose, 2010, p. 49). results in the reformation of the swedish school system in the 1990’s, the municipalities took over the responsibility for the schools, that were given increased power over the pedagogy and educational content. educational goals have since then been formulated at a national level, whereas local authorities and schools decide how to reach these. as a consequence, the curricula from 1994 onwards are characterized by a more general stance and higher abstraction than their forerunners. also, they are considerably shorter, giving little or no room for motivations, arguments and reasoning and allowing radically less space for subject matter and methodological considerations. the 2011 curricula of sw and sws are short, even scanty. they are structured in sections: an opening paragraph, aims section, required knowledge, main content and grading criteria. the primary school and lower secondary school curriculum is partly divided into sections corresponding to grade 1–3, 4–6 and 7–9. paradigms in swedish as a second language – curricula for primary school and secondary school in swedish as a second language / magnusson 71 primary and lower secondary school: the initial section and aims section in primary and lower secondary school, the curricula of sws and sw are similar in many respects, but there are deviations, e.g. a stronger focus on form in the sws curriculum. the initial section of sws and sw, as well as of other language curricula, is very similar. here, the text states general functions of language: we use language for thinking, communicating and learning, as well as for living in society. also, through language we express our identity, thoughts and feelings. here, the curriculum thus seems to adhere to a personal growth paradigm with streaks of constructivism. the aim of sws is according to the curriculum for students to develop “knowledge in and about the swedish language” (skolverket, 2011b, p. 227), i.e. the curriculum expresses a communicative (i.e. knowledge of how to use swedish) and an academic paradigm (i.e. theoretical knowledge of the language of swedish). a formulation that is unique to sws is that students must have plenty of opportunities to communicate without too early “demands on language correctness”. this latter formulation probably argues against a norms voice, which is however not explicitly expressed. also the formulations that students must write different kinds of texts is an expression of a communicative or skills paradigm, which is formulated against the background of the lower results of swedish students in international tests. this formulation is followed by the stipulation that students shall express themselves through “different forms of aesthetic expressions” which again echoes the personal growth paradigm but – again – without making explicit this putative opposition between paradigms. the formulations on literature in the aims section emanate from three paradigms. an academic paradigm or high literature states that students shall “meet and acquire knowledge about literature from different periods and different parts of the world”. in the same paragraph, non-fiction/subject-oriented prose is mentioned (sw. sakprosa), i.e. a completely different kind of reading which may be assigned to a communicative paradigm. again, personal growth is also heard, when students are to learn about the world and develop their identity through “encountering different types of texts, performing arts and other aesthetic narratives”. the third set of paradigms in the aims section is grammar and norms, i.e. elements of the academic paradigm, stating that students must learn about swedish and its norms, structure, pronunciation and lexis. a deviation from the academic paradigm, and probably an echo of the communicative paradigm, is the formulation that this knowledge about language will strengthen students’ “awareness of and belief in their own language and communicative ability”. the underlying assumption seems to be that meta-knowledge supports action, in this case language use. primary and lower secondary school: the core content section the core content sections in sws and sw are similar in many respects but the communicative and skills paradigms are stronger in sws. a difference in year 1–3 is a stronger emphasis, belonging to a skills paradigm, in sws on strategies: e.g. strategies for “adapting reading to the form and content of texts” and for “writing different kinds of texts adapted to their typical structures and language features”. sws also lays more stress upon linguistic form, e.g. morphology and sentence structure. also, there is a contrastive approach in sws which is absent in sw, as swedish pronunciation and spelling is to be compared to that of the mother tongue. international electronic journal of elementary education vol.6, issue 1, 61-82, 2013 72 the formulations on text and literature are not easily related to any paradigm, as they are worded as nominal phrases such as “narratives and poems”, “rhymes, jingles, songs” etc., but may mirror an ambition of presenting children with a wide variety of texts and literary experiences. the paradigm of personal growth is also discernible, as students are to read texts “which provide an insight into people’s experiences”. a metalinguistic perspective also is heard, as the structure of narratives, reports and instructions is to be treated in year 1–3. in later school years, reading is described in more interpretative and analytic terms, e.g. discern themes, motifs and aims. primary and lower secondary school: knowledge requirements section the most evident paradigms in the knowledge requirements section of sws are the communicative and the skills paradigms. the academic paradigm is weak: knowledge about language is not prominent, nor is knowledge about the history of literature or literary analysis. also, the personal growth paradigm or the voice of the individual are weak or absent; some formulations however indicate a personal growth perspective, i.e. the students are required to discuss “prominent messages in the texts, and relate these to their own experiences”. as regards reading, emphasis is put on reading with fluency, to understand and to summarize. in year 9, aspects such as “reasoning about messages” and “reference to its author” are mentioned. in writing, variation between text types is important. in year 4–6 and 7–9 students are required to write different texts with “variation in language”. in relation to writing, some genres are sporadically mentioned, e.g. narratives in school year 3. linguistic variation between contexts is an important aspect of advanced second language use (ortega & byrnes 2008) and has also been shown to be critical in l2 texts in comparison to l1 texts (magnusson 2013). however the formulations in the curriculum are vague. also, the few formulations on language do not relate to the variation mentioned or to text types or genres. we will return to this in the discussion section. upper secondary school: the opening paragraph and aims sections the differences between sws and sw are more pronounced in upper secondary than in primary and lower secondary school, for which reason these are given somewhat more space below. swedish. in the first sentence, sw establishes that the core of sw is “language and literature”.3 literature is a dominant theme in the rest of the text, which is one of the traits most obviously distinguishing sw from sws. literature is strongly connected to personal growth in the initial part of the sw curriculum. through literature, the students are to discern “both that which is distinguishing and that which is universal in time and space”, and literature and film are to be a “source of understanding of one self and other people”. also, the education should “challenge” students, so that they develop “new ways of thinking” and are “open to new perspectives”. literature and “other types of texts” are to be put in relation to students’ own experiences and interests. the opening sentence, stating that language and literature constitute the core of sw, is also the beginning of a declaration of the functions of language for an individual: language is, according to sw, the principal tool for “reflection, communication and knowledge development", and with language a person expresses her personality, i.e. 3 english translations of the curriculum for upper secondary school are mine. paradigms in swedish as a second language – curricula for primary school and secondary school in swedish as a second language / magnusson 73 both the communicative and personal growth paradigms. the same personal growth paradigm recurs in the formulation stating that education should stimulate students’ “desire to speak, write, read and listen and thus support their personal development” as well as their “confidence in their own language competence”. language is however not confined to personal growth and the voice of the individual in sw, and as observed by hellberg (2002, 2008) in earlier curricula, different voices compete in the text. in the aims section, the voice of grammar also strives for space by stating that students shall learn about the structure and origins of swedish and reflect on language variation. the communicative paradigm also has its say, as students of swedish are to have the sort of knowledge about “communication that is required in society”. swedish as a second language. literature, personal growth and experience pedagogy are strongly diminished in the sws curriculum.4 instead, the opening paragraph establishes that sws gives “students with a mother tongue other than swedish the possibility of developing their communicative language competence”, i.e. an expression of adherence to the communicative paradigm. the opening paragraph also strikes the “cross-curricular” perspective, which according to hellberg (2002) is a subtype of the skills paradigm: “a rich language is a prerequisite for obtaining new knowledge, for further studies and participation in social life and on the labor market”. the third voice of the opening paragraph of the sws curriculum is that of identity: “it is through language that we express our personality and communicate with others”. identity is limited to “plurilingual identity”, which the school subject should strengthen. the focus on language also pervades the aims section in sws, with an emphasis on the communicative and the skills paradigms. whereas the aim of swedish is that students shall develop their capacity to communicate and to work with texts, from all the aspects described above, understand others through literature, learn about the swedish language and be stimulated to use language, the sws curriculum establishes that the aim of sws is for students to develop “skills in and knowledge about swedish” and to reflect on their own multilingualism. the aims further maintain that students shall have plenty of opportunities to “meet, produce and analyze” language, and to “compare language knowledge and linguistic experiences” with that of others. literature is also allotted a role for the students’ language, as it shall “give the students the opportunity to develop a varied and nuanced language”. these formulations indicate a basis in the skills and the communicative paradigm. perhaps the formulations on “language” can be described as “naked” or “vague”, in comparison to the richer descriptions in sw, where “language” is related to “lust”, “personal development” and “confidence”. in sws, these aspects are not mentioned and there is no trace of language for personal growth in sws. a voice of diversity (cf. hellberg, 2002) is heard in the aims section of sws, which states that “multilingualism is valuable for the individual and society”. the clause is separated by a mere comma from the declaration that students must have the possibility of developing an understanding of the functions of language, by means of comparison of linguistic knowledge and experiences. the connection between the 4 it should be noted that this analysis concerns the sws curriculum in comparison to the swe curriculum. this latter curriculum of 2011 has in other analyses turned out to have less of the features discussed here than earlier swe curricula, e.g. less literature (liberg et al. 2012) or less focus on fundamental values (lundström et al. 2011:11). international electronic journal of elementary education vol.6, issue 1, 61-82, 2013 74 multilingual voice and the metalinguistic understanding in the remainder of the sentence is not totally clear. upper secondary school: knowledge requirements section also in the knowledge requirements section, differences are found between sws and sw. for oral achievements, the knowledge requirements in sws addresses the content of oral production, as the sws curriculum requires that the student can discuss both everyday as well as more public issues and complex subjects and adapt language to the situation. in the sw curriculum, the content is not mentioned, but the assessment concerns the performance itself, e.g. the degree of contact with the listeners and the appropriateness, ease and confidence of the performance. sws thus assesses everyday vs. specialized language, which, strictly speaking, are not aspects of oral language in particular, but concern a duality of language use which is not typical of or exclusive to the oral mode. in sw, in contrast, the assessment concerns factors of the oral performance specifically. in both curricula, the assessment of writing concerns the appropriateness of the text in relation to a context, e.g. readers and “text type”, and its disposition and structure. in sw, norms of written language are mentioned – i.e. the norms voice – whereas sws refers to a sufficient standard of lexicon and grammar, i.e. degree of acquisition. this aspect is unique to sws and does not occur in sw. in sws, reading of literature is embedded in a skills oriented description of the student’s comprehension of written and spoken texts, e.g. “the student can with detail and nuances render the main content of spoken and written texts”. in this context, literature seems to be placed on an equal footing with “simple texts”: “the student can render the content of simple texts and modern literature”. this is the only time that literature is mentioned in the required knowledge section of sws. in the criteria for the highest grade (a), students are also required to relate the content to “own experiences and to universal and social issues”, i.e. personal growth. there is no corresponding requirement in the criteria for the other grades (b–e). nothing is said about interpretation or more complex readings. in sw, in contrast, the required knowledge is related to a paradigm of bildung, including knowledge of central works of art and tools of analysis: the student can briefly render the content of some central swedish and international works of literature and other narrative forms. the students reflect on content and form by means of narrative and stylistic concepts. in addition, the student can account for some connections between different works of literature by offering examples of common themes and motives. as observed by hellberg (2002; 2008) in earlier curricula, there is an inherent conflict between voices and paradigms, when bildung is immediately followed by a voice echoing experience pedagogy: the student renders some observations, formulates well-founded and nuanced thoughts based on the narrative and relates the content in a pertinent way to human conditions. as opposed to the sws curriculum, the skills-oriented formulations about reading are placed before the lines on literature in sw, and are separated from these graphically by a space. paradigms in swedish as a second language – curricula for primary school and secondary school in swedish as a second language / magnusson 75 the sws and sw curricula are similar – but not identical – in one respect, related to knowledge of language. in both, students’ ability to discuss language variation in relation to “speaker” and “communicative situation” is assessed. a difference between the curricula is that sws introduces contrastive and metalinguistic perspectives when describing the student’s comparison of swedish and the mother tongue and “wellfounded and nuanced” reflections on his/her language use and learning. the contrastive perspective is absent from the sw curriculum. the mother tongue curricula for primary school and secondary school on the whole, the most prominent paradigms in the mother tongue curriculum for primary and secondary school are the communicative, the skills and the crosscurricular paradigms. the personal growth and the high literature/bildung paradigms are weaker than in the sw curricula. also, the mother tongue curriculum has a stronger focus on language in comparison to literature than the sw curriculum. this focus chiefly is on language use and communication and not on knowledge of linguistics and grammar. on the whole, doing seems to be emphasized at the expense of knowing. the aim of mother tongue (mt) in primary and lower secondary school is for students to “develop knowledge in and of their mother tongue” and “develop their oral and written language so that they have confidence in their capacities and can use language in different contexts and for different purposes”, i.e. expressions of the communicative and skills paradigms. further, students shall be aware of the importance of language for learning in school subjects, i.e. a cross-curricular perspective. in upper secondary school, the aim is the following: the education aims at developing skills in speaking, reading, writing and listening in their mother tongue. the students shall also be given the possibility of developing knowledge about their mother tongue. furthermore, education shall give the students the possibility of developing knowledge about literature. as is the case in sw and sws, context sensitivity and ability to adapt language to different situations are highlighted: “ability to write different text types with a rich vocabulary and with consideration of the addressee and situation” (upper secondary school). the concept of identity is found, but is weaker than in sw, for example in the following: [t]he students shall be given the possibility of developing a rich language, that gives self-confidence, safety in different situations and the possibility of expressing their personality and strengthening their identity (upper secondary school). in the same vein, the personal growth paradigm is echoed but is not as prominent as in sw. the weaker position of questions of identity and personal growth is coherent with a weaker focus on literature in mt than in sw, even though literature is also part of mt. in the following quote, knowledge about literature and understanding of others through literature are mentioned: in the teaching the pupils should meet and acquire knowledge of literature, other aesthetic narratives and different forms of non-fiction in the mother tongue. in this way, the pupils should be given the opportunity to develop their language, their identity and their understanding of the surrounding world (primary school). a theme that lacks correspondence in the sw curriculum, and is weaker in sws, is culture. for example, mt in upper secondary school shall contribute to the students’ international electronic journal of elementary education vol.6, issue 1, 61-82, 2013 76 “anchoring both in the culture of the language in question and in the swedish culture”, and “development of an intercultural outlook”. appraisal in the sw and sws curricula as opposed to earlier curricula of sw, the sws curriculum for upper secondary school (aims section) of 2011 has several instances of ska (eng. shall), a “legislative” (martin & rose, 2010, p. 56) modal signaling incontestable obligation: the education in swedish as a second language shall aim at the students’ development and knowledge of the swedish language. the students shall also be given the opportunity to reflect on their multilingualism and their prerequisites for conquering (sw. erövra) and developing a rich second language in the swedish society. (my italics) the existence of deontic modality may seem natural in a regulating text like a curriculum, but it is actually a difference between this curriculum and its forerunners. these signals of obligation are found in the aims section, where we are told what the education in the subject will lead to. in this section, an instance of ska is found in each sentence. another type of modal expression in the curriculum of sws is possibility (sw. möjlighet), which is found in the aims section: the students shall be given the possibility of reflecting on their multilingualism. literature, different texts, film and other media shall give the students the possibility of developing a varied and nuanced language. the modal resources shall, possibility and can (e.g. “the student can reads”; core content) are the only expressions of modality in the curriculum. the aims section is followed by the central content section, which consists of phrases and not full clauses and lacks expressions of modality (“reading of and conversations about modern literature by women and men that gives insights into different cultures”). the absence of resources of modality or concession is due to a lack of argumentation or discussion. for example, in the first paragraph of the curriculum what we may call “functions” of language are given, presumably as motivations for the school subject, i.e. language as a means of understanding the world, interact with others and express a personality: “the subject gives students with a mother tongue other than swedish the opportunity to develop their communicative language competence. a rich language is a prerequisite for obtaining new knowledge and for further studies and for taking active part in society. also, it is through language that we express our personality and interact with others in different situations.” the excerpt expresses conceptions about language but it does not tell how these conclusions are reached or admit other views or arguments for the subject of sws. parallel to the absence of signals of modality and concession, there are no expressions of source. the text does not report or quote any sources or voices. in sum, the 2011 curriculum of sws is meagre in terms of modals, concessions and expressions of source. signals of appreciation – i.e. evaluation of processes and things – are more prominent. for example, in the opening paragraph, the voice of the curriculum appears in positive evaluations of the individual’s language competence, presumably as parts of arguments for the subject itself: “a rich language is a prerequisite for active participation in society” (sw. ta aktiv del i samhällsliv). in the paradigms in swedish as a second language – curricula for primary school and secondary school in swedish as a second language / magnusson 77 aims section, expressions of evaluation point out the goals and raison d’être of the sws subject; the subject shall contribute to the students’ “conquering and development of a functional and rich second language in the swedish society”, and the students must have opportunities to develop a “varied and nuanced language” and a “better understanding of the functions of language”. also, the curriculum states that “multilingualism is valuable for the individual and society”. as expected in a section describing the achievements of students, expressions of appreciation are found in the knowledge requirements section, for example: “the student can perform oral presentations in which the coherence is easily grasped. in these the student can efficiently bring out the main idea. (grading criteria for a) the student can make simple reflections on the way linguistic variation is related to speaker, aim and situation of communication. (grading criteria for e)” discussion in the syllabi of 2000, which preceded the curricula analyzed in the present paper, the curricula of sw and sws were nearly identical. the similarity was probably confusing to teachers and headmasters – why separate two subjects that are almost one and the same? the confusion has presumably been one contributing factor to the low status of the subject. it is reasonable to assume that a clear conception of the subject, anchored in assumptions about favorable teaching, second language development, literacy in bilinguals etc. would strengthen its position. the curricula of 2011 represent an effort to formulate a separate school subject; the identity of this subject however still seems to be diffuse and vague. the above analysis suggests that the sws curriculum clearly puts more emphasis on language competence and skills than the sw curricula and is more influenced by a skills and a communicative paradigm, with few traces of the paradigms of bildung, knowledge about language or personal growth. identity is narrowed down to multilingual identity. in other words, it is vague as regards perspectives indicating a content and broader purpose or contextualization of “language competence” or “language development”. the streaks of other paradigms that do exist are found in the initial aims section but are weaker in the core content or required knowledge sections – which may be the parts that actually guide teaching, as noted by liberg et al. (2012). it may be natural and expected that a curriculum aimed at second language learners is more focused on language than a curriculum designed for first language users. still, one has to be careful not to limit the language use, language knowledge and literacy of the l2 students to a reductionist view on language competence. further, the conceptions about literature are vague, and seem to treat literature as related to skills, and few distinctions are made between reading of fiction and non-fiction. the curriculum does not mediate any idea about learning through literature (cf. lundström et al., 2011, p. 12) or literature as a value in its own right. the number of possible readings of literature is strongly reduced. the mother tongue curriculum, in turn, resembles that of sws, in the sense that the communicative and skills paradigms are stronger than in sws whereas the personal growth and academic paradigms are weaker. similar issues have been debated previously in relation to the school subject of swedish, but regarding social class rather than l1/l2 students. malmgren (1992) found differences in the swedish tuition between two year vocational programs and three year preparatory programs (cf. other studies on the sw subject above), that realized a skills oriented subject and a bildung oriented and academic subject respectively. ask (2005) found similar differences between programs regarding writing, as students in international electronic journal of elementary education vol.6, issue 1, 61-82, 2013 78 general programs were found to write genres valued in higher education, e.g. arguments and analyses, whereas students in vocational programs did not. there is a similar risk of creating differences between students, in the sense that swedish as a second language might be construed as a “needs-and-interest” subject, compared to swedish. in his analysis of sw curricula, hellberg (2002) concluded that few of the voices found can be derived from academic disciplines such as scandinavian languages or literature. for example, a voice that hellberg called “the voice of fiction” speaks of literature for understanding of one self and others, but not of literature as “aesthetic experiences, as source of history of ideas or as historical or intertextual context” (2002, p. 98), which would have been more in accordance with the academic discipline. a voice of literary history does exist in the text but is weak and unsupported. neither does the voice that hellberg calls the voice of harmony, advocating a unity of language and literature, have any counterpart in the swedish universities, where linguistics and literature are held apart disciplinary and institutionally. similarly, the academic discipline scandinavian languages is neither in any obvious way related to the voice of grammar in the curricula, as the former tends to stress investigations of language for its own sake, for the understanding of language as a human capacity and means of communication, and not primarily as a means of developing one’s own language (2002, p. 101). if sws is not derived from the same paradigms as sw, which are found instead? are there second language paradigms that have guided the formulations of the text, pedagogic principles that are known to be favorable for second language learners, or insights about language traits that are critical in second language development? to what extent is the representation of the school subject of sws in the curricula influenced by the academic disciplines of swedish as a second language and multilingualism? the answer is that such dimensions are hard to find or that they are vaguely formulated. if read attentively, there are formulations that seem to go back to a conception of language use, language development and writing as being supported by knowledge about linguistic and textual phenomena, i.e. that knowledge of structure deepens the understanding of registerial variability, for example formulations that students shall know the structure of some genres: � the message, structure and content of narrative texts. how a narrative text can be organized with an introduction, sequence of events and an ending (grade 1– 3) � descriptive, explanatory, instructional and argumentative texts /s/ content of texts, structure, typical language features and their words and terms (grade 4– 6) this may be interpreted as advocacy for explicit instruction on language in reading and writing teaching. this has been shown to be advantageous for l2-students (axelsson & magnusson, 2012) who may not have the same implicit knowledge about valued language patterns in a culture as persons born in the culture. this theoretical assumption is however not made explicit, but has to be interpreted from the examples. further, there is no clear connection between the formulations on language and grammar on the one hand and writing and text on the other. another important aspect in l2 development in school that is glimpsed occasionally in the curriculum is everyday vs. specialized, content related language (sw. vardagsspråk vs. ämnesspråk). it is mentioned in relation to writing; for example students are required to use specialized words in year 6. however, this dimension is paradigms in swedish as a second language – curricula for primary school and secondary school in swedish as a second language / magnusson 79 not problematized. what does it mean to distinguish between everyday and specialized language in the subject of sws? that sws shall lend itself to the texts of other school subject, e.g. natural or social science? and, as specialized language is not a relevant variable in arts, literature or creative writing, do the formulations imply that the subject should address factual, non-literary texts? or that the specialization of the subject of sws itself shall be addressed, such as specialized knowledge of literature or language? without manifest traces of paradigms that put language development or competence in relation to a content, or a coherent theory of language and the learning of language, sws stands out as an anonymous subject. the sws curriculum mediates a vague conception of language, in terms of its functions, the way it is developed, fostered and scaffolded, differences between l1 and l2 development and the needs of l2 students. important issues that do not seem to have influenced the formulation of the text are, for example, l2 students’ access to genre and register variation and the language of schooling, including oral vs. written modes of meaning, the expansion of registers and genres and development of registerial variability and the development of advanced literacy in a l2. critical literacy – a component of the communicative paradigm – is weakly represented in the sws curriculum (cf. liberg et al. who found few traces of critical literacy in the 2011 sw curriculum). further, there is no discussion of the relation of sws to the students’ learning in different subject areas. during the last years, there has been a growing interest in genre pedagogy and content and language integrated learning among sws teachers, and several universities offer courses in these areas. this trend does however not seem to have influenced the sws curriculum much. “text types” are mentioned several times but fundamental aspects of genre pedagogy are missing. for example, there is no mentioning of genres as social activities and realization of context. grammar, or knowledge about language, is neither related to the “text types” nor to the students’ writing or in other respects manifest as a meta-language on language. an additional essential question is the diversity of the students that sws serves, which includes newcomers in sweden as well as students who moved to sweden when very young or who were even born in sweden. obviously, the needs and conditions of these students vary enormously. the curriculum does not, however, give teachers any guidance in the differences – with its many intermediate levels – between the beginner’s early language development and the more experienced l2 user’s widening linguistic repertoire in relation to an increasing number of contexts and literacy development. the analysis of appraisal has shown the absence of source in the 2011 curricula, i.e. the absence of overt reference to other voices and authorities, for example regarding the foundations of language development or what teaching that will promote it. the appraisal analysis has also revealed the absence of expressions of concession in the text, such as but, still, nevertheless, and the low frequency of modality. this is interpreted as a manifestation of the monovocal stance of the curriculum, which does not overtly recognize other voices. the same absence of concession is also due to the lack of arguments. the text neither anchors its own decrees theoretically, nor does it put these in relation to other views. as observed by martin and rose (2010, p. 60) we might easily think of the “objectiveness” of a text like this – manifest in the absence of expressions of attitude, judgment and appreciation as well as alternative voices – as a “faceless” text, whereas, in fact, it is itself a face: “a cool excluding face perhaps, but it is a face” (ibid.). the absence of modals, negations, concessives and expressions of source, which was uncovered in the appraisal analysis above, reflects the lack of international electronic journal of elementary education vol.6, issue 1, 61-82, 2013 80 argumentation and discussion in the sws curriculum. it strengthens the impression that sws is not theorized. as noted above, hyltenstam and milani (2012) characterize mother tongue tuition in sweden a “vicarious marker” for a plurilingual ideology that is not however anchored in society at large or in a more profound conception of multilingualism and multilingual students’ learning. it is worth noting that l2 perspectives, multilingual issues or contrastive grammar are totally absent from the sw curriculum, which is studied by many multilingual students. the curricula of sws and mother tongue harbor issues of multilingualism (cf. kulbrandstad, 2002 who found few or no discussions on linguistic diversity and plurilingualism in norwegian textbooks). with the above stated differences between the subjects in mind, it is most welcome that the swedish government has initiated a commission on the sws subject. the issue of content is of immediate interest in the coming reforms. • • • acknowledgements i am grateful to professor siv björklund for comments on earlier drafts and to anne reath warren for editing the manuscript. ulrika magnusson is senior lecturer at the department of language education, stockholm university. her research interests include second language learning, second language literacy and systemic-functional linguistics. references ask, s. (2005). tillgång till framgång. växjö: institutionen för humaniora, växjö universitet. axelsson, m. & magnusson, u. (2012). flerspråkighet och kunskapsutveckling. in k. hyltenstam, m. axelsson & i. lindberg (eds). flerspråkighet – en forskningsöversikt (pp. 247–367). (vetenskapsrådets rapportserie 5:2012.) stockholm: vetenskapsrådet. ball, s., kenny, a. & gardiner, d. (1990). literacy, politics and the teaching of english. in: i. goodson & p. medway (eds). bringing english to order (pp. 47–86). london: falmer. fraurud, k. & boyd, s. (2006). the native/non-native speaker distinction and the diversity of linguistic profiles of young people in swedish multilingual urban contexts. in f. hinskens (ed.). language variation – european perspectives (pp. 53–69). amsterdam: benjamins. fridlund, l. (2011). interkulturell undervisning – ett pedagogiskt dilemma. talet om undervisning i svenska som andraspråk och i förberedelseklasser. (gothenburg studies in educational sciences 310.) göteborg: institutionen för pedagogik och specialpedagogik, göteborgs universitet. hellberg, s. (2002). svenskämnets röster. in b. falkevall & s. selander (eds). svenskämne i kris (pp. 83–103). stockholm: hls förlag. hellberg, s. (2008). konflikter i grundskolans kursplaner för svenska. språk och stil. tidskrift för svensk språkforskning 18, 6–37. paradigms in swedish as a second language – curricula for primary school and secondary school in swedish as a second language / magnusson 81 hyltenstam, k. & milani, t. (2012). flerspråkighetens sociopolitiska och sociokulturella ramar. in k. hyltenstam, m. axelsson & i. lindberg (eds). flerspråkighet – en forskningsöversikt (pp. 17–152). (vetenskapsrådets rapportserie 5:2012.) stockholm: vetenskapsrådet. kulbrandstad, l-a. (2002). større språklig mangfold – didaktiske utfordringer for norskfaget. tospråklig oplælring og inkludering i den flerkulturelle skolen. rapport fra konferanse. høgskolen i oslo 15–16 september 2001 (pp. 304–316). oslo: senter for kompetanseutvikling i den flerkulturelle skolen. liberg, c., wiksten folkeryd, j. & af geijerstam, å. (2012). swedish – an updated school subject? education inquiry 3(4), 477–493. lindberg, i. (2009). i det nya mångspråkiga sverige. utbildning & demokrati. tidskrift för didaktik och utbildningspolitik 18(2), 9–37. lundström, s., manderstedt, l. & palo, a. (2011). den mätbara litteraturläsaren. en tendens i lgr11 och en konsekvens för svensklärarutbildningen. utbildning & demokrati 20(2), 7–26. magnusson, u. (2013). skrivande på ett andraspråk. in k. hyltenstam, & i. lindberg (eds). svenska som andraspråk – i forskning, undervisning och samhälle (pp. 633–660). lund: studentlitteratur. malmgren, g. (1992). gymnasiekulturer: lärare och elever om svenska och kultur. (pedagogiskt utvecklingsarbete vid lunds universitet 92:188.) lund: lunds universitet. malmgren, l-g. (1996). svenskundervisning i grundskolan. lund: studentlitteratur. martin, jr. & rose, d. (2010). working with discourse. meaning beyond the clause. london & new york: continuum. myndigheten för skolutveckling (2004). kartläggning av svenska som andraspråk. stockholm: myndigheten för skolutveckling. ortega, l. & byrnes, h. (2008). the longitudinal study of advanced l2 capacities. new york: routledge. sawyer, w. & van de ven, p. (2007). paradigms of mother tongue education. l1 – educational studies in language and literature 7(1), 1–20. skolinspektionen [swedish schools inspectorate] (2010). läsprocessen i svenska och naturorienterande ämnen, årskurs 4–6. rapport 2010:5. stockholm: fritzes. skolverket [swedish national agency for education] (2004). utbildningsinspektionen 2004 ur ett nationellt perspektiv – en analys av inspektionsresultaten. stockholm: fritzes. skolverket [swedish national agency for education] (2005). svenska och svenska som andraspråk årskurs 9. nationella utvärderingar av grundskolan 2003. stockholm: fritzes. skolverket [swedish national agency for education] (2008). med annat modersmål – elever i grundskolan och skolans verksamhet. stockholm: fristzes. skolverket [swedish national agency for education] (2011). utbildningsresultat. riksnivå. sveriges officiella statistik om förskoleverksamhet, skolbarnomsorg, skola och vuxenutbildning. rapport 356:2011. stockholm: fritzes. skolverket [swedish national agency for education] (2011b). curriculum for the compulsory school, preschool and the recreation centre, 2011. stockholm: fritzes. skolverket [swedish national agency for education] (2013). skolverkets lägesbedömning 2013. rapport 387. stockholm: fritzes. skolverket [swedish national agency for education] (2013b). grundskolan – betyg och prov – riksnivå. tabell ic: elever som avslutat årskurs 9 läsåret 2011/12 och har behörighet till nationellt program. retreived from: http://www.skolverket.se/statistik-ochanalys/statistik/2.4290/2.4291 international electronic journal of elementary education vol.6, issue 1, 61-82, 2013 82 taguma, m., kim, m., brink, s. & telteman, j. (2010). oecd reviews of migrant education, sweden. (oecd reviews of migrant education 2077-6802.) stockholm: oecd, sverige. utbildningsoch kulturdepartementet, 2006. likvärdighet i utbildningen. tematisk granskning. sveriges landrapport. (utbildningsoch kulturdepartementets skriftserie, rapport nr 9.) stockholm: fritzes. van de ven, p. (2004). stabilities and changes in (mother tongue) education. in s. kiefer & k. sallamaa (eds). european identities in mother tongue education (p. 75–94). linz: universitätsverlag rudolf trauer. international electronic journal of elementary education, 2015, 7(2), 107-124. issn:1307-9298 copyright © iejee www.iejee.com prospective elementary and secondary school mathematics teachers’ statistical reasoning  rabia karatoprak  university of iowa, united states gülseren karagöz akar boğaziçi university, turkey bengü börkan boğaziçi university, turkey received: 25 december 2014 / revised: 3 january 2015 / accepted: 1 february 2015 abstract this study investigated prospective elementary (pemts) and secondary (psmts) school mathematics teachers’ statistical reasoning. the study began with the adaptation of the statistical reasoning assessment (garfield, 2003) test. then, the test was administered to 82 pemts and 91 psmts in a metropolitan city of turkey. results showed that both groups were equally successful in understanding independence, and understanding importance of large samples. however, results from selecting appropriate measures of center together with the misconceptions assessing the same subscales showed that both groups selected mode rather than mean as an appropriate average. this suggested their lack of attention to the categorical and interval/ratio variables while examining data. similarly, both groups were successful in interpreting and computing probability; however, they had equiprobability bias, law of small numbers and representativeness misconceptions. the results imply a change in some questions in the statistical reasoning assessment test and that teacher training programs should include statistics courses focusing on studying characteristics of samples. keywords: statistical reasoning, prospective teachers, statistics education, sra. introduction the importance of statistical reasoning is increasing because in today’s world individuals face daily issues such as participating in a debate or community action or being aware of some phenomena such as crime rates, population growth, spread of diseases, industrial production, educational achievement, employment trends etc. (nctm, 2005; watson & callingham,  this article is a written from the first author’s master thesis called assessing preservice mathematics teachers’ statistical reasoning.  rabia karatoprak, faculty of education, university of iowa, usa. e-mail: karatoprak.rabia@gmail.com http://www.iejee.com/ international electronic journal of elementary education vol.7, issue 2, 107-124,2015 108 2003). however, an individual's statistical reasoning is not always intuitive (nctm, 2005); therefore, strengthening and developing individual’s statistical reasoning as an outcome of schooling is necessary (gal, 2002; watson & callingham, 2003; garfield & ben-zvi, 2007). in this respect it is important for prospective teachers to have statistical reasoning (garfield, 2002) because what teachers know is what their students know (fennema & franke, 1992; heaton & mickelson, 2002). however, some small scale studies pointed to the fact that both pemts and psmts lack the reasoning necessary to determine when and why to use statistical constructs such as mean, median, variance and distribution (canada, 2008; groth & bergner, 2006; makar & confrey, 2005; leavy, 2006). by the same token, a study done with 66 pemts and psmts on their probability misconceptions showed that both groups had equiprobablity bias and misconception of law of small numbers (jendraszek, 2010). although small scale studies provided important information about prospective teachers’ existing meanings of some statistical constructs and probability misconceptions, these studies pointed to the lack of knowledge and reasoning in statistics in parts such as understanding variability (e.g., makar & confrey, 2005), understanding mean (e.g., groth & bergner, 2006) and misconceptions regarding probability, etc. (e.g.,jendraszek , 2010). on the other hand, garfield (2003) provided the field with a paper-and-pencil statistical reasoning assessment test (sra) to examine large number of individuals’ statistical reasoning. in this regard, this study aimed at determining prospective elementary and secondary mathematics teachers' statistical reasoning types. examining a comparatively large number of prospective teachers’ statistical reasoning (n=173) with sra test might further assist teacher preparation programs with what to focus on specifically (shaugnessy, 2007; bulut, 2001) due to sra’s capacity for the design of instructions since responses provide in-depth information about both correct reasoning skills and misconceptions (sundre, 2003). studying both pemts’ and psmts’ statistical reasoning types in juxtaposition to each other with participants coming from a metropolitan city of turkey might provide a description of the knowledge repertoire of these prospective teachers with respect to the statistical concepts. also, since both groups come from a different educational program at the university level, comparing their statistical reasoning might add to the literature the differences among these groups based on which recommendations specific to each program can be made. being informed by the aforementioned studies the following research questions were investigated: i. what are the statistical reasoning types of prospective elementary school mathematics teachers? ii. what are the statistical reasoning types of prospective secondary school mathematics teachers? iii. is there any difference between statistical reasoning types of prospective secondary school mathematics teachers and elementary school mathematics teachers? in particular, this research study is situated in turkey, done with turkish prospective teachers educated in a different context than their peers in different countries. yet, results pointed to the similarities with previous research (e.g., jendraszek, 2010) and extended it by focusing on prospective teachers’ reasoning regarding each sub-scale. in this regard, results of this study might further assist teacher preparation programs, courses for teaching statistics, to focus specifically (shaugnessy, 2007; bulut, 2001) on prospective teachers’ knowledge of variables and sampling. teachers’ statistical reasoning / karatoprak, karagöz akar & börkan 109 literature review the outcomes of statistics education ranging from elementary school to university involve three different levels: statistical literacy, statistical reasoning and statistical thinking. although they overlap with each other at the content level (garfield & ben-zvi, 2007; delmas, 2002; rumsey, 2002; chance, 2002; garfield, 2002), delmas (2002) pointed to the differences among these three domains in terms of the cognitive engagement they require from an individual. for instance, statistically literate person can know when to use mean, median and mode or critically evaluate the statistical statement. however, the ability to compare and contrast data, to be able to explain the add-divide procedure in finding the mean or relating the concepts with each other such as independence of an outcome and representativeness belong to statistical reasoning. on the other hand, applying the ideas to the new problems and asking questions of one’s own such as choosing the best way to analyze data or diagnosing weaknesses in the statistics procedures is related to one’s statistical thinking. table 1. delmas’s (2002) distinction between three domains basic literacy reasoning thinking identify describe rephrase translate interpret read why? how? explain (the process) apply critique evaluate generalize although, the differences between these three domains are blurred, researchers emphasized the development of statistical reasoning as an outcome of schooling (gal, 2002; watson & callingham, 2003; garfield & ben-zvi, 2007). this is especially because statistical reasoning is to understand and reason with statistical information and make interpretations based on sets of data (garfield, 2002). such level is important for individuals to participate in community action and be aware of such as population growth, spread of diseases, educational achievement, employment trends etc. (watson & callingham, 2003). in this respect, garfield (2002) postulated correct reasoning skills learners are expected to gain and misconceptions learners should not develop in order to be counted as having statistical reasoning. correct reasoning skills, misconceptions and research on teacher education the correct reasoning skills involve reasoning about; data, representations of data, statistical measures, uncertainty, samples, and, association. researchers concluded that once students have these reasoning skills, they might be considered as having statistical reasoning (garfield, 2002, 2003; garfield & gal, 1999). however, they also contend that for someone to be able to reason statistically, they should not have some particular misconceptions. these are; misconceptions about averages, outcome orientation, bias about good samples, the law of small numbers, representativeness misconception and equiprobability bias. in the following paragraphs, all these aspects of statistical reasoning are discussed relating to the research on teacher education. statistics is based on data (moore, 1990; garfield & ben-zvi, 2008) and reasoning about data involves examining a phenomenon identifying variables embedded in it (mickelson & heaton, 2004). since data are only numbers without context (moore, 1990), within the context, type of data-qualitative or quantitative-can be determined. then inferences can be drawn based on the type of data (garfield & ben-zvi, 2008). once the type of data is determined, then one can represent data constructing and/or modifying graphs and read them international electronic journal of elementary education vol.7, issue 2, 107-124,2015 110 to interpret data and recognize the general characteristics (garfield, 2003). while interpreting data some statistical measures are used such as measures of center (mean, median and mode), spread and position. while interpreting data knowing about when and how to use them is necessary (garfield, 2003). nevertheless, some might think that averages are the most frequent number or regard mean and median as the same. moreover, some might argue that groups are compared according to their averages and average is calculated with add-divide procedure regardless of the outliers. researchers stated that students should not be using these types of (incorrect) reasoning while analyzing statistical information (garfield, 2002, 2003; garfield & gal, 1999). nevertheless, groth and bergner (2006) found out that only 3 out of 46 prospective elementary mathematics teachers could think about hypothetical situations in which one of the mean, median and mode might be a better measure of center although all of them could discuss the procedures of these measures. by the same token, makar and confrey (2005) found out that psmts use of language while describing variation and distribution emerged from the shape of the distribution rather than their understanding of variation while comparing two data sets. these researchers concluded that psmts reasoning about measures of distribution was weak. similarly, some other research studies focusing on pemts’ statistical concepts in analyzing and comparing data sets showed that pemts were not able to take into account measures of spread (canada, 2008; leavy, 2006). reasoning about uncertainty is also an indispensable component in statistical reasoning. it refers to understanding and using the ideas of randomness, chance, and likelihood to know the ways of determining the probability of events (garfield, 2003). outcome of an event cannot be determined certainly even if possible outcomes are definite (moore, 1990). at the same time, each outcome is uncertain. thus the phenomenon under investigation is called as random (moore, 1990) and its likelihood could be measured. however, one might have the tendency to decide by looking at just a single event rather than series of the events (konold, pollatsek, well, lohmeier, & lipson, 1993). in this case outcome orientation approach develops. for instance, as can be seen in sra, one who has outcome orientation sees 70% likelihood of rain in ten days as it should rain in each of the ten days (garfield, 2003), although there is still a 30% chance of not to rain for each day. by the same token, one might also think that the likelihood of events is the same since it happens by chance (lecoutre, 1992); concluding that they are equiprobable by nature. one who thinks this way has the equiprobability bias (lecoutre, 1992). for example, when two dice are simultaneously thrown, the possibility of obtaining two fives and the possibility of obtaining one five and one six on two rolls can be seen as equally likely by someone who has this conception. statistical reasoning also involves reasoning about samples that requires being able to know the part-whole relationship between sample and population (watson & moritz, 2000). sample is a subset of the population and by examining this subset; results of the sample can be generalized to population. to be able to generalize the results randomness, representativeness and bias are critical notions to be considered (watson & moritz, 2000). knowing about characteristics of samples also allows individuals to interpret the relationship between variables, the association (garfield, 2003). whether there is a relationship between variables or to what degree the relationship exists are the concerning questions. furthermore, in order to select a representative sample, members should be selected randomly (gay et al., 2009). however just random selection does not guarantee the representativeness since sample size also effects the results (tversky & kahneman, 1971). small samples even if randomly selected might not show all of the properties of the population since variability exists within the population (watson & moritz, 2000).on the other hand, some might think that good samples have to represent a high percentage of the population. that is they might think that if sample size is a large percentage of a population then it is a good sample (garfield teachers’ statistical reasoning / karatoprak, karagöz akar & börkan 111 & ben-zvi, 2008; garfield, 2003). one who has this misconception thinks that if population size increases then sample size should also increase. however for a reliable estimation absolute size of the sample is important rather than the sample size relative to the population (smith, 2004). a well-chosen sample can provide sound estimations even if the sample is not a high percentage of the population (garfield & ben-zvi, 2008). in other words, as stated earlier, randomly chosen sufficient number of participants as a sample can provide representativeness and unbiased results. however, there is a subtle but an important issue such that randomly drawn two samples are similar to each other since they are randomly chosen. however these two samples might be different from each other since random samples varies, especially the small ones (garfield & ben-zvi, 2008). thus, one who only takes into consideration the randomness ignoring the sample size might think that randomly drawn any sample should have the same characteristics of the population without considering sample size (tversky & kahneman, 1971). therefore, they might conclude that small samples resemble the populations in making inferences (garfield, 2003) since drawn randomly. this is called as the law of small numbers misconception (kahneman & tversky, 1974). interestingly, jendraszek (2010) investigated pemts and psmts (n=66) probability misconceptions and found out that both groups had equiprobability bias and the law of small numbers misconception. by the same token, one might think that the likelihood of a sample being representative depends on how it resembles to the population (kahneman & tversky, 1974). one who thinks this way has representativeness misconception because s/he tends to use heuristics instead of probability principles. for instance, as a result of flipping a fair coin 6 times, obtaining hththt can be seen as less likely than obtaining htttht, by those holding the representativeness misconception. some research done with pemts showed that only a few of them (10 out of 54) were able to point to the characteristics of sample which is representative part of the whole (groth & bergner, 2005). similarly, heaton and mickelson (2002) examined pemts’ integration of statistical investigation to the elementary school curricula. they found that pemts ignored the representativeness of the data they used. özen (2012) also found out that pemts used the sample size as the only aspect of statistics and they failed to relate it with the context. aforementioned research points to prospective teachers’ lack of knowledge in statistical concepts such as understanding the use of mean or variability etc. doing further research with sra might provide detailed information on both pemts’ and psmts’ statistical reasoning types holistically. that is, results might further shed light on the different sub-understandings and misconceptions simultaneously and in juxtaposition to each other. this in turn might assist teacher educators to determine specific pathways to help prospective teachers to reason statistically. method research design research design of the study is a descriptive research in which statistical reasoning of prospective mathematics teachers, is described with quantitative data and presented existing differences between pemts and psmts (gay et al., 2009). groups are established according to the grouping variable which is the teaching level of prospective mathematics teachers. participants target population of the study is senior students studying in the elementary and secondary school mathematics teaching programs in istanbul, turkey. convenience sampling was used international electronic journal of elementary education vol.7, issue 2, 107-124,2015 112 as a sampling method. prospective teachers who participated in the teaching methods course at the time of the data collection constituted the sample. in turkish education system, while elementary school education covers 5th through 8th grades, secondary school education covers 9th through 12th grades. teachers teaching at those grades are required a bachelor’s degree in a related field. unless the degree is awarded by a college of education, teachers are expected to have a teaching certificate given by college of education. the participants of this study consist of 91 psmts (80% of the prospective secondary sub-population) and 82 pemts (50% of the prospective elementary sub-population). table 2. sample groups sample size % elementary school level 82 50 secondary school level 91 80 instrument researchers and teachers use different methods while assessing teaching, learning or the development of statistical reasoning such as performance assessments or interviewing. however these methods are not practical in large groups of people (garfield, 2003; tempelaar, 2004). statistical reasoning assessment (garfield, 2003), on the other hand, is a paper-pencil test which is easy to administer and score. moreover, it is helpful for the design of instructions since data obtained through this test provide information about correct reasoning skills and misconceptions as well (sundre, 2003). by the same token, the statistical reasoning types assessed in the test are covered in in the secondary and elementary school national mathematics curriculum. therefore statistical reasoning assessment (sra) test was used in this study. the test includes 20 multiple choice items about probability and statistics. alternatives of the items are statements indicating correct reasoning, misconception or simply false instances. some items have more than one correct alternatives and alternatives indicating misconceptions. the list of these alternatives and items are given in table 3 and 4. sample items from the original instrument are given in appendix. scoring procedure according to the original scoring procedure, there are two main categories; overall correct reasoning skills score and overall misconceptions score. since there are 8 subscales under these two main categories, 16 scores are obtained from these subscales. (see table 3 and table 4) overall correct reasoning skills score is obtained per person in the following way: someone selecting the correct alternative (choice) gets 1 point otherwise 0 point. these scores of particular responses of the items contributing each scale are added and then divided by the number of items since each scale includes different number of responses. in this way, scores of each sub-scale change on a scale of 0 to 2. for instance the scale of correctly interprets probabilities is composed by alternative d of 2nd and 3rd items. if someone answers correctly one of them and incorrectly the other, s/he gets 0.5 point for correctly interprets probabilities sub-scale since (1+0)/2 equals 0.5. the other correct reasoning subscales scores are calculated in the same way. then, after scores of each subscales generating correct reasoning skills score per person are found, they are added in order to obtain an overall correct reasoning score. this procedure is repeated in order to find each participant's score. the same procedure is applied for obtaining overall misconceptions score but at this part if someone selects the alternative signaling a misconception, then s/he gets 1 point. therefore 0 means for teachers’ statistical reasoning / karatoprak, karagöz akar & börkan 113 misconception part, s/he does not have the misconception. for instance the scale of representativeness misconception is composed by alternative a, b, d of the 9th, e of the 10th and c of the 11th items. if someone selects all of these alternatives for these items, s/he gets 1.67 point for representativeness misconception table 3. correct reasoning skills correct reasoning skills items and alternatives sub-scale 1: correctly interprets probabilities 2d, 3d sub-scale 2: understands how to select an appropriate average 1d, 4c, 17c sub-scale 3: correctly computes probability a. understands probabilities as ratios 8c b. uses combinatorial reasoning 13a, 18b, 19a, 20b sub-scale 4: understands independence 9e, 10c-d-f, 11e sub-scale 5: understands sampling variability 14b, 15d sub-scale 6: distinguishes between correlation and causation 16c sub-scale 7: correctly interprets two-way tables 5: 1 d sub-scale 8: understands importance of large samples 6b, 12b, 7e-f table 4. misconceptions misconceptions: items and alternatives sub-scale 1: misconceptions involving averages a. averages are the most common number. 1a, 17e b. fails to take outliers into consideration when computing the mean 1c c. compares groups based on their averages 15b-f d. confuses mean with median 17a sub-scale 2: outcome orientation misconception 2e, 3ab, 11a-b-d, 12c, 13b sub-scale 3: good samples have to represent a high percentage of the population 7b-c, 16a-d sub-scale 4: law of small numbers 12a, 14c sub-scale 5: representativeness misconception 9a-b-d, 10e, 11c sub-scale 6: correlation implies causation 16b-e sub-scale 7: equiprobability bias 13c, 18a, 19d, 20d sub-scale 8: groups can only be compared if they are the same size 6a sub-scale since (1+1+1+1+1)/3 equals 1.67. the other misconceptions subscales scores are found in the same way. then, after scores of each misconception subscales’ scores are found, they are added in order to obtain an overall misconceptions score. this procedure is repeated in order to find each participant's score. turkish version of the instrument forward translation technique was used to translate the test from source language (english) to target language (turkish). then, validity and reliability evidences were collected for the turkish version of the test. firstly, it was translated into turkish by a professional translator, a graduate student studying at a mathematics education and the researchers of this study, international electronic journal of elementary education vol.7, issue 2, 107-124,2015 114 independently. after translation, the first and second author revised the translated instruments independently and after reaching on agreement, final version was constructed. then, a turkish teacher controlled the grammar of the test. finally four experts’ opinions on equivalence of both versions were obtained and then the test was finalized. afterwards, in order to establish empirical evidence for linguistic equivalence, 61 university students who were competent at both languages, took firstly the original instrument and three weeks later they took the turkish version. the number of participants should be at least 30 for this equivalence study because of the parametric analysis requirement (gay et al., 2009). also at least two weeks should be left between test administrations as a precaution (aksayan & gozum, 2002). if shorter than two weeks, participants might remember the items and higher scores might appear. therefore, they retook the turkish version of the instrument, three weeks later. the equivalence of the forms was examined at the item level. participants’ responses were coded as 1 for correct answers and 0 for incorrect answers. this coding procedure was done for each correct alternative for the questions with multiple correct answers. also blank answers were coded as 0. since variable of interest is dichotomous, data were matched-pair and data could be represented by 2x2 table, mcnemar test was used in analysis of the items (basturk, 2010). significant difference was not appeared in the items, except 1st, 2nd, 4th, 10th-c, 11th, 13th and 17th. back translated versions of these items and original versions were compared by a ph.d. candidate in mathematics education who is competent in both languages. since no difference was recognized, cross-tables of mcnemar test were examined. according to mcnemar analysis of these items, significant difference stemmed from the difference between incorrect answers in administration of original version and correct answers in administration of translated version. it was seen that most of the participants answered incorrectly in the original instrument whereas they answered correctly in turkish version. furthermore 1st, 4th and 17th items assess selecting an appropriate average and 2nd, 10th-c, 11th, and 13th items assess outcome orientation misconception. since these items are related to two subjects and their answers were incorrect in the original version while correct in turkish version, it is accounted for the data that significant difference might have resulted from participants’ learning of these subjects rather than the language. also, correlation coefficients were examined between overall scores obtained from turkish and english version of the test. first, normal distribution assumption was checked by kolmogorov-smirnov test for both overall correct reasoning skills score and overall misconceptions scores of turkish and english version (buyukozturk, 2010). for both overall scores in each group, normal distribution assumption were hold (p>.05) and statistically significant correlation coefficients were obtained; for overall correct reasoning skills score, r= 0.639, p < .01 and for overall misconceptions score, r= 0.337, p < .01. since test-retest reliability coefficient was calculated in the original study in which the test was developed, it was calculated in this study, too. the same 61 university students took turkish version of the sra three weeks later. kolmogorov-smirnov test for both overall scores in each administration shows that the distributions are normally distributed. statistically significant and high pearson correlation coefficient between results of two administrations for correct reasoning skills score [r= 0.756, p < .01] and misconceptions score, [r= 0.627, p < .01] were found. in the original study, reliability for the correct reasoning skills scale and for the misconceptions scale were found .70 and .75, respectively. so reliability results are parallel with the original study. teachers’ statistical reasoning / karatoprak, karagöz akar & börkan 115 procedures and data analysis data were collected during the teaching methods course of the secondary and elementary school mathematics education programs in each university. participation to the study was voluntary. for data analysis two sets of eight sub-scale scores and two overall scores were obtained (see instrument section for how to calculate scale scores). normal distribution assumption was checked by kolmogorov-smirnov test for each sub-scale and results showed that normality assumption was failed for each of them (p<.05). therefore comparison between groups was analysed by mann whitney u test for all of the subscales except distinguishes between correlation and causation and groups can only be compared if they are the same size subscales. these subscales were compared by chi-square test since there is only one item contributing to these subscales. at the same time, scores of both psmts and pemts were analysed descriptively for each subscale. results in this section, particular subscales (types) of correct reasoning skills and misconceptions were examined for both groups. then, statistical reasoning scores were compared across two groups. in terms of how to select an appropriate average, both groups’ mean scores were 0.454 and 0.447 and the percentages were 30.8% and 34.1% for the 1st and 46.2% and 36.6% for the 4th item in the sra. this showed that a small portion of prospective teachers could select mean as the meaningful average. this result was also consistent with the misconceptions involving averages subscale regarding the first item such that 61.6% of psmts and 56.1% of pemts had difficulties in determining the appropriate measure of center. for instance almost half of these prospective teachers who have misconceptions chose mode in place of mean and the other half of them overlooked the outlier in the data. however, results showed that both groups of prospective teachers (around 6% for both) were successful in not confusing mean and median while choosing the appropriate average for the given data set. as the data indicate in table 5, there was no statistically significant difference (u= 3676.5; p > .05, r=-.01) between pemts and psmts’ scores regarding how to select an appropriate average. table 5. mann whitney u test for how to select an appropriate average groups n mean rank sum of ranks u z p elementary level 82 86.34 7079.50 3676.5 -.174 .862 secondary level 91 87.60 7971.50 results from the 8th item, understanding probabilities as ratios, showed that both groups of prospective teachers were very successful (95.6% of psmts, 93.9% of pemts) in correctly computing probabilities. on the other hand, they were unsuccessful at using combinatorial reasoning in correctly computing probabilities, although for this domain there was not a statistically significant difference between the two groups (u= 3577; p > .05, r=-.04) (see table-6). in particular, the frequencies for answering correctly the items which assess use of combinatorial reasoning ranged between 11.0% and 31. 7%. these results were also consistent with the misconceptions regarding the equiprobability bias assessed by the same items. results showed that both groups of prospective teachers thought for instance the possibility of obtaining two 5 s and the possibility of obtaining one 5 and one 6 on two rolls equally likely since these events depend on chance. international electronic journal of elementary education vol.7, issue 2, 107-124,2015 116 table 6. mann whitney u test for correctly computes probabilities groups n mean rank sum of ranks u z p elementary level 82 88.88 7288 3577 -.538 .591 secondary level 91 85.31 7763 results from the understanding independence subscale also showed that although each group were successful in determining the equiprobability of obtaining for instance hhhtt or hthth for tossing a coin five times in the 9th item, they were not successful in identifying the plausible explanations for such occurrences. in particular, the explanation of “if you repeatedly flipped a coin five times, each of these sequences would occur about as often as any other sequence”, was chosen by only 31.9% of the psmts and 35.4% of the pemts as an explanation for the problem situation. similarly, relatively low percentage of them chose the c option, “any of the sequences could occur”, as an alternative explanation. on the other hand, 76.9% of the prospective secondary and 63.4% of the prospective elementary mathematics teachers identified “every sequence of five flips has exactly the same probability of occurring” as one of the correct explanations. these results injuxtaposition to each other suggested that both groups were not able to think of the probabilities in frequencies. interestingly, when prospective teachers’ responses were examined regarding the representativeness misconception, results showed that both groups did not have this misconception. although these important information was gained based on the analysis from the mean values and the frequencies, the mann whitney u test result showed that there was no statistically significant difference between both groups of prospective teachers (u=3644.5; p > .05, r=-.02) for the understanding independence subscale. (see table-7) table 7. mann whitney u test for understands independence groups n mean rank sum of ranks u z p elementary level 82 85.95 7047.50 3644.5 -.273 .785 secondary level 91 87.95 8003.50 results regarding the understanding sampling variability also showed no significant difference between the two groups (u= 3532; p > .05, r=-.06) (see table-8). the frequencies, 5.5% and 24.2% for psmts and 9.8% and 14.6% for pemts, from the 15th and the 14th items assessing this sub-scale were very low. these aligned with the results from the sub scale assessing misconceptions regarding averages such that 70.3% of the prospective secondary and 62.2% of the prospective elementary mathematics teachers considered only mean while comparing data from two independent groups without paying attention to the groups’ variances. table 8. mann whitney u test for understands sampling variability groups n mean rank sum of ranks u z p elementary level 82 84.57 6935 3532 -.805 .421 secondary level 91 89.19 8116 similarly, results from the items assessing the misconception, good samples representing a high percentage of the population showed that psmts and pemts did not take into account the absolute size of the sample. rather, they considered relative size of the sample having importance while making inferences about the population. yet, results from the mann whitney u test showed no differences between these two groups (u=3179.5; p > .05, r=-.13). (see table-9). teachers’ statistical reasoning / karatoprak, karagöz akar & börkan 117 table 9. mann whitney u test for good samples have to represents a high percentage of the population groups n mean rank sum of ranks u z p elementary level 82 80.27 6582.50 3179.5 -1.763 .078 secondary level 91 93.06 8468.50 results about understanding the importance of large numbers showed that both groups were successful in answering the problems assessing this sub-scale. however, for the 7th item, frequencies of both psmts (41.8%) and pemts (41.5%) who chose “the average could be a poor estimate of the spending of all teenagers given that teenagers were not randomly chosen to fill out the questionnaire” showed that most of them did not take into account randomness in sampling. on the other hand, results reporting on assessing prospective teachers’ understanding the importance of large numbers showed that there was no statistical difference between psmts and pemts (u=3571; p > .05, r=-.03). (see table-10) table 10. mann whitney u test for understands the importance of large numbers groups n mean rank sum of ranks u z p elementary level 82 85.05 6974.50 3571.5 -.505 .614 secondary level 91 88.75 8076.50 also related with the sampling, it seemed that not many of them had law of small numbers misconception since means of psmts (.335) and pemts (.402) was relatively low. however, when the items were examined at a fine grained level, results from the 14th item showed that quite a high percentage (50.5%) of psmts and (61%) of pemts chose the option indicating law of small numbers misconception. this suggested that sample size is not a matter of fact in sampling for both psmts and pemts. also this result aligned with their low scores in sampling variability. supporting high percentages of both groups in law of small number misconception, there was no statistically significant difference between psmts and pemts (u=3276; p > .05, r=-.12). (see table-11). table 11. mann whitney u test for law of small numbers groups n mean rank sum of ranks u z p elementary level 82 92.55 7589 3276 -1.531 .126 secondary level 91 82 7462 in terms of distinguishing between causation and correlation, results showed that relatively very small portion (31.9%) of psmts and (23.2%) of pemts were able to identify the relationship between two variables as association. at the same time the percentage of pemts (36.6%) and psmts (42.9%) who did not choose any of the alternatives indicating correlation implies causation misconception was not high. so it seemed that attributing a causal relationship between associated variables was quite common. chi-square test results in terms of distinguishing between causation and correlation also showed that scores from both prospective teacher groups did not differentiate from each other (x2=1.628; p > .05, φ=.09). (see table-12) international electronic journal of elementary education vol.7, issue 2, 107-124,2015 118 table 12. chi-square test for distinguishes between causation and correlation true false total x2 df p elementary level 19 63 82 1.628 1 .202 secondary level 29 62 91 total 48 125 173 discussion this study investigated statistical reasoning of pemts and psmts and the differences between their statistical reasoning. although one might expect otherwise, there was no statistical difference between pemts’ and psmts’ statistical reasoning on any of the subscales. jendrazsek (2010) argued that prospective mathematics teachers' success on probability and/or statistics is related with taking courses during their entire education. in this study, psmts’ (85.7%) and pemts’ (90.2%) took statistics courses. also the contention is that both groups might have been educated under the same national curricula. therefore, aligned with the findings of jendrazsek (2010), no statistically significant difference between these two groups’ reasoning becomes meaningful. on the other hand, results regarding different reasoning subscales both supported the earlier research results (groth & bergner; 2006, canada, 2008; leavy, 2006) and extended it by reporting on what specific pitfalls prospective teachers have in statistical reasoning. in particular, results from selecting appropriate measures of center and understanding variability together with the misconception subscales showed that psmts and pemts selected mode rather than mean as an appropriate average. they also did not take into account outliers in the data set. these indicated that they might not have paid attention to the categorical and interval/ratio variables while examining the data. that is, their difficulties in selecting appropriate average might have stemmed from their lack of reasoning about different types of variables since use of average is determined according to the type of data (gay et al., 2009). although they did not have the misconception that groups can only be compared if they are the same size, they lacked reasoning about measures of spread; a high percentage of both groups did not take into account the variability in the data while comparing different groups. in terms of garfield’s (2003) statistical reasoning domains, these results indicated that both groups of prospective mathematics teachers lacked the reasoning about data and statistical measures. results regarding the computing probabilities and the misconceptions about equiprobability bias showed that prospective teachers were very successful in solving a problem /item, taking out a marble from a box, very similar to those they encounter in turkish textbooks. yet, a high percentage of them were unsuccessful in computing probabilities requiring combinatorial reasoning. this indicated that they have equiprobability bias. these results also suggested that they were not able to take into account the sample space. by the same token, results from the understanding independence subscale showed that both groups were able to determine the equiprobability of obtaining hhhtt or hthth from tossing a coin five times. however, a low percentage of them explained their reasoning by choosing i) “if you repeatedly flipped a coin five times, each of these sequences would occur about as often as any other sequence.” and ii) “any of the sequences could occur”. whereas, a high percentage of them chose “every sequence of five flips has exactly the same probability of occurring”. in fact, the problems/items such as tossing a coin in different number of times are also seen frequently in turkish textbooks. in this respect, these results together with the findings from computing probabilities suggested that these prospective teachers might have known the theoretical probabilities; however, they might not have thought of constructing sample teachers’ statistical reasoning / karatoprak, karagöz akar & börkan 119 spaces experimentally, indicating a limitation on their part in terms of using the frequency approach . similarly, over 90 % of prospective teachers’ solving these problems theoretically indicated that they did not have representativeness misconception. however, not choosing i and ii as explanatory statements for equiprobability of the given sequences might indeed be taken as evincing that the representativeness misconception could be concealed by prospective teachers’ success in computing the probabilities in these items theoretically. in this regard, we propose to modify and/or change items assessing representativeness misconception in sra. prospective teachers’ lack of reasoning with sample space also corresponded with the results from assessing good samples represents a high percentage of the population and law of small numbers misconceptions. neither group of prospective teachers did take into account the absolute size of the sample nor did they pay attention to the sample size. quite a high number of them ignored the effect of small samples on results; that is, results vary more in small samples (well, pollatsek & boyce, 1990). so they could not take into account variation in sampling. similarly, both groups were successful in answering the problems assessing the understanding importance of large samples subscale. however, a low percentage of both groups of prospective teachers choosing “the average could be a poor estimate of the spending of all teenagers given that teenagers were not randomly chosen to fill out the questionnaire” suggested that prospective teachers did not take into account the importance of random choice in making inferences from samples to populations. in terms of garfield’s (2003) statistical reasoning domains, all these results injuxtaposition to each other suggested that, these prospective teachers lacked reasoning with sampling since randomness, representativeness and bias are critical notions to be considered to generalize results from samples to population (watson & moritz, 2000). as literature suggests, teachers' knowledge affect students' knowledge (heaton & mickelson, 2002; yolcu, 2012).therefore, if prospective teachers have difficulties in understanding these concepts such as reasoning about data, statistical measures, sampling and combinatorial reasoning or misconceptions such as equiprobability bias and representativeness misconception students will have lack of understanding in these concepts or these misconceptions. in order to develop correct reasoning, mathematics education programs were recommended to include teaching probability and statistics courses (bulut, 2001). however in our study 90.2% of the pemts took a course to teach probability and statistics. 85.7% of the psmts took statistics courses. so, contents of statistics courses in teacher education programs might be improved taking the results of this study into consideration. moreover previous research showed that the more courses prospective teachers take, the higher their performance is in probability (jendraszek, 2010). therefore, the number of courses to teach statistics might be increased in mathematics education programs so that their correct reasoning skills might develop. additionally, participants of this study are pemts and psmts studying in istanbul. also participants were not selected randomly because of practical reasons. so the results of this study are contextual and cannot be generalized to all prospective mathematics teachers. therefore more studies with different participants especially from other regions of turkey need to be conducted in order to have an idea about statistical reasoning of turkish prospective teachers. lastly, instrument was obtained from sra which was not developed originally for turkish context. there are correctly answered items by most of the participants in both groups. for instance, the 8th item which involved the context of drawing out marbles from two boxes might be replaced with new ones. likewise, results showed that most of the participants international electronic journal of elementary education vol.7, issue 2, 107-124,2015 120 answered the 2nd item correctly. this item could be excluded from the test or the context of the item could be changed in order to lessen the familiarity with the problem situation. regarding the context of turkish education system, especially for university entrance exam, students frequently solve similar and/or much more difficult items during preparation. therefore, prospective teachers’ familiarity might result in high scores and less information about their reasoning in these items. at the same time, there are items that definitely should be kept in sra. for instance, items like the 13th, 18th,19th or 20th could be used to determine prospective teachers’ use of combinatorial reasoning in computing probabilities. taken all these into account, a new instrument might be developed or sra might be modified. • • • rabia karatoprak graduated from secondary school integrated bs & ms program in teaching mathematics program of boğaziçi university in 2011. she earned m.s. degree from the department of secondary school science and mathematics education at boğaziçi university in 2014. currently she is a ph.d. student in the university of iowa educational measurement and statistics program. her research interests include instrument development and statistics education. gulseren karagoz akar earned her b.s degree from the department of secondary science and mathematics education at the middle east technical university in 1996. she pursued on her master and doctorate studies in mathematics education field at the pennsylvania state university and got her ph.d. in 2007. currently, she is a faculty member at the department of secondary school science and mathematics education at boğaziçi university. her current interests include students’ learning of mathematics through their own activity, conceptual understanding in mathematics, and prospective teachers’ conceptualization processes of effective mathematics teaching. bengu borkan received her bs (1999) from middle east technical university in chemistry education, and her ms (2001) and her phd degree (2006) from the ohio state university in quantitative research, evaluation and measurement in education. she also holds a minor degree in cognitive psychology. from march 2007 to december 2007, she worked as an instructor in dokuz eylül university in i̇zmir/turkey before joining bogazici university in january 2008. her current research interests include cognitive psychology in survey research, web-based survey, attitude measurement, measurement in education and program evaluation. references aksayan s., & gözüm. s. (2002). kültürlerarası ölçek uyarlaması için rehber i: ölçek uyarlama aşamaları ve dil uyarlaması. hemşirelik araştırma dergisi; 4(1):9-14 baştürk, r. (2010). bütün yönleriyle spss örnekli nonparametrik istatiksel yöntemler. ankara: anı yayıncılık. bulut, s. (2001). matematik ögretmen adaylarının olasılık performanslarının i̇ncelenmesi. hacettepe universitesi egitim fakültesi dergisi, 20, 33-39. buyukozturk, s. (2006). sosyal bilimler i̇çin veri analizi el kitabı. pegem yayınları, ankara. canada, d. l. (2008). conceptions of distribution held by middle school students and preservice teachers. in c. batanero, g. burrill, c. reading, and a. rossman (eds.), teachers’ statistical reasoning / karatoprak, karagöz akar & börkan 121 proceedings of the icmi study 18 and 2008 iase round table of conference. retrieved from http://iase-web.org/documents/papers/rt2008/t2p9_canada.pdf. chance, b. (2002). components of statistical thinking and implications for instruction and assessment. journal of statistics education, 10(3). delmas, r. c. (2002). statistical literacy, reasoning, and learning: a commentary. journal of statistics education, 10(3). fennema, e., & franke, m. l. (1992). teachers' knowledge and its impact. in d. a. grouws (ed.), handbook of research on mathematics teaching and learning: a project of the national council of teachers of mathematics (p. 147-165). macmillan publishing co, inc. fritz, c. o., morris, p. e., & richler, j. j. (2012). effect size estimates: current use, calculations, and interpretation. journal of experimental psychology: general,141(1), 2. gal, i. (2002). adults' statistical literacy: meanings, components, responsibilities. international statistical review, 70(1), 1-25. garfield, j. (2002). the challenge of developing statistical reasoning. journal of statistics education, 10(3), 1-11. garfield, j. b. (2003). assessing statistical reasoning. statistics education research journal, 2(1), 22-38. garfield, j., & ben‐zvi, d. (2007). how students learn statistics revisited: a current review of research on teaching and learning statistics. international statistical review, 75(3), 372-396. garfield, j. b., & ben-zvi, d. (2008). developing students’ statistical reasoning. connecting research and teaching practice. the netherlands: springer. garfield, j., & gal, i. (1999). teaching and assessing statistical reasoning. nctm 1999 yearbook: mathematical reasoning, pp. 1-17. retrieved from https://apps3.cehd.umn.edu/artist/articles/garfield01.pdf. gay, l. r., mills, g. e., & airasian, p. (2009). educational research: competencies for analysis and applications. upper saddle river, nj, usa: pearson education. groth, r. e., & bergner, j. a. (2005). preservice elementary school teachers' metaphors for the concept of statistical sample. statistics education research journal, 4(2), 27-42. groth, r. e., & bergner, j. a. (2006). preservice elementary teachers' conceptual and procedural knowledge of mean, median, and mode. mathematical thinking and learning, 8(1), 37-63. heaton, r. m., & mickelson, w. t. (2002). the learning and teaching of statistical investigation in teaching and teacher education. journal of mathematics teacher education, 5 (1), 35-59. jendraszek, p. (2010). an analysis of a misconception of probability among future mathematics teachers. journal of mathematics education at teachers college, 1 (1). kahneman, d., & tversky, a. (1974). subjective probability: a judgment of representativeness. in the concept of probability in psychological experiments (pp. 25-48). springer netherlands. konold, c., pollatsek, a., well, a., lohmeier, j., & lipson, a. (1993). inconsistencies in students' reasoning about probability. journal for research in mathematics education, 24, 392-392. leavy, a. m., (2006). using data comparison to support a focus on distribution: examining preservice teacher's understandings of distribution when engaged in statistical inquiry. statistics education research journal, 5, (2), 89-114. lecoutre, m. p. (1992). cognitive models and problem spaces in “purely random” situations. educational studies in mathematics, 23(6), 557-568. makar, k., & confrey, j. (2005). variation-talk: articulating meaning in statistics. statistics education research journal, 4(1), 27-54. international electronic journal of elementary education vol.7, issue 2, 107-124,2015 122 mickelson, w. t., & heaton, r. m. (2004). primary teachers’ statistical reasoning about data. in the challenge of developing statistical literacy, reasoning and thinking (pp. 327-352). springer netherlands. milli eğitim bakanlığı, (2013a). i̇lköğretim matematik dersi 6-8. sınıflar öğretim programı ve kılavuzu. ankara. milli eğitim bakanlığı, (2013b). ortaöğretim matematik dersi 9-12. sınıflar öğretim programı ve kılavuzu. ankara. moore, d. s. (1990). uncertainty. in steen, l. a. (ed.). on the shoulders of giants: new approaches to numeracy (pp. 96-137). national academies press. nctm. (2005). principles and standards for school mathematics. reston, va, usa: national council of teachers of mathematics. özen, m. (2012), investigation of pre-service mathematics teachers’ critical thinking processes through statistical and probabilistic knowledge in the context of popular media texts. middle east technical university. unpublished m. s. thesis. rumsey, d. j. (2002). statistical literacy as a goal for i̇ntroductory statistics courses. journal of statistics education, 10(3), 6-13. shaughnessy, j. m. (2007). research on statistics learning. in f. k. lester (ed.), second handbook of research on mathematics teaching and learning (p. 957-1009). iap. smith, m. h. (2004). a sample/population size activity: is it the sample size of the sample as a fraction of the population that matters? journal of statistics education, 12(2). sundre, d. l. (2003). assessment of quantitative reasoning to enhance educational quality. in american educational research association meeting. chicago. tempelaar, d. (2004). statistical reasoning assessment: an analysis of the sra instrument. in the artist roundtable conference on assessment in statistics. appleton, wisconsin, citeseer. tversky, a., & kahneman, d. (1971). belief in the law of small numbers. psychological bulletin, 76(2), 105. ulutaş, f., & ubuz, b. (2008). research and trends in mathematics education: 2000 to 2006. i̇lköğretim online dergisi, 7. watson, j., & callingham, r. (2003). statistical literacy: a complex hierarchical construct. statistics education research journal, 2(2), 3-46. well, a. d., pollatsek, a., & boyce, s. j. (1990). understanding the effects of sample size on the variability of the mean. organizational behaviour and human decision processes, 47(2), 289-312. yolcu, a. (2012). an investigation of eighth grade students’ statistical literacy, attitudes towards statistics and their relationship. m. s. thesis. middle east technical university. unpublished m. s. thesis. teachers’ statistical reasoning / karatoprak, karagöz akar & börkan 123 appendix a sample items of the statistical reasoning assessment (garfield, 2003) 1. a small object was weighed on the same scale separately by nine students in a science class. the weights (in grams) recorded by each student are shown below. 6.2 6.0 6.0 15.3 6.1 6.3 6.2 6.15 6.2 the students want to determine as accurately as they can the actual weight of this object. of the following methods, which would you recommend they use? __ a. use the most common number, which is 6.2. __ b. use the 6.15 since it is the most accurate weighing. __ c. add up the 9 numbers and divide by 9. __ d. throw out the 15.3, add up the other 8 numbers and divide by 8. 4. a teacher wants to change the seating arrangement in her class in the hope that it will increase the number of comments her students make. she first decides to see how many comments students make with the current seating arrangement. a record of the number of comments made by her 8 students during one class period is shown below. student initials number of comments a.a. r.f. a.g. j.g. c.k. n.k. j.l. a.v. 0 5 2 22 3 2 1 2 she wants to summarize this data by computing the typical number of comments made that day. of the following methods, which would you recommend she use? __ a. use the most common number, which is 2. __ b. add up the 8 numbers and divide by 8. __ c. throw out the 22, add up the other 7 numbers and divide by 7. __ d. throw out the 0, add up the other 7 numbers and divide by 7. 9. which of the following sequences is most likely to result from flipping a fair coin 5 times? __ a. h h h t t __ b. t h h t h __ c. t h t t t __ d. h t h t h __ e. all four sequences are equally likely 14. half of all newborns are girls and half are boys. hospital a records an average of 50 births a day. hospital b records an average of 10 births a day. on a particular day, which hospital is more likely to record 80% or more female births? __ a. hospital a (with 50 births a day) __ b. hospital b (with 10 births a day) __ c. the two hospitals are equally likely to record such an event. 15. forty college students participated in a study of the effect of sleep on test scores. twenty of the students volunteered to stay up all night studying the night before the test (no-sleep group). the other international electronic journal of elementary education vol.7, issue 2, 107-124,2015 124 20 students (the control group) went to bed by 11:00 p.m. on the evening before the test. the test scores for each group are shown in the graphs below. each dot on the graph represents a particular student’s score. for example, the two dots above the 80 in the bottom graph indicate that two students in the sleep group scored 80 on the test. test scores: no-sleep group test scores: sleep group examine the two graphs carefully. then choose from the 6 possible conclusions listed below the one you most agree with. __ a. the no-sleep group did better because none of these students scored below 40 and the highest score was achieved by a student in this group. __ b. the no-sleep group did better because its average appears to be a little higher than the average of the sleep group. __ c. there is no difference between the two groups because there is considerable overlap in the scores of the two groups. __ d. there is no difference between the two groups because the difference between their averages is small compared to the amount of variation in the scores. __ e. the sleep group did better because more students in this group scored 80 or above. __ f. the sleep group did better because its average appears to be a little higher than the average of the no-sleep group. 18. when two dice are simultaneously thrown it is possible that one of the following two results occurs: result 1: a 5 and a 6 are obtained. result 2: a 5 is obtained twice. select the response that you agree with the most: __ a. the chances of obtaining each of these results is equal __ b. there is more chance of obtaining result 1. __ c. there is more chance of obtaining result 2. __ d. it is imposible to give an answer. (please explain why) international electronic journal of elementary education, 9(3), 645-666, march, 2017 issn:1307-9298 copyright © iejee www.iejee.com relations between prejudice, cultural intelligence and level of entrepreneurship: a study of school principals ali baltaci a  a ministry of national education, turkey received: 16 january 2017 / revised: 20 february 2017 / accepted: 28 february 2017 abstract the aim of this study is to determine the mediating role of prejudice in the relationship between the cultural intelligence of school principals and the level of entrepreneurship. the design of this study was classified as correlational survey research. this study was designed by quantitative research method. the universe of this study constitutes 642 school principals. in this study, the entire universe was reached and a "complete counting" technique was used. relations between the variables of this study were examined by correlation and hierarchical regression analysis. findings have shown that cultural intelligence is positively related to prejudice and entrepreneurship. as a result of the hierarchical regression analysis, the prejudice has been completely mediated-agent between the cultural intelligence and the level of entrepreneurship. the positive relationship between cultural intelligence and prejudice has been identified for the first time and this finding contributes exceptional data for literature. this research is also particularly important in that it is the first study to examine the relationship between the level of entrepreneurship, the prejudice, and the cultural intelligence. the findings are helpful to the educational and organizational literature, but in practice, they will provide strategic alternatives for school principals to engage in entrepreneurial activities. keywords: prejudice, cultural intelligence, level of entrepreneurship, school principals. introduction schools are social organizations within which many different kinds of problems can be encountered, although this kind of problems can be overcome with a support that can be taken from the outside. there are presences of various elements (such as, legal regulations, an inadequacy of capital and information, experience and communication problems) that restrict the entrepreneurial activities of school systems (balcı, 2008; çınkır, 2010). it is important that the internal factors are associated with decision makers (principals, managers etc.) among the factors that restrict the entrepreneurial activities of the schools. this is because, in schools, decisions are usually taken more centrally by principals and also, the decisions cannot be questioned by any school employee (ersoy, 2010; schein, 2010; sommer, 2010). additionally, the characteristics of the manager  address for correspondence: ali baltacı, ministry of national education, ankara, turkey. 06100. phone: +90 532 3514389 e-mail: alibaltaci@meb.gov.tr http://www.iejee.com/ international electronic journal of elementary education, 9(3), 645-666, march, 2017 646 (prejudice, personality, attitude and thoughts) directly influence the actions, climate, and productivity of the school (balcı et al., 2012). school principals' positive attitudes towards entrepreneurial actions can affect the school's success. at the same time, entrepreneurship comes to the forefront as an individual characteristic of the school principal. the entrepreneurial character that the school principal has is also an indication of his or her cultural accumulation (elenkov & manev, 2009). at this point, cultural intelligence, which is part of individual cultural accumulation, becomes evident. cultural intelligence is proposed to enable school administrators to carry out entrepreneurial activities such as having information about different people, interpreting this information, learning different cultural components and directing verbal or non-verbal behaviors towards the acquired information in the direction of supporting this entrepreneurship (aytaç & ilhan, 2007; mercan, 2016; triandis, 2006). cultural intelligence is a set of skills that enable an individual to function or succeed effectively in different cultural settings or in multicultural settings (brislin et al., 2006). cultural intelligence arises from interaction with other people, unlike cumulative cultural structures that one possesses. from this perspective, as yeşil (2010) points out, managers and leaders with cultural intelligence are the most important strategic assets for organizations. it is not the case that the cultural intelligence is considered separately from the prejudices of the person. prejudice is a concept associated with cultural intelligence. prejudice is a situation or a precondition that there has been a decision beforehand, without getting enough information about a person or a thing (dovidio & gaertner, 1986). prejudice is a form of behaviour with negative outcomes and has wide-ranging implications both on the individual and the organizational basis. one of the behaviours affected by prejudice is entrepreneurship, which is an important act of increasing organizational effectiveness and is usually manifested by managers. entrepreneurship can open new paths for executives to advance a new insight and purpose. the level of entrepreneurship, on the other hand, is a positive attitude towards managers' entrepreneurial activities (shapero & sokol, 1982). recent research has shown that managers' cultural intelligence and level of entrepreneurship are contributing positively to the entrepreneurial activities of organizations (antonio et al., 2014; crowne, 2013; fidan & ozturk, 2015; javalgi et al., 2014). in addition, sommer (2010) reports that managers approach to entrepreneurial activities without prejudice may affect the level of entrepreneurship. in the literature, studies on the level of entrepreneurship seem to be mostly confined to determining the personal characteristics of entrepreneurs (bozkurt & erdurur, 2013; ersoy, 2010; fiş & wasti, 2009; ramirez, 2010; sookhtanlo et al., 2009). in addition, cultural intelligence studies have been worked on with different concepts such as personality traits, negotiation, individual performance, interpersonal trust, organizational citizenship (balcı et al., 2012; ilhan & cetin, 2014; imai & gelfand, 2010; lee & sukoco, 2010; rockstuhl & ng, 2008). entrepreneurship and cultural intelligence have been the subject of various researches. cultural intelligence, however, has been identified as a concept emerging in research (arastaman, 2013; chen et al., 2012; crowne, 2013; elenkov & manev, 2009; yeşil, 2010,). there are studies on the concepts of cultural intelligence and prejudice (çetin, 2014; greenwald, 2014; lin et al., 2012; mercan, 2016; rentschler, 2012; simpson & yinger, 2013; vernon, 2014; yilmaz & kaya, 2016). however, there are no published studies that deal with the concepts of cultural intelligence, prejudice, and entrepreneurship at the same time. this study aims to determine the nature of the relationship between concepts. in addition, eliminating the shortcomings of the field is another purpose. the study focuses primarily on the relations between prejudice, cultural intelligence and level of entrepreneurship / baltacı 647 intermediary role of the prejudice between the level of cultural intelligence that is desired to be studied but which has not yet been studied, and the level of entrepreneurship. the present study revealed unique associations that were not found in the current research results in the literature. research has the distinction of being the first and foremost research that studies cultural intelligence, prejudice, and level of entrepreneurship with descriptive analysis. theoretical framework as a concept, cultural intelligence can be expressed as the ability to adjust and effectively manage the relationship with different people and cultures. in addition, cultural intelligence is the ability to solve problems (early & mosakowski, 2004). cultural intelligence (cq) is a composite of intercultural talents and skills (earley & ang, 2003). gardner's (1999) cultural intelligence, based on multiple intelligence theory, is associated with cognitive intelligence (iq) and emotional intelligence (eq) which are widely used in the field of management. today, cognitive, emotional, and social intelligence are not enough for global managers (brislin et al., 2006), and they are required to have the cultural intelligence to solve problems and be successful in multicultural environments. global corporations perceive cultural intelligence as a strategic skill and competitive advantage. cultural metacognition, together with cultural knowledge and skills, constitutes cultural intelligence, resulting in behaviour that includes cultural intelligence (thomas et al., 2008). the concept of cultural intelligence, which is relatively new and up-to-date in the field of organization and management, is defined differently in many studies. although there is no general consensus among the definitions, there are certain characteristics of cultural intelligence. cultural intelligence is characterized by adapting to new cultural environments (earley & ang, 2003), intercultural interactions (van dyne & ang, 2005), understanding the basis of intercultural interaction and rational approach, creating skills and behaviours that can be applied in different intercultural settings (moon, 2010) can be determined to successfully adapt to unusual, diverse and new cultural environments (earley et al., 2006) and to interact with people with cultural differences (thomas & inkson, 2009). cultural intelligence is defined as "a system of knowledge and skills connected with the cultural metacognition in which people are able to adapt and shape the cultural conditions in their environment" (thomas et al., 2008). the components of cultural intelligence are classified as metacognitive, cognitive, motivational, and behavioural, based on the multi-dimensional intelligence model of sternberg and detterman (1989). metacognitive cultural intelligence includes mental processes that the individual has in understanding and acquiring knowledge about different cultures (özsoy et al., 2009; simpson & yinger, 2013). cognitive cultural intelligence refers to the individual's knowledge of the economic, social or legal structures and cultural values of different cultures (lee & sukoco, 2010). motivational cultural intelligence is the capacity to direct the individual's energy and attention in the context of different cultures in the direction of obtaining information about these cultures (ang & van dyne, 2015). behavioural cultural intelligence refers to the capacity of the individual to exhibit verbal or non-verbal behaviours in the context of different cultures (vernon, 2014). similarly, managerial cultural intelligence expresses the capacity of managers to successfully interact with all stakeholders that are associated with the organization in order to be able to successfully interact with the individuals from different cultures and at the same time to reach the goals and objectives of the organization (stevenson & jarillo, 2007). international electronic journal of elementary education, 9(3), 645-666, march, 2017 648 in the literature on cultural intelligence: the effects of cultural management training on cultural intelligence (eisenberg et al., 2013), the five-factor personality traits and fourfactor cultural intelligence relationship (van dyne et al, 2006), the differences between emotional and social intelligence and cultural intelligence (kim et al., 2011; moon, 2010), the relationship between international experience and cultural intelligence (crowne, 2008; shannon & begley, 2008; tay et al., 2008), the effects of short-term intercultural experience on the dimensions of cultural intelligence (engle & crowne, 2014; wood & peters, 2014) have been examined. in a number of studies, it has been found that intercultural management training is more effective on the metacognitive and cognitive dimensions of cultural intelligence (eisenberg et al., 2013). in some studies, it has been determined that short-term abroad experiences are influential in all dimensions of cultural intelligence, except the behavioural dimension (wood & peters, 2014). as a personality trait, it turned out that being outwardly related to all dimensions of cultural intelligence (ang et al., 2007). yeşil (2009) examines the concept of cultural intelligence as a strategy used in the management of cultural diversity and analyses the practices of different businesses related to the topic. yeşil (2010) interrogates the concepts of culture, intelligence, and cultural intelligence. sahin (2011) examined the leader's cultural intelligence on subordinates' organizational citizenship behaviour and the effects of job satisfaction on leaders and subordinates working in an international military organization. it has been determined that the cultural intelligence components that are effective are motivational and behavioural dimensions. as a result of research conducted by şahin and gürbüz (2012) on multinational employees of a global organization, it has been found that cultural intelligence dimensions are positively related to organizational citizenship behaviour and task performance. sahin and colleagues (2013) studied the existence of relationships between cultural intelligence, personality, and emotional intelligence on two separate samples involving employees and students in international accommodation operations. i̇şçi and colleagues (2013) also revealed the level of cultural intelligence in hospital enterprises with field research. as a result of the research, it was determined that the duration of institution and occupation was weakly correlated with the behavioural dimension and general score of cultural intelligence in the negative direction. çapraz and colleagues (2009), conceptually examined managerial intelligence. in the study is emphasized that analytical, practical, creative, emotional, social, moral, spiritual, strategic, aesthetic, linguistic, spatial and kinaesthetic intelligence together with cultural intelligence constitute managerial intelligence in total. the concept of entrepreneurship refers to enriching the workplace and realizing innovations in the production process (covin & slavin, 1991). the level of entrepreneurship refers to the extent to which an organization is engaged in entrepreneurial activities, and it can be determined by measures such as performancebased measures, structural measures, and behavioural measures (sullivan, 1994). entrepreneurship, which constitutes the basic dynamic of economic development, has a special precaution in terms of contributing to individual and social welfare. the concept of entrepreneurship, expressed in different forms in the historical process, in essence, can be defined as organizing jobs to make profits (baltacı & balcı, 2017). it is possible to define entrepreneurship as the work of establishing a new organization and taking the initiative to organize some socioeconomic mechanisms and accept the risk of failure (shaphero, 2002). hisrich (1990), described entrepreneurship as the duration of composing something dissimilar with importance by dedicating the indispensable time and endeavour, assuming the accompanying fiscal, psychological, and social risks, and receiving the resulting relations between prejudice, cultural intelligence and level of entrepreneurship / baltacı 649 rewards of monetary and personal satisfaction. the concept of entrepreneurship, in the early periods, was defined as to establish a business and undertake risks, subsequently, the concept was expanded by introducing innovative ideas, transforming these ideas into products and services, and adding marketing activities (fidan & balcı, 2016; kirzner, 2015). in this point, to manage the investment process derived from the new ideas, to catch and to evaluate of opportunities, to risk of resources to use advantage of opportunities and ultimately the creation of a value is one of the most important qualities that brings a person entrepreneurial character and differentiates him from other individuals in society (hisrich & peters, 2002). the level of entrepreneurship is the cumulative amount of innovative thinking that exists in the person (sommer, 2010). if one has entrepreneurial qualities, the degree of showing these qualities indicate his level of entrepreneurship (yukl, 2002). the level of entrepreneurship can be increased by supporting the innovative and entrepreneurial aspects of the person. from this point of view, the level of entrepreneurship defines a learning process (fiş & wasti, 2009). however, the level of entrepreneurship is influenced by various demographic variables, including age, educational background, and seniority. it is also emphasized in the literature that the managerial skills and managerial experience of a person also affect the level of entrepreneurship (aytaç & i̇lhan, 2007). the level of entrepreneurship is also based on one's previous experiences. entrepreneurial activities in one's career constitute his entrepreneurial experience (takeuchi, 2005). entrepreneurial experience can also be achieved in a learning outcome (crow, 1992). entrepreneurship orientation is a learning process that increases the frequency of showing entrepreneurial behaviour patterns in the person. entrepreneurship orientation is a training process; one learns how to show entrepreneurial behaviour during this training (covin & slevin, 1986). prejudice is a way of taking sides in intra-organizational relations (arastaman, 2013). it is also used to mean an unconditional support for an ideological idea or point of view. the prejudice is that a person's decisions are unilaterally emerging in a predominant way. prejudice is also used to express that a person's decisions are subjective. prejudice does not claim that a person does not use objective values while taking decisions. in short, the values that constitute the judgments of a person bring a discourse against the integrity. prejudice can influence both individual and organizational decisions, and it can interfere with all activities of the organization. prejudiced managers can make misleading decisions on many issues, including entrepreneurship and innovation. prejudice expresses the negative orientation of managers towards entrepreneurial organization activities (çubukçu, 2009; kirzner, 2015). in literature, the concepts of prejudice and instinct are often used in the same sense. prejudice is often a negative attitude, while instinct is good or bad information about a phenomenon. prejudice has been the subject to many kinds of studies in the field of management. these studies are usually focused on the prejudiced negative processes of decision making and discrimination behaviour. most studies in the literature are theoretical, and in these studies, discrimination is discussed with a social psychological approach and prejudice is defined as discrimination, stereotyping, authoritarian personality (göregenli, 2012; paker, 2012). prejudice also has important implications in organizational life. it is a fact that in organizations, individuals have various prejudices because of their age, their profession, their sex, and race (tutkun & koç, 2008). in organizations, prejudices can emerge in various forms. prejudices can also arise between employees, decisions, beliefs; etc. such prejudices lead to behaviour that damages innovation, entrepreneurship, and productivity in organizations. these behaviours are humiliating jokes, swearwords, harassment, and international electronic journal of elementary education, 9(3), 645-666, march, 2017 650 bullying; more confidential applications might be low pay, slow promotion, and systematic removal from various acquisitions, tighter control, high expectations and poor communication. the most prevalent studies in social psychology, psychology, and education in recent years include ethnic or racial prejudices (autry, 2010; gürses, 2005), gender-based discriminations or prejudices (koca & bulgu, 2005; kurt, 2007; riley, 2016; swank & raiz, 2010), prejudices against ethnic or cultural groups (coşgun, 2004; gorski, 2008; kunstman, 2011; zick et al., 2001), prejudices against disabilities (ballard, 1997; deal, 2007; tufan, 2008), prejudices in written texts (ağaçsapan, 1996), and prejudice-religion relations (sezen, 2002). in the field of education, gender-based prejudices (baç, 1997; skliar, 2007; şimşek, 2010), religious prejudices (kunduz, 2009), administrative prejudices (bilir, 2000), and prejudices against foreign cultures (gencer, 2009) were researched. in addition, issues of reducing or preventing prejudices are frequently discussed (göregenli, 2013; harrell, 2011). it is also possible to find prejudices among the employees in the schools. in a research that examines the obstacles of female teachers to become managers, male managers have shown prejudiced attitudes towards female teachers (şimşek, 2010). according to this, males are convinced that male managers have little desire for female teachers to work with them. female teachers are not as tied up as men to their careers and are not as durable as men. also, they are emotional and do not have the ability to make effective decisions. the male managers' belief in this pattern of judgment leads to negative and prejudiced attitudes towards the appointment of female teachers as administrators. there are few studies in the literature on obstacles in front of schools' entrepreneurship. among the most important obstacles faced by schools in these researches are: qualified manpower (chau & pederson, 2000; uluğ, 1998), fear of competition (karagöz, 2006; leonidou, 1995), organizational and operational barriers (balcı, 1988; hamill & gregory, 1997), limited information about stakeholders (leonidou, 2004; su, 2003), technical impossibilities (balcı, 2008; czinkota & ronkainen, 2013; kume et al., 2001) and constrained financial resources (korkmaz, 2005; leonidou, 1995). in addition to the mentioned obstacles, administrative inadequacies are also important (çınkır, 2010). the centralized decision-making mechanism in schools is quite common and administrators can reflect their own personal characteristics, behaviours, values, prejudices and cultures on these decisions. this can be explained by the “upper echelons theory." according to this theory, the functions of organizations, the strategies they perform and the performance ratings are influenced by the character of the manager (hambrick, 2007, hambrick & mason, 1984). hambrick and mason (1984), the prejudices of the managers influence strategic decisions, and these are directly reflected in the entrepreneurship level of the organization. barney's resource based view (1991) allocates the resources to be identified into three categories to provide prioritized strategic direction. in this categorization, which is divided into material resources, human resources, and organizational resources, managers are considered as internal factors affecting the organization. according to barney, these resources belonging to the organization must be valuable, rare, easily substitutable and non-imitative. in the same way, the development of these four criteria in managerial talents provides a competitive advantage to the organization in terms of internal resources. in this context, the importance of the manager in entrepreneurial activities is to create both business performance and competitive advantage. while some managers efficiently fulfil their managerial roles in intercultural environments, others do not. yukl (2002) states that besides other features of the relations between prejudice, cultural intelligence and level of entrepreneurship / baltacı 651 managers have to understand the point of views of employees and other sharers who came from different cultures and to respect on their behaviours. with a similar approach, schein (2010) notes that the culture and the manager are like 'two sides of a penny'. cultural intelligence can be explained by understanding, understanding and internalizing the cultural differences that are so important for managers. briefly, the components of cultural intelligence also include in: to be aware of a foreign cultural differences, to be knowledgeable about those cultures, to be willing to learn those cultures, and to reflect cultural images and values in their verbal or non-verbal behaviours (neff et al, 2005; rentschler, 2002). in the literature, studies on this subject have shown that managers with cultural intelligence show their talents better in entrepreneurial activities (ang et al., 2007, deng & gibson, 2008). in this context, it is possible to say that managers take an active role in organizational entrepreneurship by taking into account the ‘theory of the upper echelons.’ also, in the literature, the cultural intelligence of school principals in entrepreneurial activities was found to be significantly higher rather than the cultural intelligence of non-entrepreneurial school administrators (crow, 1992; snell et al., 2016). cultural intelligence is significant in terms of the prejudice of the administrators when evaluated with the composition of cognitive cultural intelligence. hence, having knowledge about other cultures influences the behaviours and decisions of individuals (swedberg, 2006). behaviours and decisions are also motivated by prejudice attitudes, and judgments by the individual are emerging in this direction (greenwald, 2014; sanbonmatsu & fazio, 1990). in their research, romano and platania (2014) mentioned that cultural intelligence influenced by individual prejudices. other research on cultural intelligence reveals that individuals with high cultural intelligence have a more effective decision-making mechanism in entrepreneurial settings (ang & inkpen, 2008). at the same time, past entrepreneurial experiences and foreign language knowledge can also influence the individual's prejudice in the positive direction. research on this subject has also been related to the fact that the entrepreneurial experience of the managers’ contributes to the ability to live in different cultural environments and to have the ability to do business with these cultures (bandura, 1997; takeuchi et al, 2005). on the other hand, school principals who can develop themselves on the entrepreneurial skills can make more strategic decisions on this issue (knight & kim, 2009; wiklund et al., 2009). when assessed in this direction, it is possible to say that cultural intelligence developed with past experiences and foreign language knowledge is influential on the prejudices of the individual. on the other hand, the personal characteristics, behaviours, and values of the principals who play an important role in the schools, directly affect the level of entrepreneurship of the school (hambrick, 2007, morgan, 2007). hence, behaviour and decision-making are motivated by personal prejudices. in this context, it can be said that the entrepreneurial levels of organizations can be restricted by the prejudices of the manager against the entrepreneurial activities. the most prominent research that revealed this relationship in the related field was done by knight (2001). according to knight, the entrepreneurial tendencies of the managers are controlled by their prejudices, which directly indicate the performance of the organization. in fact, it will be conceivable to say that the prejudice of the manager against the entrepreneurial activities will be moulded by the cultural intelligence and will directly affect the performance of the organization. research in this connection, firstly, shows that school principals' entrepreneurial experiences and their knowledge of foreign languages play a vital role in international projects and other entrepreneurial pursuits (etemad & wright, 2003). entrepreneurial experience and prejudice are among the elements of international electronic journal of elementary education, 9(3), 645-666, march, 2017 652 cultural intelligence (early & ang, 2003; thomas & inkson, 2009). wilson and stewart (2009) found that increasing the entrepreneurial experience and the level of cultural intelligence of the principals, have lessened their prejudice. shokef and erez (2008) noted that multicultural teamwork also enhances cultural intelligence and entrepreneurship, yet diminish prejudice. de palma and dobes (2010) designated that entrepreneurial experiences enhance cognitive cultural intelligence. managers with high cultural intelligence found to be disassociated from prejudice (romano & platania, 2014). positive prejudice, however, affects positively on school performance (knight, 2001). from all these discussions, the aims and hypotheses of the research are presented below. purpose and hypotheses of research the main purpose of this study is to investigate the mediating role of prejudice between the cultural intelligence of school principals and the level of entrepreneurship. for this purpose, this study examined the relationship between school principals' cultural intelligence, prejudice, and level of entrepreneurship. hypothesis 1 (h1): the cultural intelligence of school principals affects the level of entrepreneurship of the school positively. hypothesis 2 (h2): school principals' cultural intelligence are positively influencing on their prejudices. hypothesis 3 (h3): school principals' prejudices about entrepreneurship affect the level of entrepreneurship of the school positively. hypothesis 4 (h4): there is a mediating effect of school principals' prejudice between cultural intelligence and level of entrepreneurship. method research design a causal-comparative design was used as a descriptive survey model in this research. in this phase, the ‘cultural intelligence, level of entrepreneurship and prejudice (cilep) scale’ was used to determine the school principals’ views. furthermore, some statistical techniques including the exploratory factor analysis (efa), the confirmatory factor analysis (cfa), correlation and hierarchical regression analysis were used to analyse the data. sampling this study was designed by quantitative research method. the universe of this study constitutes 642 elementary school principals in ankara in 2016. in this study, the entire universe was reached, and a "complete counting" technique was used. it was determined that the survey scales reached to the target universe were completely filled. the elementary school principals included in the sample of this study has dwelled in the central and country districts of ankara. since access to all schools is difficult, the data collected from school principals were provided by official letters of the ministry of national education and official e-mail. in addition, the researcher in order to gather the data by went to the schools individually which were easy to access. as a result, 642 elementary school principals were reached in this study. the schools in this study were evaluated according to the number of employees: 9% micro scale schools (number of employees is 9 or less), 32.7% middle scale schools (10-50 employees) and 58.3% of them in large scale schools (the number of employees is 51-250). the average age of school principals participating in the study was 40.48 years (sd = 7.77) and the management experience was 7.57 years (sd = 2.04). principals whose majority was male (86.7%); 28% of them had graduate level education, and 72% had education at undergraduate level. relations between prejudice, cultural intelligence and level of entrepreneurship / baltacı 653 measures the data in the study were collected by questionnaire method. the questionnaire includes the demographic information of the principals, as well as the variables and the control variables to be used in the study. the questionnaire used a 20-items scale developed by the researcher as a cultural intelligence scale. for the analysis, kaiser-meyer-olkin (kmo) value was found to be 0.941 and barlett's test was significant (p <0.01). there is a total of 20 items in total, and it is determined that there are 4 factors that will be the result of exploratory factor analysis. correlations between the factors were found to vary between .31 and .54. as a result of the item analysis, it was determined that the total correlations of corrected items of the items on the scale ranged from .49 to .92. on the scale, reliability coefficients of cultural intelligence components were found to be above .70 (metacognitive α= .91, cognitive α= .84, motivational α= .85 and behavioural α= .89). the cultural intelligence scale was subject to confirmatory analysis. the results of the analysis of the cultural intelligence scale appear to be acceptable. (χ2 = 1098.96; p= 0.00, df= 346, χ2 /df = 3.17; ifi= .97, rfi = .95, rmr = .057, gfi = .83, agfi = .80, cfi = .97, nnfi = .97, nfi = .96 and rmsea = .075). prejudice scale consisted of 7 items developed by the researcher by drawing from the literature on prejudice. this scale’s kaiser-meyer-olkin (kmo) value was found to be 0.937 and barlett's test was significant (p <0.01). on this scale, 7 items combine under one factor and reliability factor of the scale (α) is .80. it was determined that the total correlations of items on the scale ranged from .32 to .84. the prejudice scale was examined by confirmatory factor analysis. the analysis values for the scale are acceptable (χ2= 1041,63, p<0.05, df= 326, χ2 /df= 3.19; rmsea = 0.053, nfi = 0.98, nnfi = 0.98, cfi = 0.99, ifi = 0.99, gfi = 0.98, agfi = 0.96, srmr = 0.025). the entrepreneurship scale, developed by the researcher, and it consisted of 12-items. this scale’s kaiser-meyer-olkin (kmo) value was found to be 0.897 and barlett's test was significant (p <0.01). on this scale, 12 items combine under one factor. it was determined that the total correlations of items on the scale ranged from .42 to .86 and also, the confirmatory analysis was applied to this scale, and the results were acceptable (χ2 = 1051.17, p<.000, df= 316, χ2 /df= 3.32; rmsea= 0.063, s-rmr= 0.048, gfi= 0.91, agfi= 0.89, cfi= 0.98, nnfi= 0.98, ifi= 0.98). the level of entrepreneurship scale’s reliability factor (α) is .83. as a result, cilep has 3 dimensions and 39 items (cultural intelligence: 20, prejudice: 7 and level of entrepreneurship: 12) and also, these items explain 74.85% of the total variance. the overall cronbach alpha coefficient for cilep is .88. according to this measurement tool, the entrepreneurship level of schools was determined with 5 different proportional measures. each proportional criterion on this scale corresponds to a value between zero and one, and the sum gives the level of entrepreneurship. in the direction of the suggestions in the related literature, age and managerial experience of the principals (ferwerda et al., 2013), entrepreneurship experience (ang & van dyne, 2015) and entrepreneurship orientation (covin & slevin, 1986) used for the control variables in this study, results before testing, research hypotheses, decomposition validity of variables, were determined by using 'confirmatory factor analysis'. thereafter, three different scales were integrally subjected to confirmatory analysis. in this analysis, three-factor (single-factor cultural intelligence + prejudice + entrepreneurship) model statistics (χ2= 1074.96; p= 0.00, df= 320, χ2 /df = 3.35; rmsea = .079, gfi = .95, cfi = .93, ifi = .91, nnfi = .89) were found to be acceptable. according to these results, the data obtained from the research showed that international electronic journal of elementary education, 9(3), 645-666, march, 2017 654 it complies with the one-factor model of cultural intelligence, and the analysis procedures continued in this direction. in the study, correlation analysis was performed to determine the level of relations between variables. table 1 shows the results of the correlation analysis between the main variables of the study (the cultural intelligence, the prejudice, the level of entrepreneurship) and control variables (age, managerial experience, and entrepreneurship experience and entrepreneurship orientation). table 1. mean (m), standard deviation (sd) and correlation analysis results of variables variables sd 1 2 3 4 5 6 7 1 age 40,48 7,77 2 managerial experience 7,57 2,01 .48** 3 entrepreneurship experience 3,67 1,92 .17* .01 4 entrepreneurship orientation 4,54 0,67 .01 .02 .07 5 cultural intelligence 4,96 0,98 .04 -.18* .16* .25** 6 prejudice 4,65 2,02 .13 -.14 .04 .26** .65** 7 level of entrepreneurship 0,32 0,11 -.04 -.29** .13 .12 .37** .44** **p<.01 *p<.05 when the results in table 1 are examined, it is found that cultural intelligence, which is an independent variable, is positively related to entrepreneurial experience (r= .16, p<.05) and entrepreneurship orientation (r= .25, p<.01) and negatively related to managerial experience (r= -.18, p<.05). there is a positive correlation between cultural intelligence and prejudice (r = .65, p <.01). the level of entrepreneurship is found to be positively correlated with both cultural intelligence (r = .37, p <.01) and prejudice (r = .44, p <.01). a 'hierarchical regression analysis' was applied in the direction of baron and kenny's (1986) four-step approach in order to determine the mediating role of related prejudice between cultural intelligence and level of entrepreneurship in the direction of the research. in the first stage of analysis, age, managerial experience, entrepreneurial experience and entrepreneurship orientation were entered as control variables. in the second stage, it has been determined that the effect of cultural intelligence on the level of entrepreneurship has not diminished, or has not fully developed, through prejudice. the results of the regression analysis are given in table 2. table 2. hierarchical regression analysis results variables prejudice level of entrepreneurship step 1 step 2 step 1 step 2 control variables age .26** .14* .10 .03 .10 -.03 managerial experience -.27** -.10 -.34** -.24** -.34** -.20** entrepreneurship experience -.03 .02 .12 .13 .12 .13 entrepreneurship orientation .26** .11 -.06 .15* -.06 .19* independent and instrumental variables cultural intelligence .60*** .36*** .13 prejudice .37*** r2 .142 .458 .118 .233 .118 .309 f 6.00*** 24.29*** 4,84** 8.76*** 4.84** 10.65*** ***p<.001; **p<.01; *p<.05 the values in the table are standardized beta (β) coefficients. relations between prejudice, cultural intelligence and level of entrepreneurship / baltacı 655 according to the results in table 2, it was found that cultural intelligence, which is the independent variable of the study, has a positive effect on the level of entrepreneurship (β = .36, p<.001). at this point, the first hypothesis of the study (h1) is accepted. the cultural intelligence is positively related to prejudice, which is instrumental variable (β = .60, p <.001) and with this result, the second hypothesis (h2) is accepted. moreover, the prejudice which is an instrumental variable has also a positive effect on the level of entrepreneurship (β = .37, p<.001) and with this result, the third hypothesis of research (h3) is accepted. when the instrumental variable (prejudice) has added on the model, the cultural intelligence effect on the level of entrepreneurship has decreased from β= .36 (p<.001) to β=.13 (p>.05). this result shows that the variable of prejudice is completely mediated by the influence of cultural intelligence on the level of entrepreneurship. afterward, the mediation role of the prejudice on the relationship between cultural intelligence and entrepreneurship has examined by the sobel test (1982), and the result of the mediation test has been found to be significant (z= 3.41, p<.01). for this reason, the fourth hypothesis (h4) is accepted directly. conclusions, discussions and recommendations there was not enough measurement between cultural intelligence and entrepreneurship in the field. in addition, it has been mentioned that studies on prejudice are mostly limited to measuring the frequency and existence of the behaviour. the examining purpose of this research is to fill this gap in the literature. that is, the focus is on the relationship between the cultural intelligence and the level of entrepreneurship. it also concentrates on the concept of prejudice, which is thought to be important for both cultural intelligence and entrepreneurship and, which is not yet adequately studied in the literature. the main motivation for this study has been the lack of comparative research among the prejudice and the other concepts in the field. this research is also particularly important in that it is the first study to examine the relationship between the level of entrepreneurship, the prejudice, and the cultural intelligence. this study aims to provide a theoretical contribution by examining the notion of cultural intelligence, prejudice, and level of entrepreneurship, which are a subject that have not been studied extensively and, which have become increasingly important in the literature. the aim of this study is to determine the mediating role of prejudice in the relationship between the cultural intelligence of school principals and the level of entrepreneurship. findings show that the cultural intelligence is positively associated with both the prejudice and the level of entrepreneurship. it is also noticed that the relationship between the prejudice and the level of entrepreneurship is also positive. in this situation, the first, second and third hypotheses of the study are accepted. the result of the hierarchical regression analysis is that the prejudice is completely mediated in the relationship between the cultural intelligence and the level of entrepreneurship. in other words, school principals who have the cultural intelligence are not the prejudiced about the level of entrepreneurship. school principals without prejudice, execute it easier for the school to realize entrepreneurial activities and with these results, the fourth hypothesis of the research is also accepted. the research findings were thoroughly answered by the research question, and it can be determined that hypotheses established within the scope of the research are also fully answered. according to the results of the research, the level of entrepreneurship of the school principals who increase cultural intelligence will also increase. on the other hand, the increase in the prejudice is positively related to the cultural intelligence. this finding is interesting. school principals are expected to reduce their prejudice when they learn about different cultures. however, this research has shown that this is not the case. as cultural intelligence in the people increases, they become more intimate with other international electronic journal of elementary education, 9(3), 645-666, march, 2017 656 people, and this extra information may be a source of prejudice for them. likewise, cultural intelligence leads people to use their knowledge from the past, and people can increase them even more, instead of taking away the wrong beliefs they once had. whatever happens, this study's findings are brand new in the literature. it is thought that it will have an important effect on the literature. however, the reasons for such a result should be investigated. this study has provided a lot of new problems in the literature. this gap in the literature is important for future research. recent study results show that managers' cultural intelligence is reflected in their prejudices and that developing ideas in this direction have an impact on entrepreneurship. cultural intelligence, unlike generally known intelligence, can be improved over time (early & ang, 2003). research has shown that entrepreneurial experiences and foreign language knowledge develop cultural intelligence (simpson & yinger, 2013; takeuchi et al., 2005). at this point, it also says that school principals' entrepreneurial experiences, foreign language knowledge, and education also affect the entrepreneurial activities of the school positively (calof & beamish, 1995; etemad, 2003). some research shows that managers take a key role in the entrepreneurship of schools (knight, 2001). the prejudices, perspectives, behaviours, and decisions of principals are seen as the most important factors influencing entrepreneurship (lloyd-reason & mughan, 2002; morgan, 2007). prejudice-related research shows that behaviour and decision-making are influenced by prejudices (greenwald, 2014; sanbonmatsu & fazio, 1990). according to knight (2001), principals' entrepreneurial tendencies influence their prejudices, which directly reflect the performance of the organization. it demonstrates that school principals' prejudices are effective in the entrepreneurial activities of the school and supports this study' results. when broadly assessed, prejudices towards the entrepreneurship of the manager with cultural intelligence are developing positively (romano & platania, 2014) and this unprejudiced approach increases the degree of entrepreneurship. this indicates that the findings obtained in the study are in the different direction as the related literature. the result is different in this present research. in the studies mentioned above the cultural intelligence behaviour increases whereas prejudice decrease. the increase in the cultural intelligence must have obviously triggered different behaviours that were ignored in this research. these new and unpredictable behaviours may also be composed of the prejudices. it should be determined what caused this phenomenon. the social sciences, on the other hand, are based on the belief that any phenomenon can be changed. social scientists know that there are no stationary universal rules in any place. that is, if a result is different, it is important. this research differs from the prevailing context and concept of the cultural intelligence. with this new finding to filling a significant gap in the literature, and it has also provided an important alternative way to new research. in particular, what causes the new data presented in this study should be investigated. thus, with this study, more contributions can be made to the literature. initial studies in the literature on cultural intelligence have focused on individuals in organizations, and these studies have demonstrated the positive results of the cultural intelligence on an individual level, focusing on the ability of the individual to adapt to the intercultural environments (ang et al, 2007). however, cultural intelligence researches conducted in organizations have been investigated by associating with cases such as intercultural leadership, communication competence effect, task performance, resiliency and organizational citizenship behavior (ang & van dyne, 2015; balcı et al, 2012). later studies have shown that cultural intelligence is also effective in team work as well as on an individual level (rockstuhl & ng, 2008; shokef & erez, 2008). recent studies show that cultural intelligence is also positive at the organizational level for example; ang & inkpen relations between prejudice, cultural intelligence and level of entrepreneurship / baltacı 657 (2008), have studied cultural intelligence at the organizational level and found that the level of cultural intelligence to be possessed at the organizational level is effective in the organizations holding the entrepreneurship in the foreground. moon (2010), on the other hand, revealed that organizational cultural intelligence positively affects organizational performance. this present study has provided important findings that will contribute to the study of cultural intelligence at the organizational level which is limited in the related literature. at the same time, this study contributes valuable practical outputs for effective use. school principals with cultural intelligence are expected to be willing to learn foreign cultures (early & ang, 2003). principals who have progressed in accordance with these demands are inclined to participate in national and international events without any prejudice to the intercultural environments and to be able to organize their behaviours as having knowledge about the cultures they meet. as mentioned above, principals with entrepreneurial experiences and knowledge of foreign languages ensure that organizations succeed in entrepreneurial activities. in this context, in the schools where managers play a key role, national and international projects are organized; it is suggested that managers should participate in international congresses, conferences, and fairs and at the same time improve their cultural intelligence by advancing foreign language knowledge as much as possible. in addition to the examinations performed, this study has been applied to the community of school principals, and it relies on self-reports of these participants. various conclusions may be collected in studies in which cross papers and several groups are practiced. besides, data acquired from participants can provide researchers with further purified and broadened perspectives. as in any study, this study has some limitations. likewise, the individual and organizational levels of factors, including age, managerial experience, entrepreneurship experience, and entrepreneurship orientation were included in this study as it was realized that the studies in the literature concerning whether demographic and organizational agents have effects on the cultural intelligence and entrepreneur behaviour forms of school principals are limited. it may be advisable to conduct similar studies with various demographic and instrument variables. hereafter, research should focus on further investigating the influences of these factors on entrepreneurship, cultural intelligence, and prejudice in complex circumstances and on additional control variables including an organizational and a social structure that may have produced excluded a variable problem. another assignment for future research is an evaluation of the inter-relationships among prejudice and cultural behavior constructs and partly discussed in this study. additionally, the variety of different school types, other regions, and different countries will broaden the outcomes reached. as a result, in this study, it was revealed that cultural intelligence of school principals will affect the degree of entrepreneurship and prejudices developing in this direction will enable schools to achieve success by playing an active role in entrepreneurial activities. the connections among these behavioral patterns and positive organizational upshots (individual and organizational performance development, the improvement in student achievement, etc.) should also be investigated in future research. • • • international electronic journal of elementary education, 9(3), 645-666, march, 2017 658 references ağaçsapan, a. (1996). stereotype und vorurteile im türkischen und im deutschen eine studie zu stereotypen in authentischen und engagierten schriftlichen (unpublished doctoral dissertation). ankara: hacettepe university, institute of social science. ang, s. & inkpen, a. c. (2008). cultural intelligence and offshore outsourcing success: a framework of firm-level intercultural capability. decision sciences, 39(3), 337-358. ang, s. &van dyne, l. (2015). handbook of cultural intelligence. new york: routledge. ang, s.,van dyne, l., koh, c., ng, k. y., templer, k. j., tay, c. & chandrasekar, n. a. (2007). cultural intelligence: its measurement and effects on cultural judgment and decision making, cultural adaptation and task performance. management and organization review, 3(3), 335-371. antonio, t., lanawati, s., wiriana, t. a., & christina, l. (2014). correlations creativity, intelligence, personality, and entrepreneurship achievement. procedia-social and behavioral sciences, 115, 251-257. arastaman, g. (2013). eğitim ve fen edebiyat fakültesi öğrencilerinin öz-yeterlik inançları ve öğretmenlik mesleğine karşı tutumlarının incelenmesi. [examination of education and arts and sciences faculty students’ self-efficacy beliefs and their attitudes toward teaching profession]. ahi evran üniversitesi kırşehir eğitim fakültesi dergisi, 14(2). 205-215. autry, s. p. (2010). racial socialization and racial identity construction of african american children regarding racial issues and prejudice (unpublished doctoral dissertation). los angeles: california lutheran university. aytaç, ö. & i̇lhan, s. (2007). girişimcilik ve girişimci kültür: sosyolojik bir perspektif [entrepreneurship and entrepreneurial culture: a sociological perspective]. selçuk üniversitesi sosyal bilimler enstitüsü dergisi, 18, 101-120. baç, g. (1997). a study on gender-bias in teachers’ behaviors, attitudes, perceptions and expectations toward their students (unpublished master dissertation). ankara: middle east technical university, institute of social sciences. balcı, a. (1988). etkili okul. [effective school]. eğitim ve bilim. 12, (70). 21-29. retrieved from: http://egitimvebilim.ted.org.tr/index.php/eb/article/view/5979 balcı, a. (2008). türkiye’de eğitim yönetiminin bilimleşme düzeyi.[ evolution of educational administration as a scientific discipline in turkey]. kuram ve uygulamada eğitim yönetimi dergisi, 14(2), 181-209. balcı, a., baltacı, a., fidan, t., cereci, c. & acar, u. (2012). örgütsel sosyalleşmenin, örgütsel özdeşleşme ve örgütsel vatandaşlıkla ilişkisi: i̇lköğretim okulu yöneticileri üzerinde bir araştırma. [the relationship between organizational socialization and organizational identification and organizational citizenship. a research on elementary school principals]. eğitim bilimleri araştırmaları dergisi, 2(2), 47-74. ballard, k. (1997). researching disability and inclusive education: participation, construction and interpretation. international journal of inclusive education, 1(3), 243-256. baltaci, a. & balcı, a. (2017). complexity leadership: a theorical perspective. international journal of educational leadership and management, 5(1), 30-58. bandura, a. (1997) social learning theory. englewood cliffs, nj: prenticehall. barney, j. (1991). firm resources and sustained competitive advantage. journal of management, 17(1), 99-120. baron, r. m. & kenny, d. a. (1986). the moderator–mediator variable distinction in social psychological research: conceptual, strategic, and statistical considerations. journal of personality and social psychology, 51(6), 1173. relations between prejudice, cultural intelligence and level of entrepreneurship / baltacı 659 bilir, a. (2000). eğitim yöneticilerinin ve öğretmenlerin yönetime ilişkin önyargıları. [prejudices of education administrators and teachers on governance] (unpublished doctoral dissertion). ankara: ankara university, institute of social sciences. bozkurt, ö. & erdurur, k. (2013). girişimci kişilik özelliklerinin girişimcilik eğilimindeki etkisi: potansiyel girişimciler üzerinde bir araştırma [the contribution of education to entrepreneurial sentiments: an example of kütahya vocational school the department of machine]. girişimcilik ve kalkınma dergisi, 8(2). 33-56. brislin, r., worthley, r. & macnab, b. (2006). cultural intelligence understanding behaviors that serve people’s goals. group and organization management, 31(1), 40-55. calof, j. l. & beamish, p. w. (1995). adapting to foreign markets: explaining internationalization. international business review, 4(2), 115-131. chau, s. & pedersen, s. g. (2000). small is beautiful: the emergence of new micro business utilising electronic commerce. in proceedings of the 11th australasian conference on information systems.1-8. chen, x. p., liu, d. & portnoy, r. (2012). a multilevel investigation of motivational cultural intelligence, organizational diversity climate, and cultural sales: evidence from us real estate firms. journal of applied psychology, 97(1), 93. coşgun, ş. (2004). kültürlerarası i̇letişim sürecinde kalıp düşüncelerin ve önyargıların rolü: “antalya’da yaşayan güneydoğulular i̇le antalya yerlileri arasındaki kalıp düşünceler ve önyargılar” [the role of mold thinking and prejudices in the process of cultural communication: "mold thinking and prejudices between the southeast sides living in antalya and antalya locals"] (unpublished master dissertion). ankara: ankara unversity, institute of social sciences. covin, j. g. & slevin, d. p. (1986). the development and testing of an organizational-level entrepreneurship scale. frontiers of entrepreneurship research, 1(1986), 626-639. covin, j. g. & slevin, d. p. (1991). a conceptual model of entrepreneurship as firm behavior. entrepreneurship theory and practice, 16(1), 7-25. crow, g. m. (1992). the principal in schools of choice: middle manager, entrepreneur, and symbol manager. the urban review, 24(3), 165-174. crowne, k. a. (2013). cultural exposure, emotional intelligence, and cultural intelligence: an exploratory study. international journal of cross cultural management, 13(1), 5-22. czinkota, m. r. & ronkainen, i. a. (2013). international marketing. cengage learning publishers, u.s.a. capraz, b., kesken, j., ayyildiz, n. a. & ilic, d. (2009). yönetsel zeka’ya dogru: yönetsel zeka ve bileşenlerini tanımlamaya yönelik kavramsal bir çalışma [a directive towards managerial intelligence: a conceptual study to identify managerial intelligence and its components]. ege academic review, 9(1), 187-211. çınkır, s. (2010). problems of primary school headteachers: problem sources and support strategies. elementary education online, 9(3), 1027-1036. cubukcu, f. (2009). learner autonomy, self regulation and metacognition. international electronic journal of elementary education, 2(1), 53-64. çetin, ç. k. (2014). kültürel zekâ: uzakyol kaptanları ve uzakyol birinci zabitleri üzerinde bir araştırma [cultural intelligence: a research on master mariners and chief officers]. i̇ktisadi ve i̇dari bilimler fakültesi dergisi, 16(2), 134-155. deal, m. (2007). aversive disablism: subtle prejudice toward disabled people. disability and society, 22(1), 93-107. deng, l. & gibson, p. (2008). a qualitative evaluation on the role of cultural intelligence in crosscultural leadership effectiveness. international journal of leadership studies, 3(2), 181-197. international electronic journal of elementary education, 9(3), 645-666, march, 2017 660 de palma, r. & dobes, v. (2010). an integrated approach towards sustainable entrepreneurship– experience from the test project in transitional economies. journal of cleaner production, 18(18), 1807-1821. dovidio, j. f. & gaertner, s. l. (1986). prejudice, discrimination, and racism: historical trends and contemporary approaches. academic press. earley, p. c. & ang, s. (2003). cultural intelligence: individual interactions across cultures. ca: stanford university press. earley, p. c. & mosakowski, e. (2004). cultural intelligence. harvard business review, 82(10), 139146. eisenberg, j., lee, h. j., brück, f., brenner, b., claes, m. t., mironski, j. & bell, r. (2013). can business schools make students culturally competent? effects of cross-cultural management courses on cultural intelligence. academy of management learning and education, 12(4), 603-621. elenkov, d. s. & manev, i. m. (2009). senior expatriate leadership's effects on innovation and the role of cultural intelligence. journal of world business, 44(4), 357-369. engle, r. l. & crowne, k. a. (2014). the impact of international experience on cultural intelligence: an application of contact theory in a structured short-term programme. human resource development international, 17(1), 30-46. ersoy, h. (2010). kültürel çevrenin girişimcilik tercihine etkisi [the effects of region and regional culture in entrepreneurship]. organizasyon ve yönetim bilimleri dergisi, 2(1), 71-77. etemad, h. & wright, r. w. (2003). internationalization of smes: toward a new paradigm. small business economics, 20(1), 1-4. ferwerda, j., kattenberg, m., chang, h. h., unger, b., groot, l. & bikker, j. a. (2013). gravity models of trade-based money laundering. applied economics, 45(22), 3170-3182. fidan, t. & oztürk, i. (2015). the relationship of the creativity of public and private school teachers to their intrinsic motivation and the school climate for innovation. procedia-social and behavioral sciences, 195, 905-914. fidan, t. & balcı, a. (2016). principal proactivity: school principals’ proactive. applied chaos and complexity theory in education, 29. fiş, a. m. & wasti, s. a. (2009). örgüt kültürü ve girişimcilik yönelimi ilişkisi [relation between organizational culture and entrepreneurship orientation]. metu studies in development (muhan soysal special issue), 35, 127-164. gardner, h. (1999). intelligence reframed: multiple intelligences for the 21st century. basic books. gencer, ö. (2009). eğitim fakülteleri alman dili eğitimi anabilim dalı derslerinde yabancı kültürle karşılaşma ve önyargılar: ampirik-nitel bir araştırma [faculties of education german language education faculties and prejudices against foreign cultures in their subject areas: an empiricalqualitative research] (unpublished doctoral dissertion). çanakkale: çanakkale onsekiz mart university, institution of social sciences. gorski, p. (2008). the myth of the``culture of poverty''. educational leadership, 65(7), 32. göregenli, m. (2012). temel kavramlar: önyargı, kalıpyargı ve ayrımcılık. ayrımcılık çok boyutlu yaklaşımlar [basic concepts: prejudice, constitutionalism and discrimination. discrimination multidimensional approaches] in (ed. kenan çayır, müge ayan ceyhan), i̇stanbul: i̇stanbul bilgi üniversitesi yayınları l, 17-27. göregenli, m. (2013). ayrımcılığın meşrulaştırılması [legitimation of discrimination]. m. çınar (der.), medya ve nefret söylemi, 39-54. greenwald, a. g. (2014). why attitudes are important: defining attitude. attitude structure and function, 429. relations between prejudice, cultural intelligence and level of entrepreneurship / baltacı 661 gürses, i̇. (2005). önyargının nedenleri [causes of prejudice]. uludağ üniversitesi i̇lahiyat fakültesi dergisi, 14(1), 143-161. hambrick, d. c. (2007). upper echelons theory: an update. academy of management review, 32(2), 334-343. hambrick, d. c. & mason, p. a. (1984). upper echelons: the organization as a reflection of its top managers. academy of management review, 9(2), 193-206. hamill, j. & gregory, k. (1997). internet marketing in the internationalisation of uk smes. journal of marketing management, 13(1-3), 9-28. harrell, d. c. (2011). effects of multicultural education courses and biblical education courses in changing attitudes and prejudice. proquest llc. 789 east eisenhower parkway, po box 1346, ann arbor, mi 48106. hisrich, r. d. (1990). entrepreneurship/intrapreneurship. american psychologist, 45(2), 209. hisrich, r. d. & michael. p. peters (2002). entrepreneurship. london: mcgraw-hill. imai, l. & gelfand, m. j. (2010). the culturally intelligent negotiator: the impact of cultural intelligence (cq) on negotiation sequences and outcomes. organizational behavior and human decision processes, 112(2), 83-98. i̇lhan, m. & çetin, b. (2014). kültürel zekâ ölçeği'nin türkçe formunun geçerlik ve güvenirlik çalışması [validity and reliability study of the turkish version of the cultural intelligence scale]. hacettepe üniversitesi eğitim fakültesi dergisi, 29(29-2). i̇şçi, e., söylemez, ö. & kaptanoğlu, a. y. (2013). örgütlerde kültürel zekâ ve hastane işletmelerinde kültürel zekâ düzeyinin belirlenmesine yönelik bir araştırma [a study to determine the level of cultural intelligence in organizations and the level of cultural intelligence in hospital enterprises]. kafkas üniversitesi i̇ktisadi ve i̇dari bilimler fakültesi dergisi, 4(5), 1-18. javalgi, r. g., hall, k. d. & cavusgil, s. t. (2014). corporate entrepreneurship, customer-oriented selling, absorptive capacity, and international sales performance in the international b2b setting: conceptual framework and research propositions. international business review, 23(6), 1193-1202. karagöz, b. k. (2006). okul yöneticilerinin yönetim süreçleri açısından karşılaştıkları problemler [problems faced by school administrators in terms of management processes] (unpublished master dissertation). edirne: university of trakya, institute of social sciences. kirzner, i. m. (2015). competition and entrepreneurship. university of chicago press. kim, d., basu, c., naidu, g. m. & cavusgil, e. (2011). the innovativeness of born-globals and customer orientation: learning from indian born-globals. journal of business research, 64(8), 879-886. knight, g. a. (2001). entrepreneurship and strategy in the international sme. journal of international management, 7(3), 155-171. knight, g. a. & kim, d. (2009). international business competence and the contemporary firm. journal of international business studies, 40(2), 255-273. koca, c. & bulgu, n. (2005). spor ve toplumsal cinsiyet: genel bir bakış [sport and gender: an overview]. toplum ve bilim, 103. 163-184. korkmaz, i̇. (2005). i̇lköğretim okullarının karşılaştıkları finansman sorunları [faced financing problems of elementary school]. selçuk üniversitesi sosyal bilimler enstitüsü dergisi, (14), 429434. kume, h., anderson, p. & de oliveira júnior, m. (2001). non tariff barriers to trade in mercosur: how do brazilian exporting firms perceive them?. retrived at: https://papers.ssrn.com/sol3/papers2.cfm?abstract_id=292945 international electronic journal of elementary education, 9(3), 645-666, march, 2017 662 kunduz, c. ö. (2009). an exploration of the predictors of prejudice among students with and without headcover (unpublished master dissertation). i̇stanbul: boğaziçi university. institute of social sciences. kunstman, j. (2011). white≠ poor: status threats enhances intragroup bias (unpublished doctoral dissertation). miami: the florida state unıversıty. kurt, b. (2007). dansçı erkeğe yönelik önyargılı bakışın" sinop köçekleri" örneği üzerinden incelenmesi [analysis of the prejudices against the male dancer in the case of "sinop köçeks"] (unpublished doctoral dissertation). i̇stanbul: i̇stanbul kültür university, institution of social sciences. lee, l. y. & sukoco, b. m. (2010). the effects of cultural intelligence on expatriate performance: the moderating effects of international experience. the international journal of human resource management, 21(7), 963-981. leonidou, l. c. (1995). empirical research on export barriers: review, assessment, and synthesis. journal of international marketing, 29-43. leonidou, l. c. (2004). an analysis of the barriers hindering small business export development. journal of small business management, 42(3), 279-302. lin, y. c., chen, a. s. y. & song, y. c. (2012). does your intelligence help to survive in a foreign jungle? the effects of cultural intelligence and emotional intelligence on cross-cultural adjustment. international journal of intercultural relations, 36(4), 541-552. lloyd-reason, l. & mughan, t. (2002). strategies for internationalisation within smes: the key role of the owner-manager. journal of small business and enterprise development, 9(2), 120-129. mercan, n. (2016). çok kültürlü ortamlarda kültürel zekânın kültürler arası duyarlılık i̇le i̇lişkisine yönelik bir araştırma [a study on the relationship between cultural intelligence and inter-cultural sensitivity in multi-cultural environments]. ömer halisdemir üniversitesi i̇ktisadi ve i̇dari bilimler fakültesi dergisi, 9(1), 1-13. moon, t. (2010). emotional intelligence correlates of the four-factor model of cultural intelligence. journal of managerial psychology, 25(8), 876-898. morgan, k. (2007). the learning region: institutions, innovation and regional renewal. regional studies, 41(s1), 147-159. neff, g., wissinger, e. & zukin, s. (2005). entrepreneurial labor among cultural producers:“cool” jobs in “hot” industries. social semiotics, 15(3), 307-334. ozsoy, g., memis, a. & temur, t. (2009). metacognition, study habits and attitudes. online submission, 2(1), 154-166. paker, m. (2012). psikolojik açıdan önyargı ve ayrımcılık [psychological prejudice and discrimination]. çok boyutlu yaklaşımlar, in (ed. kenan çayır, müge ayan ceyhan), i̇stanbul: i̇stanbul bilgi üniversitesi yayınları l, 41-53. ramirez, a. r. (2010). impact of cultural intelligence level on conflict resolution ability: a conceptual model and research proposal. emerging leadership journeys, 3(1), 42-56. rentschler, r. (2002). the entrepreneurial arts leader: cultural policy, change and reinvention. university of queensland press. riley, s. r. (2016). re-racing the nation: from cuba—a drama of freedom to the cultural performance of negro history week. in performing race and erasure (pp. 101-140). palgrave macmillan uk. rockstuhl, t. & ng, k. y. (2008). the effects of cultural intelligence on interpersonal trust in multicultural teams. handbook of cultural intelligence: theory, measurement, and applications, 206-20. relations between prejudice, cultural intelligence and level of entrepreneurship / baltacı 663 romano, j. & platania, j. (2014). attitudes towards internationalism through the lens of cognitive effort, global mindset, and cultural intelligence. the journal of international education and leadership, 4(1), 1-11. sanbonmatsu, d. m. & fazio, r. h. (1990). the role of attitudes in memory-based decision making. journal of personality and social psychology, 59(4), 614. schein, e. h. (2010). organizational culture and leadership (vol. 2). new york: john wiley and sons. sezen, a. (2002). dinin önyargılar üzerindeki çift yönlü etkileri [bi-directional effects on prejudices of religion] (unpublished master dissertation). i̇zmir: dokuz eylül university, institute of social sciences. shannon, l. m. & begley, t. m. (2008). antecedents of four-factor model of cultural intelligence (pp. 41-55). me sharpe. shapero, a. (2002). the displaced, uncomfortable entrepreneur. entrepreneurship: critical perspectives on business and management, 2, 251-265. shapero, a. & sokol, l. (1982). the social dimensions of entrepreneurship. encyclopedia of entrepreneurship, 72-90. shokef, e. & erez, m. (2008). cultural intelligence and global identity in multicultural teams. handbook of cultural intelligence: theory, measurement, and applications, 177-191. simpson, g. e. & yinger, j. m. (2013). racial and cultural minorities: an analysis of prejudice and discrimination. springer science and business media. skliar, o. s. (2007). gender representations and gender bias in elt textbooks published in the middle east: a case study of elt textbooks published in turkey and iran (unpublished doctoral dissertation). ankara: middle east technical unıversity, institute of social sciences. snell, r., chan, m. & zou, t. (2016). key practices of leadership for service in hong kong. in international conference on service leadership education in service economies. hong kong. sommer, l. (2010). internationalization processes of small-and medium-sized enterprises—a matter of attitude?. journal of international entrepreneurship, 8(3), 288-317. sookhtanlo, m., rezvanfar, a., hashemi, s. m. & karaj, i. (2009). psychological capabilities affecting agricultural students' entrepreneurship level: a comparative study. research journal of agriculture and biological sciences, 5(2), 175-184. sternberg, r. j. & detterman, d. k. (eds.). (1986). what is intelligence?: contemporary viewpoints on its nature and definition. nj: praeger pub text, norwood. stevenson, h. h. & jarillo, j. c. (2007). a paradigm of entrepreneurship: entrepreneurial management. in entrepreneurship (pp. 155-170). springer berlin heidelberg. su, s. (2003). export barriers insights from small and medium-sized firms. international small business journal, 21(4), 403-419. sullivan, d. (1994). measuring the degree of internationalization of a firm. journal of international business studies, 25(2), 325-342. swedberg, r. (2006). the cultural entrepreneur and the creative industries: beginning in vienna. journal of cultural economics, 30(4), 243-261. swank, e., & raiz, l. (2010). attitudes toward gays and lesbians among undergraduate social work students. affilia, 25(1), 19-29. şahin, f. (2011). liderin kültürel zekâsının astların örgütsel vatandaşlık davranışı ile i̇ş doyumu üzerine etkisi [effects of leader cultural intelligence on subordinates organizational citizenship behavior and job satisfaction], savunma bilimleri dergisi, 10, (2). 80-104. şahin, f., gürbüz, s., köksal, o. & ercan, ü. (2013). measuring cultural intelligence in the turkish context, international journal of selection and assessment, 21(2), 35-144. international electronic journal of elementary education, 9(3), 645-666, march, 2017 664 şahin, f. & gürbüz, s. (2012). kültürel zekâ ve öz-yeterliliğin görev performansı ve örgütsel vatandaşlık davranışı üzerinde etkisi: çokuluslu örgüt üzerinde bir uygulama [the effect of cultural intelligence and self-efficacy on task performance and organizational citizenship behavior: a study in a multinational organization]. i̇ş, güç endüstri i̇lişkileri ve i̇nsan kaynakları dergisi, 14(2), 23-140. şimşek, n. (2010). kadın öğretmenlerin yönetici olmalarını engelleyen önyargı ve diğer faktörlerin incelenmesi [examination of prejudice and other factors that prevent female teachers from becoming managers] (unpublished master dissertation). i̇stanbul: yeditepe university, institute of social sciences. takeuchi, r., tesluk, p. e., yun, s., & lepak, d. p. (2005). an integrative view of international experience. academy of management journal, 48(1), 85-100. tay, c., westman, m., & chia, a. (2008). antecedents and consequences of cultural intelligence among short-term business travelers. handbook of cultural intelligence: theory, measurement, and applications, 126-144. thomas, d. c., elron, e., stahl, g., ekelund, b. z., ravlin, e. c., cerdin, j. l., ... & maznevski, m. (2008). cultural intelligence: domain and assessment. international journal of cross cultural management, 8(2), 123-143. thomas, d. c. & inkson, k. c. (2009). cultural intelligence: living and working globally. new york: berrett-koehler publishers. triandis, h. c. (2006). cultural intelligence in organizations. group and organization management, 31(1), 20-26. tufan, i. (2008). prejudices against, and social responsibilities towards, the disabled. social behavior and personality: an international journal, 36(1), 67-76. tutkun, ö. f. & koç, m. (2008). mesleklere atfedilen kalıp yargılar [template judgments attributed to professions]. journal of faculty of educational sciences, 41(1). 255-273. uluğ, f. (1998). eğitim sisteminde değişime yapısal uyum sorunları [problems of structural change in education system]. kuram ve uygulamada eğitim yönetimi dergisi, 4(2), 153-166. van dyne, l. & ang, s. (2005). cultural intelligence: an essential capability for individuals in contemporary organizations. new york: global edge. van dyne, l., ang, s., ng, k. y., rockstuhl, t., tan, m. l., & koh, c. (2012). sub‐dimensions of the four factor model of cultural intelligence: expanding the conceptualization and measurement of cultural intelligence. social and personality psychology compass, 6(4), 295-313. vernon, p. e. (2014). intelligence and cultural environment (psychology revivals). london: routledge. wiklund, j., patzelt, h., & shepherd, d. a. (2009). building an integrative model of small business growth. small business economics, 32(4), 351-374. wilson, c. e. & stewart, a. c. (2009). developing ethically and culturally, intelligent leaders through international service experiences. in annual meeting of the academy of management, chicago, il, us. wood, e. d. & st. peters, h. y. (2014). short-term cross-cultural study tours: impact on cultural intelligence. the international journal of human resource management, 25(4), 558-570. yeşil, s. (2009). kültürel farklılıkların yönetimi ve alternatif bir strateji: kültürel zekâ [the management of cultural diversity and an alternatıve strategy: cultural intelligence]. kmu i̇i̇bf dergisi, vol.11, 16. 100-131. yeşil, s. (2010). 21. yüzyılın küresel örgütleri için kültürel zekâ [cultural intelligence for 21st century global organizations]. çukurova üniversitesi sosyal bilimler enstitüsü dergisi, 19(2). relations between prejudice, cultural intelligence and level of entrepreneurship / baltacı 665 yılmaz, g. ö. & kaya, g. d. (2016). turizm sektöründe kültürel zekânın önemi: konaklama i̇şletmelerinde bir araştırma [the importance of cultural intelligence in the tourism sector: a research in the hospitality industry]. turizm akademik dergisi, 2(2), 29-44. yukl, g. a. (2002). leadership in organizations. 5. ed. upper saddle river, nj: prentice hall. zick, a., wagner, u., van dick, r., & petzel, t. (2001). acculturation and prejudice in germany: majority and minority perspectives. journal of social issues, 57(3), 541-557. international electronic journal of elementary education, 9(3), 645-666, march, 2017 666 this page is intentinally left blank. www.iejee.com http://www.iejee.com/ international electronic journal of elementary education, 2016, 8(3), 507-524 the effect of formative testing and selfdirected learning on mathematics learning outcomes mohamad syarif sumantri  jakarta state university, indonesia retni satriani jakarta state university, indonesia received: 8 december 2015 / revised: 28 january 2016 / accepted: 7 february 2016 abstract the purpose of this research was to determine the effect of formative testing and self-directed learning on mathematics learning outcomes. the research was conducted at an elementary school in central jakarta during the 2014/2015 school year. seventy-two fourth-grade students who were selected using random sampling participated in this study. data were obtained through testing and were analyzed using a two-line analysis of variance (anova) according to the treatment design and level of self-directed learning. the results showed that (1) mathematics learning outcomes differ between students who are given formative essay tests and those who are given formative multiple choice tests; (2) there is an interaction effect between formative testing and self-directed learning on mathematics learning outcomes; (3) students with high levels of self-directed learning have better learning outcomes when given formative essay tests than when given formative multiple choice tests; and (4) students with low levels of self-directed learning show no difference in mathematics learning outcomes based on whether they are given formative essay tests or multiple choice tests. keywords: formative test, self-directed learning, mathematics outcomes. introduction the trends in international mathematics and science study mullis, martin, foy, & arora, 2011) conducted in 2011 indicated that indonesia’s students have only a basic knowledge of mathematics, which is not sufficient to solve routine questions and problems or to manipulate mathematic forms and apply the necessary logical strategies. indonesian students rank 38 out of 42 countries in mathematical achievement, with an average score of 386 based on the timss scales (which range between 300 and 700). other evidence is available from the results of the program for international students assessment (pisa), which reported in 2012 that indonesian students scored 375 for mathematical capability.  mohamad syarif sumantri, jakarta state university, indonesia department of elementary education post graduate studies program (pps), universitas negeri jakarta jalan rawamangun muka, jakarta, 13220. phone: +622147213040, email: dikdas.pps@unj.ac.id international electronic journal of elementary education vol.8, issue 3, 507-524, 2016 508 other student achievement indicators include the national examination, which is conducted every year and has resulted in scores below 7, which are unsatisfactory. the average mathematics scores on the national examination (scored on a scale of 0-10) for public elementary schools in jakarta are shown in table 1 below. table 1. the average national examination mathematics scores for public elementary schools in dki jakarta and kepulauan seribu for 2011/2012 and 2012/2013 region/city academic year 2011/2012 2012/2013 central jakarta north jakarta east jakarta west jakarta south jakarta kepulauan seribu 7.04 6.77 7.33 6.57 7.29 6.24 6.73 6.64 7.34 6.55 7.13 6.32 average 6.87 6.78 source: www.simdik.info/hasilun/index.aspx. the data above show that mathematics achievement is relatively low. students struggle with mathematics for several reasons, including the abstract nature of mathematical figures, the fact that most students view mathematics as a frightening subject, students’ difficulty in comprehending mathematics or lack of interest in studying mathematics, and students’ disabilities related to mathematics learning skills. the national council for accreditation of teacher education (2003) stated that mathematics teachers should “possess a deep understanding of how students learn mathematics and use the pedagogical knowledge specific to mathematics teaching and learning” (as cited in osborne, 2015, p. 23). osborne (2015) concluded that educators must embrace current best practices for teaching mathematics. to do so, educators must immerse themselves in current research, strive to incorporate the best methods of mathematics instruction, and become educational experts regarding the common core state standards for elementary school mathematics. davis (2015) suggested that in mathematics, a profound understanding of the subject matter taught is a necessary but far from sufficient precondition for providing insightful instruction. rahman and lee (2014) explain that the majority of teachers agree that communication in the teaching and learning process is important for enhancing students' understanding. effective communication, improved students’ understanding of the topics being taught, which allowed them to solve high-level questions correctly. according to lai, zhu, chen, and li (2015), mathematics is among the most objective, logical, and practical academic disciplines. however, in addition to cognitive skills, mathematical problem solving also involves affective factors. some research supports the conclusion that mathematics anxiety may impede mathematics performance by affecting cognitive process. mathematics is about problem-solving. mathematics talks about how to find the best answer of mathematical questions into mathematical statements. the accuracy of answer given by an individual depends on the depth of mathematical knowledge he/she decision (tella, 2008). büyükkarci (2014) stated that evidence indicates that high-quality formative assessment has a powerful impact on student learning. in general terms, formative assessment is aims to help students improve their own learning. in practice, formative assessment is a self-reflective process that aims to promote student achievement. formative testing and mathematics outcomes / sumantri & satriani 509 the formative test is a measurement tool for determining learning quality and encouraging student learning activity. when formative evaluations are conducted frequently during the learning process, student learning outcomes are improved. badger and thomas (1992) explain that assessment motivates students to study and includes at least two purposes: encouraging understanding and an increased frequency of studying and determining a method of subject matter learning that the student can understand. formative assessment (fa) is intended to help learners and teachers track students’ progress in an informal way and to take remedial action when learning difficulties emerge (srivastava, waghmare, & vagha, 2015). formative assessment motivates the student to be more focused and provides an opportunity to monitor different aspects of student learning (lucas & spencer, 2014). gronlund and linn (1990) explain that the purposes of assessing learning progress are to (1) establish learning output goals; (2) determine learning needs; (3) monitor learning development and difficulty; (4) assess learning outcomes; and (5) use the evaluation outcome to develop study and learning aids. these purposes mean that the development and implementation of learning aids also serve to influence learning quality and productivity. according to arifin (2009), schools generally use only one of many available assessment tools for evaluations: the written test. the written test can take two forms: a subjective test and an objective test. both tests have different strengths and weaknesses. essay (subjective) tests require students to organize, interpret, and connect the knowledge that they already have. they also require students to consider, recognize, and improve their existing knowledge while applying a high level of creativity. in contrast, multiple choice (objective) tests examine students’ knowledge about specific concepts and skills (wiersma & jurs, 1990). the use of formative assessment tools in the form of essay and multiple choice tests has been predicted to affect students’ self-directed learning. self-directed learning is an intentional psychological activity that students direct and control with the aim of acquiring knowledge and understanding about a specific subject. students’ level of motivation for self-directed learning has an effect on their learning outcomes, which are indicators that the student’s knowledge and understanding of a specific subject is improving. gronlund and linn (1990) noted that tests are systematic procedures for measuring behavior or for determining how an individual acts when compared with others or when certain assignments need to be completed. furthermore, morrow, mood, disch, and kang (2005) stated that a test is an instrument that is used to measure a specific skill. such instruments can include written, oral, physiological, and/or mechanical devices. these opinions are supported by hopkins (1986), who argue that a test is an instrument, tool or procedure that contains assignments that students should address and that provides results that can be used to measure certain aspects of students’ knowledge. cronbach (1994) and nitko (2001) stated that a test is a systematic procedure for monitoring and describing one or more student characteristics using a numerical scale or classification scheme and/or a numerical standard or category system. anastasi and urbina (1997) wrote that a test is an instrument with objective standardization, and its results can be used broadly (for example, to compare psychological circumstances or individual behaviors). regarding the function and purpose of tests, popham (1995) stated that they are useful for diagnosing students’ strengths and weaknesses, determining student development, deciding student rankings, and determining the efficacy of further international electronic journal of elementary education vol.8, issue 3, 507-524, 2016 510 learning. in fact, hopkins (1986) cites the final item on this list as the main purpose of testing in terms of quantifying student learning outcomes. regarding the function of tests, gronlund and linn (1990) stated that they can be classified into four categories: placement tests, formative tests, diagnostic tests and summative tests. a placement test is used to determine the student’s ability at the beginning of the learning process; a formative test is used to monitor learning output; a diagnostic test is used to diagnose learning difficulty; and a summative test is used to evaluate achievements. according to gronlund and linn (1990) and boston (2002), formative assessment is designed to provide feedback to students and teachers during the learning process. this feedback should improve students’ awareness of the differences between the purposes of instruction and the knowledge, understanding, and skills that they already have, and it should help them to achieve the purposes of learning. furthermore, gregory (2000) noted that formative tests aim to determine the degree to which the student is “formed” after participating in the learning process for a certain period of time. formative testing is usually conducted throughout the learning program, that is, at the end of a teaching unit. the subjective test is one of the best test types for determining scores that are affected by other’s opinions or assessments. essay tests require students to formulate their own answers; students cannot select a single correct answer but must answer in their own words. the responses to essay questions should be read one at a time, compared with the correct answer supplied by the test maker, and then scored according to previously provided guidelines. stiggins (2001) stated that the essay test is a type of written test that consists of questions and requires students to create their own answers. hopkins (1986) noted that essay tests can provide insight into several student abilities: (1) the capability to think, analyze, synthesize, and evaluate; (2) the maximum capability for developing thinking skills; (3) the capability to argue/express opinions; (4) the capability for written expression; and (5) the maximum capability for naturally organizing logical thinking. morrow et al. (2005) observed that the open-question nature of the essay test makes it very effective for measuring students’ abilities to organize, analyze, synthesize and assess compared with other tests. essay tests can also effectively measure participants’ arguments and attitudes. the essay test offers several advantages, such as (1) assessing the mental processes that students use to shape their ideas into correct answers, (2) measuring students’ ability to answer a question in their own words, (3) encouraging students to actively learn and to arrange, construct and explain their logical thinking, (4) encouraging students to be courageous when making an argument and to construct their arguments in their own words, and (5) understanding how deeply the student understands and has addressed a problem based on the knowledge taught in the class. in comparison, multiple choice tests are flexible and of high quality, which allows them to be used for many types of tests and examinations (gronlund & linn, 1990). hopkins (2005) explained that multiple choice tests consist of two parts: incomplete questions or statements and two or more possible answers plus a few other answer options as decoys. the decoys serve to distract the participant when he or she is uncertain of the correct answer. cangelosi (2007) said that to facilitate a participant’s ability to answer the questions, multiple choice questions should be composed according to certain guidelines, including (a) how the student is expected to select the option, (b) how many options may be selected for each question, (c) whether the answer options include one correct answer or the most formative testing and mathematics outcomes / sumantri & satriani 511 correct answer, and (d) other significant information (for example, whether predicting the answer is allowed). dodge (2009) explained that multiple choice tests are used in all areas that require written tests for several reasons: 1) the questions can be analyzed and scored efficiently, quickly and correctly; 2) participants are less hesitant to answer multiple choice tests compared with other test types; 3) questions with two or more possible answers are difficult to answer by guessing; 4) multiple choice tests can be used to measure higherlevel skills, such as application and analysis; and 5) they can be used to measure some objective knowledge. the multiple choice test has several benefits: 1) it can be used to assess several broad learning outcomes; 2) it does not allow the student to write or describe the answer, which minimizes guessing; 3) it prioritizes reading and thinking because the items make minimal demands on memory; 4) it allows students to deduce answers based on those provided; and 5) its use of decoy options can determine whether the student possesses the relevant perception and knowledge (nitko, 2001). the factors that affect mathematics learning outcomes in elementary schools are tied to directed learning. directed learning implies independence. students in directed learning environments do not depend on someone else for their learning; their efforts are selfinitiated, and they must take responsibility in their daily activities. directed learning is also known as selfor autonomous learning. seifert and hoffnung, who were quoted by eggen and kauchak (2007), defined autonomy as the ability to control and arrange one’s ideas and feelings, to freely act out of self-motivation, and to overcome a lack of selfconfidence and self-doubt. based on the research result, teachers with higher self-efficacy belief are different from those with lower self-efficacy belief. the difference can be seen from the level of efforts, persistence when handling different kinds of students, openness to new ideas and methods, believing in students’ achievements and success, and building warm relationship with students rather than with their parents (nurlu, 2015). nash (2014) stated that the idea of self-directed learning originated with dewey’s response to what he saw as the demands of a modern democratic society. his thinking influenced education at the turn of the twentieth century and continues to inform educational policy to this day. for dewey (2012), the ideal aim of education in a democracy is the creation of self-control that can guide one’s personal freedom to choose. self-directed learning is the effort to remove oneself from one’s parents and find oneself through self-identification, which is the directed development of a stable and independent individual (good & brophy, 2007). the directed attitude is not egoistic or selfish; rather, it is a willingness and ability to develop one’s own life. self-directed learning is marked by the individual’s ability to adapt his/her behavior, take responsibility, make his/her own decisions, show initiative and creativity, and be able to solve problems without intervention from others. self-directed learning can occur under a number of conditions, including when: 1) teachers act as facilitators rather than as sources of content; 2) learners are involved in selecting learning resources and strategies, and 3) learners are involved in self-assessment of their learning outcomes (knowles, 1975). petty (2009) described self-directed learning as a humanistic approach in which teachers serve as facilitators. the benefits of independence are that it encourages active learning and extends the responsibility for learning to the student. furthermore, scott (2006) and tavani and losh (2003) stated that independence/directed learning promotes a child’s ability to select correct options, determine his/her options, and be responsible for her/his determination. independent children have higher self-confidence and motivation, and their behaviors do not depend on anyone else. van merriënboer and sluijsmans international electronic journal of elementary education vol.8, issue 3, 507-524, 2016 512 (2009) concluded that instruction that provides a solid base for self-directed learning contains three elements: learners must (a) perform the tasks, (b) assess their task performance, and (c) select future tasks for improving their performance. independent learning is recognized as an important predictor of students’ academic motivation and achievement. it is a pivotal construct in contemporary accounts of effective academic learning. independent learning is essential to the learning process through which students direct their acquisition of academic knowledge (winne, 1995). schunk (2005) argued that independent learning is a state or condition in which students pursue learning activities on their own without relying on others; this condition or state is always consistent, and self-directed learners are eager to learn anywhere. independent learning is mostly driven by the student’s own initiative, choice and selflearning responsibility. zimmerman (1990) defined independent learning as a process of learning that results from the influence of thoughts, feelings, strategy, and self-behavior that are oriented toward achieving a goal. thomas (1993) suggested that independent learning is an unrestricted and responsible student behavior geared toward determining learning objectives, planning and implementing, and maintaining and assessing the results of learning activities without relying on others. zimmerman (2008) viewed independent learning as a proactive process rather than a reactive event that happens to students due to impersonal forces such as teaching. based on the above opinions, independent learning depends on the student’s attitude and ability to complete learning activities independently or alone and be responsible for achieving a goal. independent learning in mathematics results from a student’s attitude and ability to learn mathematics independently, master a competency and be responsible for completing the task with reduced guidance from others (cleary & chen, 2009). during the self-learning process, students require independence if the learning process able to progress properly and maximize student achievement. metallidou and vlachou (2010) said that self-learning is learning in which students are allowed to determine the learning goals; plan the learning process and the strategies using learning resources that he or she has chosen; make academic decisions; and perform activities that contribute to the achievement of learning goals. banarjee and kumar (2014) stated that independent learning has an effect on academic achievement, including in mathematics. this effect occurs because children begin to believe that they have the ability, discipline, and enthusiasm to pursue achievement; as a result, they do not feel inferior and are ready to solve any problems that arise, and this attitude increases achievement. according to banarjee and kumar (2014), independent learning and academic achievement are positively correlated, and there is a significant positive relationship between independent learning and academic achievement in science. in the context of mathematics classrooms, students are challenged to solve tasks, to conceptualize their own opinions and to adapt strategies to task demands (kramarski & revach, 2009). the research by lopez vargas, hederich-martinez, and camargo uribe (2012), ocak and yamaç (2013), camahalan (2006), metallidou and vlachou (2010) and çiftçi and koza (2015) indicates that to improve and maintain mathematics learning outcomes, it is necessary to consider factors related to the students’ independence with mathematics. when a student’s level of independence is high, it will affect the results of that student’s mathematics learning. the greater the student’s independence, the higher the mathematics learning outcomes; conversely, the lower the student’s independence, the lower his or her mathematics learning outcomes will be. formative testing and mathematics outcomes / sumantri & satriani 513 the research conducted by yang and li (2013) assessing the use of animated self-directed learning activity modules for children’s number sense development found that even without a teacher’s instruction in class, children can develop number sense through selfdirected learning. research conducted by munasco (2013) in banda aceh concluded that students’ physics learning outcomes are better when essay tests are used than when multiple choice tests are used, after controlling for initial capability. the formative testing research conducted by putri and indra (2010) with elementary students in palembang concluded that the students who were given essay tests showed higher mathematics learning outcomes than the students who were given multiple choice tests, after controlling for initial capability. method this research aimed to determine the effects of and interactions between formative testing and self-directed learning and their relationship with the mathematics learning outcomes of elementary students. the research was conducted in a public elementary school in central jakarta. the research subjects were students who were in their first semester of fourth grade. the research began as an experiment to examine the validity of an instrument (which took the form of essay tests and multiple choice tests) for assessing learning outcomes. the research method used was an experimental method with a 2 x 2 treatment-by-level design. the variables in this research were dependent variables (the formative tests), an independent variable (mathematics learning outcomes), and an attribute variable (selfdirected learning). the formative tests (a) included two forms: a formative essay test (a1) and a multiple choice test (a2). student self-directed learning (b) was classified as either high (b1) or low (b2). there were four groups tested: a group of essay tests and students with high self-directed learning (a1b1), a group of multiple choice tests and students with high self-directed learning (a2b1), a group of essay tests and students with low selfdirected learning (a1b2), and a group of multiple choice tests and students with low selfdirected learning (a2b2). table 2. design treatment by level self-directed learning (b) formative test (a) essay (a1) multiple choice (a2) high (b1) a1b1 a2b1 low (b2) a1b2 a2b2 the two-treatment method was designed to determine the effect of the treatments on mathematics learning outcomes. the examined treatments were formative tests and independent/self-directed learning. the formative test treatment involved giving the students formative essay tests and formative multiple choice tests. formative essay tests are tests that require students to create their own answers. multiple choice tests provide several possible answers and require the student to determine which is correct. the formative multiple choice test that was used in this study provided four answer options for each question. the students’ self-direction levels were assessed via a questionnaire. the research sample pool comprised all of the students in the fourth grade at a public elementary school in central jakarta. the sample was selected using a random sampling technique. two classes were used as sampling sources for this research. two classes were selected so that one class could take the formative essay test and the other could take the formative multiple choice test. grade iv a was assigned to take the formative essay test, international electronic journal of elementary education vol.8, issue 3, 507-524, 2016 514 and grade iv b was assigned to take the formative multiple choice test. the students in grade iv a and grade iv b had the same relative academic capabilities because neither group of students was categorized as having superior skills. the sample population comprised 72 students. to measure self-directed learning, the students were grouped according to whether they reported having high or low levels of directed learning. the sample included 40 students, 20 of whom took essay tests and 20 of whom took multiple choice tests. ten students in each group were chosen randomly for the self-directed learning assessment. the experiment was conducted using formative essay tests and multiple choice tests. the students who took the formative essay tests were tested 3 consecutive times at the end of each lesson, as were the students who took the formative multiple choice tests. furthermore, to avoid differences in test scores that may have been caused by the formative tests that were used for practice, the final test for each student group included a bundle of essay or multiple choice tests, as appropriate for the group. the data collection instruments were (1) a mathematics learning outcome instrument and (2) a studentdirected learning instrument. the student-directed learning data were measured using a non-test instrument that used frequency scales with five options. these data were collected before the treatment. the data were intended to group the respondents according to their self-directed learning levels (high and low). the mathematics learning outcome data were measured using a learning outcomes test, which consisted of essay and multiple choice tests. both tests were developed by the researcher and tested on students with characteristics similar to those of the research respondents. validity testing was conducted using tests of content validity and construct validity. construct validity was tested using expert judgment. content validity was tested by referring to the elementary school curriculum for fourth grade. the multiple choice questionnaire used a dichotomy formula based on biserial points. the tests were created using microsoft excel 2013 software (microsoft operations pte, ltd., singapore, reagents: sistech kharisma company). the validity of each questionnaire was determined by comparing the correlation coefficient (r value) with the critical biserial correlation number (r table) based on a significance level of 5%, as follows: 1) if r item> r table and α =0.05, then the item was considered valid; 2) if r item ≤ r table and α = 0.05, then the item was considered invalid. based on this calculation, 16 questions on the multiple choice test had a value of r phi > 0.355 and a significance level of α = 0.05. the validity test for the essay tests, based on the product moment formula, showed that the entire test (7 questions) had a value of r value > r table (0.355) and a significance level of α = 0.05. to determine the reliability of instrument, the hoyt formula was used. before the hypothesis was tested, we conducted an analysis requirement test, which consisted of a normality test and a homogeneity test. the normality test used the lilliefors formula. the data were found to have a normal distribution when l0 < l table with a significance level of 0.05. the variance homogeneity test of group y for x1 and x2 was intended to test the similarity of two population variants with normal distribution patterns. the homogeneity of the data were tested using bartlett’s test. the data were homogeneous if 2count <2table, with a significance level of 0.05. formative testing and mathematics outcomes / sumantri & satriani 515 results the hypothesis was tested using a two-line variance analysis to examine the interactions, and then a one-line analysis was used to test the hypothesis regarding the simple effect. the two-line analysis results were used to test the hypothesis regarding the main effect, which was that the average learning outcomes of the student groups that took the formative essay tests would differ from those of the student groups that took the formative multiple choice tests. furthermore, the hypothesis about the effect of the interaction between the formative test treatments and self-directed learning on mathematics learning outcomes was tested. the results for these hypotheses are summarized in table 3 below. table 3. two-line anova table for mathematics learning results with α= 0.05 and α= 0.01 variance source jk db rjk f value f table α = 0.05 α = 0.01 inter a 114.568 1 114.568 5.38* 4.08 7.31 inter b 589.114 1 589.114 27.66** 4.08 7.31 interaction a x b 134.750 1 134.750 6.32* 4.08 7.31 inside/within 852.000 40 21.300 total 1690.432 43 the results of the two-line anova above can be explained as follows: 1) the mathematics learning results of the students who were given the formative essay test treatment differed from those of the students who were given the formative multiple choice test treatment. based on the anova results in table 3 above, f value = 5.38, whereas ftable with α = 0.05 is 4.08. thus, f value > ftable, which indicates that h0 must be rejected because h1 is accepted. thus, there is a difference in the average mathematics learning outcomes of the students who received the formative essay test treatment and those who received the formative multiple choice test treatment. the average mathematics learning results for the students who took the formative tests was 71.60, whereas the students who took the formative multiple choice had an average mathematics learning result of 66.00. 2) the mathematics learning outcomes differed between the students with high self-directed learning and those with low self-directed learning. the anova results in table 3 above show that fvalue = 27.66, whereas the ftable at α=0.01 is 7.31. thus, fvalue > f table, which shows that h0 is rejected because h1 is accepted. therefore, there is a difference in the mathematics learning outcome averages between the group of students with high self-directed learning and the group with low self-directed learning. according to the mathematics learning test results, students with a high level of independence had an average formative essay test score of 83.4, while the average score on the formative multiple choice test was 70.7. 3) the formative test type and self-directed learning have an interaction effect on mathematics learning outcomes. the anova results above show that the fvalue had an interaction effect ab=6.32, whereas ftable at α= .05 was 4.08. thus, fvalue > ftable, indicating that h0 was rejected and h1 was accepted. it may be concluded that there is an interaction effect between the formative test type and self-directed learning. the effect of the interaction between the type of formative test and self-directed learning on mathematics learning outcomes can be seen in figure 1. international electronic journal of elementary education vol.8, issue 3, 507-524, 2016 516 the average f between the four groups of cells was 39.78, while the ftable at the level of α=0.05 is 2.84. thus, fcount > ftable, meaning that h0 is rejected and h1 is accepted, indicating that on average, there were differences among the four groups of cells. figure 1. the interaction between formative test type and self-directed learning. the intersection of the two lines shows that there is an interaction between the two variables toward the independent variable. in other words, the interaction means that the effect of the formative test type on the mathematics learning outcomes depends on the student’s level of self-directed learning. the intersection of the two lines also shows that there is an interaction between the two independent variables. in other words, the interaction means that the effect of the formative test type on mathematics learning outcomes depends on the student’s level of self-directed learning. as a consequence of the interaction, the two-line anova must be continued with a dunnett’s t-test. the test was performed to examine the simple effect, that is, to examine the difference in mathematics learning outcomes between the students with high selfdirection who received the formative essay test and those who received the formative multiple choice test and the difference in mathematics learning outcomes between the students with low self-direction who received the formative essay tests and those who received the formative multiple choice tests. 4) the mathematics learning outcomes differed between the students with high levels of self-directed learning who received the formative essay tests and those who received the formative multiple choice tests. the results of dunnett’s t-test showed that the average mathematics learning outcome scores differed for the students with high levels of self-directed learning who received the formative essays and those who received the formative multiple choice tests. the tvalue was 3.42, whereas the value for ttable at level α= 0.01 was 2.43; thus, tvalue > t table. as a result, h0 is accepted, and h1 is rejected, indicating that the mathematics learning outcomes differed between the highly self-directed students who received the formative essay tests and those who received the formative multiple choice tests. the data obtained for the mathematics learning outcomes of the students with high levels of self-directed learning showed that the average scores for those who received the formative testing and mathematics outcomes / sumantri & satriani 517 formative essay tests were approximately 41.9, whereas the average scores of those who received the formative multiple choice tests were approximately 35.18, indicating that the average scores of the students who were given formative essay tests were higher than those of the students who were given formative multiple choice tests. consequently, the learning output of the students with high levels of self-directed learning who were given formative essay tests was higher than that of the highly self-directed students who were given formative multiple choice tests. therefore, it may be said that giving formative essay tests to these students could increase their learning results. 5) the mathematics learning results differed between the students with low levels of selfdirected learning who received the formative essay tests and those who received the formative multiple choice tests. the dunnett’s t-test results, which aimed to differentiate the average mathematics learning results of the students with low levels of self-directed learning who were given formative essay tests from the learning results of those who were given the formative multiple choice tests, showed that the value was roughly 0.138, whereas the ttable score at α=0.05 is -1.68. therefore, t value > t table, and h0 was accepted or h1 was rejected. this result indicates that there was a difference in the mathematics learning outcomes of the students with low levels of self-directed learning who were given the formative essay tests and those who were given the formative multiple choice test. the mathematics learning outcome data of the students with low self-directed learning showed that the average score of the students who received the formative essay tests was approximately 31.09, whereas the average score of those who received the formative multiple choice tests was 31.36. these findings indicate that the learning results of those who were given the formative essay tests were lower than the results of those who were given the formative multiple choice tests. the findings also show that the learning result for students with low levels of self-directed learning were lower when they were given formative essay tests than when they were given the formative multiple choice tests. thus, for students with low levels of self-directed learning, formative multiple choice tests could improve their learning results. discussion the results of our research and statistical analyses can be explained by noting that the variation in the mathematics learning outcomes of the students who were given the formative essay tests and those who were given the formative multiple choice tests. the mathematics learning outcome results showed that the students who took the formative essay tests scored higher than the students who took the formative multiple choice tests. this finding indicates that the type of formative test has an overall effect, regardless of whether the students have high or low levels of self-directed learning. this evidence is strengthened by similar research conducted by munasco (2013), in which students who were given essay tests scored higher than students who were given multiple choice tests. the research hypothesis, which stated that mathematics learning outcomes would be higher for students who were given formative essay tests than for those who were given formative multiple choice tests, was empirically examined via data analysis. the findings are explained by comparing the characteristics of both types of formative tests. as the measured learning outcomes show, the formative essay test is more effective for measuring high levels of cognitive ability, but it lacks the ability to effectively measure low levels of cognitive ability. in contrast, the formative multiple choice test is more effective for measuring low and moderate levels of cognitive ability but lacks effectiveness for measuring high levels of cognitive ability. regarding the scope of the material, the essay test has a small scope and a small number of questions compared to the multiple choice international electronic journal of elementary education vol.8, issue 3, 507-524, 2016 518 test. the formative multiple choice test covers a large scope of material and can represent an entire subject. however, as the factors that affected the test results show, students’ scores on formative essay tests are affected by their ability to analyze, evaluate and argue their ideas on paper. theoretically, a formative test only needs to utilize one type of answer structure to allow students to describe, explain, compare, and reason by answering questions in their own words and by using their analytical thinking and interpretation skills to address the questions. the essay test gives students the freedom to express their ideas, which could encourage them to express an argument/opinion through their analytical writing and to express their answer directly in a way that could increase the quality of the answer and the score. this means that all students who are given essay tests can use their imagination and explore their ideas to the best of their ability. through formative essay tests, students are motivated to develop their higher-order thinking and express it creatively. formative multiple choice tests provide different advantages. they are good for measuring students’ abilities because they do not allow students to use their imagination. multiple choice test results are affected by the student’s ability to understand the concept, which can be viewed as a learning result. a formative multiple choice test can encourage students to remember, interpret and recall other people’s ideas. consequently, it could be said that the students who were given formative essay tests showed higher mathematics learning results than the students who were given multiple choice formative tests. these facts support the opinion that the average mathematics learning outcome scores on formative essay tests are higher than those on formative multiple choice tests; consequently, these findings are evidence that the formative essay test treatment is more effective than the formative multiple choice test treatment for increasing student mathematics learning results. this finding is in line with the previously discussed characteristics of each formative test type. consequently, the stated hypothesis is scientifically proven: formative essay tests improve student learning results more than formative multiple choice tests. the second research hypothesis stated that there is an interaction effect between formative essay tests, formative multiple choice tests, and student self-directed learning on mathematics learning results, which shows the connectivity between the variables. the empirically tested data for the formative essay and multiple choice tests showed that the tests have different levels of effectiveness for measuring student learning results, depending on the student’s level of self-directed learning. the data are strengthened by the research by gavriel (2013), which stated that the effectiveness of formative assessment depends on the level of student self-directed learning. according to oyediji and okwilagwe (2015), self-directed learning is a process through which individuals consciously take responsibility and initiative, with or without the help of others, to determine their learning needs, formulate learning goals, identify resources for learning, select and implement learning strategies and evaluate learning outcomes within a given framework, thereby becoming their own learning agents (costa & kallick, 2004). the average mathematics learning results, according to the level of student selfdirected learning, were higher for the students with high levels of self-directedness who were given formative essay tests than for those who were given formative multiple choice tests. this finding shows that students who have high levels of self-directed learning can respond more effectively to formative essay tests than to multiple choice tests and that the use of essay tests could increase their learning results. however, the average score of students who had low levels of self-directed learning and received formative essay tests was lower than that of the students who were given multiple choice tests. this finding formative testing and mathematics outcomes / sumantri & satriani 519 indicates that students with low levels of self-directed learning work better with multiple choice tests and that the use of such tests could improve their learning results. the third hypothesis is that the results for students with high levels of self-directed learning who were given formative essay tests would differ from the results for those with high levels of self-directed learning who were given formative multiple choice tests. the analytical results showed a difference in learning results between the students who were given formative essay tests and the students who were given formative multiple choice tests among the groups with high levels of self-directed learning. that is, the students with high levels of self-directed learning perform better on formative essay tests than on formative multiple choice tests. this result agrees with the opinion stated by hapsari (2013), who said that students with high performance motivation show higher mathematics learning results on formative essay tests than on formative multiple choice tests. if self-directed learning is connected to formative essay tests, which are used for formative evaluation, then the characteristics of high self-directed learning are appropriately matched with the characteristics of essay tests; that is, the opportunity to apply individualized problem-solving skills, implement formulas and analyses, and take responsibility for one’s assignments. those characteristics indicate that the formative essay test requires critical thinking to find the answer. thus, the formative essay test’s characteristics are appropriately matched with a high level of self-directed learning, which reflects an ability to complete activities without depending on other people and having good self-confidence and initiative. the fourth hypothesis stated that students with low levels of self-directed learning who were given formative essay tests would have lower mathematics learning outcomes compared with the students with low levels of self-directed learning who were given multiple choice tests. the data showed that the students with low levels of self-directed learning had lower scores on the formative essay tests. this occurred because these students did not have the capability to solve the problem, the self-confidence to answer it correctly, or the analytical thinking skills to arrive at an answer. this type of student tends to lack initiative in the learning process, is more likely to have a fixed approach to the subject, depends on teachers’ explanations, is apprehensive about unclear ideas, and lacks the encouragement or effort to find the correct answer. students who have low selfdirected learning find it more difficult to solve problems on formative essay tests than on formative multiple choice tests. this is because students with low self-directed learning find it difficult to express their ideas, implement formulas and make decisions, which are required characteristics of the formative essay test. embo, driessen, valcke and van der vleuten (2010) suggest that the integration of feedback and assessment through a clearly defined learning and assessment instrument is a potentially valuable method for promoting self-directed learning and formative assessment. conclusions based on the data analysis and statistical calculations, this research draws the following conclusions. first, students who were given formative essay tests and those who were given formative multiple choice tests had different mathematics learning outcomes. the students who received formative essay tests scored higher than those who received formative multiple choice tests. second, there was an interaction effect between formative tests and self-directed learning on mathematics learning outcomes, which means that the effect of the interaction between formative tests and self-directed learning on student mathematics results in elementary school depends on the level (high or low) of self-directed learning. third, students who had high levels of self-directed learning and were given formative essay tests had higher mathematics learning outcomes compared with those who were given formative multiple choice tests. finally, among the students who had low levels of self international electronic journal of elementary education vol.8, issue 3, 507-524, 2016 520 directed learning, there was no difference in mathematics learning results between those who were given formative essay tests and those who were given formative multiple choice tests. implications based on the research findings, the direct implications for mathematics learning performance are as follows: 1) our research findings indicate that the type of formative test used has an effect on the learning results. thus, to improve students’ learning results, teachers should use different types of formative tests in the classroom, and students should be able to complete the assignments and obtain optimal scores. 2) the research findings indicate that students’ self-directed learning has an effect on the learning results. this finding implies that during the learning process, a student with a high degree of self-direction will actively participate in every learning activity in the classroom, with optimal learning results. 3) given the appropriate materials and methods, allowing self-direction in the classroom will develop students who are active, creative, and critical and will increase students’ learning motivation, which will significantly affect their mathematics learning results. • • • acknowledgements. the authors would like to thank jody c; shannon t; jodi h; and stacie m for their helpful comments and useful suggestions regarding an earlier draft of this manuscript. references anastasi, a., & urbina, s. (1997). psychological testing (7th ed.). upper saddle river, nj: prentice hall inc. arifin, z. (2009). evaluasi pembelajaran [learning evaluation]. bandung, indonesia: pt remaja rosdakarya (rosda). badger, e., & thomas, b. (1992). open-ended questions in reading. practical assessment. research & evaluation, 3(4), 1. banarjee, p., & kumar, k. (2014). a study on self regulated learning and academic achievement among the science graduate student. international journal of multidisciplinary approach & studies, 1(6), 329-342. boston, c. (2002).the concept of formative assessment. practical assessment. research & evaluation, 8(9), 1. büyükkarci, k. (2014). assessment beliefs and practices of language teachers in primary education. international journal of instruction, 7(1), 108. camahalan, f. m. g. (2006). effects of self regulated learning on mathematics achievement of selected southeast asian children. journal of instructional psychology. 33(3), 194. cangelosi, j. s. (2007). classroom management strategies: gaining and maintaining students’ cooperation (6th ed.). hoboken, nj: wiley. formative testing and mathematics outcomes / sumantri & satriani 521 çiftçi, ş., & koza. (2015). effects of secondary school students' perceptions of mathematics education quality on mathematics anxiety and achievement. educational sciences: theory & practice, 15(6), 1487. cleary, t. j.,& chen, p. p. (2009).self-regulation, motivation, and math achievement in middle school: variations across grade level and math context. journal of school psychology, 47(5), 291-314. doi:10.1016/j.jsp.2009.04.002 costa, a. l., & kallick, b. (2004). launching self-directed learners. educational leadership, 62(1), 5152. cronbach, l. j. (1994). essentials of psycological testing. new york, ny: harper & row. davis, b. (2015). the mathematics that secondary teachers (need to) know. revista española de pedagogía, 73(261), 321-342. dewey, j. (2012). democracy and education. lavergne, tn: simon & brown. dodge, j. (2009). 25 quick formative assessments for a differentiated classroom. sydney, australia: scholastic teaching resources. eggen, p., & kauchak, d. (2007). educational psychology: window on classrooms. upper saddle river, nj: pearson prentice hall. embo, m. p., driessen, e. w., valcke, m. & van der vleuten, c. p. (2010). assessment and feedback to facilitate self-directed learning in clinical practice of midwifery students. medical teacher, 32(7), e263-e269. gavriel, j. (2013). assessment for learning: a wider (classroom-researched) perspective is important for formative assessment and self-directed learning in general practice. education for primary care, 24(2), 93-96. good,t. l., & brophy, j. e. (2007). looking in classrooms (10th ed.). needham heights, ma: allyn & bacon. gregory, r. j. (2000). psychological testing: history, principles and applications (3rd ed.). boston, ma: allyn and bacon. gronlund, n. e., & linn, d. r. l. (1990). measurement and evaluation in teaching (6th ed.). new york, ny: macmillan publishing company. hapsari, s. a. (2013). kontribusi kemandirian terhadap hasil belajar matematika ditinjau dari fasilitas belajar dan jarak tempat tinggal siswa [the contribution of independence of the results of learn math in terms of facilities learning and the distance between their houses students]. retrieved from http://eprints.ums.ac.id/23458/12/naskah_publikasi.pdf hopkins, david (1986). evaluation for school development. united kingdom: open university press publishers. hopkins, k. (2005). educational and psychological measurement and evaluation (7th ed.). englewood cliffs, nj: prentice hall. knowles, m. s. (1975). self-directed learning: a guide for learners and teachers. new york, ny: association press. kramarski, b., & revach, t. (2009). the challenge of self-regulated learning in mathematics teachers' professional training. educational studies in mathematics, 72(3), 379-399. doi:10.1007/s10649-009-9204-2 lópez vargas, o., hederich-martinez, c., & camargo uribe, á. (2012). mathematics achievement, self regulated learning and cognitive style. suma psicológica, 19(2), 39-50. lai, y., zhu, x., chen, y., & li, y. (2015). effects of mathematics anxiety and mathematical metacognition on word problem solving in children with and without mathematical learning difficulties. plos one, 10(6), e0130570. doi:10.1371/journal.pone.0130570 http://eprints.ums.ac.id/23458/12/naskah_publikasi.pdf international electronic journal of elementary education vol.8, issue 3, 507-524, 2016 522 metallidou, p., & vlachou, a. (2010). children’s self regulated learning profile in language and mathematics: the role of task value beliefs. psychology in the schools, 47(8), 780-788. morrow jr., j. r., mood, d., disch, j., & kang, m. (2005). measurement and evaluation in human performance.champaign, il: human kinetics publishing house. mullis, i. v. s., martin, m. o., foy, p., & arora, a. (2011). international student achievement in the timss mathematics content and cognitive domains. in timss 2011 international results in mathetmatics (pp. 139-171). retrieved from http://timssandpirls.bc.edu/timss2011/downloads/t11_ir_m_chapter3.pdf munasco, m. (2013).the formative test form, teachers’ qualification, and students’ achievement in physics by controlling the students’ prior knowledge. jurnal evaluasi pendidikan, 4(1), 37-51. retrieved from http://www.library.gunadarma.ac.id/journal/view/11011 nash, c. (2014). founders’ continuing roles in schools supporting self-directed learning. interchange, 45(1/2), 43-57. nitko, a. j. (2001). educational assessment of student (3rd ed.). upper saddle river, nl: merrill. nurlu, ö, (2015). investigation of teachers’ mathematics teaching self-efficacy. international electronic journal of elementary education, 8(1), 24. ocak, g., & yamaç, a. (2013). examination of the relationships between fifth graders’ self-regulated learning strategies, motivational beliefs, attitudes, and achievement. educational sciences: theory & practice. 13(1),380-387. osborne, j. h. (2015). the charge is ours: teaching elementary school mathematics through best practice and common core initiatives. delta kappa gamma bulletin, 82(1), 23-25. oyediji, o., & okwilagwe, e. (2015). investigating the effects of self-directed learning and collaborative methods on junior secondary school students social studies learning outcomes in oyo state, nigeria. problems of education in the 21st century, 64, 38-52. petty, g. (2009). teaching today: a practical guide (4th ed.). cheltenham, england: nelson thornes. popham, w. j. (1995). classroom assessment: what teachers need to know. needham heights, ma: allyn and bacon. putri, r., & indra, i. (2010). pengaruh pendekatan pembelajaran dan bentuk tes formatif terhadap hasil belajar matematika dengan mengontrol ıntelegensi siswa sd di palembang [the effect of learning approach dan formative test to mathematic learning outcome: study in palembang public elementary student] (postgraduate dissertation, jakarta state university, indonesia). rahman, n. a. a. a., & lee, m. f. n. (2014). communication in teaching and learning mathematics: teachers’ perspective. aip conference proceedings, 1605, 730-733. doi:10.1063/1.4887680 schunk, d. h. (2005). self-regulated learning: the educational legacy of paul r. pintrich. educational psychologist, 40(2), 85–94. doi:10.1207/s15326985ep4002_3. scott, k. w. (2006). self-directed learners’ concept of self as learner: congruous autonomy. international journal of self-directed learning, 3(2), 1-13. srivastava, t., waghmare, l. s., & vagha, s. j. (2015). revisiting feedback practices in formative assessment of indian medical schools. national journal of physiology, pharmacy & pharmacology, 5(1), 1-4. doi:10.5455/njppp.2014.4.270620141 stiggins, r. (2001). student-involved classroom assessment (3rd. ed). upper saddle river, nj: merrill prentice-hall. tavani, c. m., & losh, s. c. (2003). motivation, self confident and expectations as predictors of the academic performances among our high school students. child study journal, 33(3), 141-152. thomas, j. w. (1993). promoting independent learning in the middle grades: the role of instructional support practices. elementary school journal, 93(5) 575-591. doi:10.1086/461741 http://timssandpirls.bc.edu/timss2011/downloads/t11_ir_m_chapter3.pdf http://www.library.gunadarma.ac.id/journal/view/11011 formative testing and mathematics outcomes / sumantri & satriani 523 tella, a, (2008). teacher variables as predictors of academic achievement of primary school pupils mathematics, international electronic journal of elementary education, 1(1) 17. van merriënboer, j. j. g., & sluijsmans, d. m. a. (2009). toward a synthesis of cognitive load theory, four-component instructional design, and self-directed learning. educational psychology review, 21(1), 55–66. doi:10.1007/s10648-008-9092-5 wiersma, w., & jurs, s. g. (1990). educational measurement and testing (2nd ed.). retrieved from http://www.clr.ui.ac.id/wp-content/uploads/2010/10/educational-measurement-andtesting.pdf winne, p. h. (1995). inherent details in self -regulated learning. journal of educational psychology, 30(4), 173–187. doi:10.1207/s15326985ep3004_2 yang, d.-c., & li, m. n. (2013). assessment of animated self-directed learning activities modules for children’s number sense development. educational technology & society, 16(3), 44-58. zimmerman, b. j. (1990). a study on self-regulated learning and academic achievement: an overview. journal of educational psychologist, 25(1), 13. zimmerman, b. j. (2008). investigating self -regulation and motivation: historical background, methodological developments, and future prospects. american educational research journal, 45(1), 166-183. doi:10.3102/0002831207312909 http://www.clr.ui.ac.id/wp-content/uploads/2010/10/educational-measurement-and-testing.pdf http://www.clr.ui.ac.id/wp-content/uploads/2010/10/educational-measurement-and-testing.pdf international electronic journal of elementary education vol.8, issue 3, 507-524, 2016 524 www.iejee.com this page is intentionally left blank http://www.iejee.com/ international electronic journal of elementary education, 2015, 8(1), 69-82 issn:1307-9298 copyright © iejee www.iejee.com instructional materials commonly employed by foreign language teachers at elementary schools i̇smail çakir  erciyes university, kayseri, turkey received: february, 2015 / revised: august, 2015 / accepted: august, 2015 abstract this study aimed to determine the teachers’ choices of instructional materials in teaching english at elementary schools. the reasons behind preferring or not preferring some certain instructional materials specified within the research were analysed. to this end, during the course of school experience, 68 prospective english teachers observed 38 teachers of english working at 14 elementary schools on a weekly basis, and they completed a questionnaire. a semi-structured interview was also conducted with five randomly selected teachers to identify their reasons for choosing certain instructional materials. the descriptive results revealed that most of the teachers were reluctant to use many of the highly beneficial materials due to reasons including overcrowded classes, limited technological knowledge, lack of time for preparation, curricular time constraints, heavy work load, burnout etc. the study suggests that apart from course-books teachers should be encouraged to use other instructional materials to motivate learners and offer an interactive foreign language teaching atmosphere. keywords: instructional materials, elementary schools, teachers, language teaching, technology. introduction changes in learning styles of students and types of instructional materials available have put a great amount of pressure on the teachers, who inevitably need to keep up with the innovative techniques in technology and teaching methods. it is a fact that teaching, which is admittedly a long and hard process, is primarily composed of five components: students, teachers, instructional materials, teaching methods, and evaluation (kitao & kitao, 1997). of these components, instructional materials in foreign language teaching can refer to a variety of things. they can be defined as any tool that teachers use to assist their students in adequately learning the target language; means used to increase students’ access to that language; every instrument that contributes greatly to students’ progress; anything which  this article was partly presented at the 3rd international conference on foreign language teaching and applied linguistics, bosnia herzegovina, may 03-05, 2013.  i̇smail çakır, erciyes university faculty of education, department of english language teaching, kayseri, turkey. phone (+90)352 2076666ext.37365. e-mail: ismailcakir@erciyes.edu.tr http://www.iejee.com/ international electronic journal of elementary education vol.8, issue 1, 69-82, 2015 70 is used by teachers and learners to facilitate the learning; and the keys to have influence on what goes on in the classroom, just to list a few (brown, 1995; crawford, 2002; jones, 2009; littlejohn, 2012; mcdonough, shaw & mashura, 2013; richards, 2010;tomlinson, 2008). as regards the use of instructional materials to foster foreign language learning, foreign language teachers tend to employ them at the right time and in the right proportion. offering a myriad of benefits to both teachers and learners in teaching and learning english as a foreign language (efl) context, a variety of instructional materials need to be included in the agendas of teachers. research proves that instructional materials highly facilitate learning and greatly draw learners’ attention to the target language (littlejohn, 2012; mcdonough, shaw & mashura, 2013; solak & çakır, 2015; tomlinson, 2012). simply put, they have a considerable influence on foreign language learners and they play an extremely influential role in the efl classes. to support this view, richards (2001) asserts that instructional materials generally serve as the basis of much of the language input that learners receive and the language practice that occurs in the classroom. thus, it is suggested that instructional materials need to be motivating and interesting. according to chomsky (1988), 99% of teaching is making the students feel interested in the material. to make the materials more interesting for foreign language learners, teachers should seek to find and present alternative techniques. in this sense, the present study with its findings and suggestions aims to contribute to the professional development of the teachers and preservice teachers so that they can be more effective and well equipped in teaching the target language. significance of the study this study intends to shed light on the issue of professional and qualified language teachers’ need to be able to use instructional materials properly. investigating this issue is important for teachers currently working at schools and student teachers who will take active roles in the classes in coming years. additionally, it can be asserted that this study is significant in that there is not much research exploring the instructional material preferences of english language teachers at elementary schools in turkey. review of literature the essence of the current century brings about an overwhelming amount of information which involves using efficient mechanisms to ameliorate learning and teaching activities (kuzu, akbulut & şahin, 2007). in this respect, as mentioned above, it is worth stressing that instructional materials play a crucial role in any efl setting. before explaining its role in language teaching settings, it would be wise to clarify the types of materials that are commonly utilized by teachers and learners. tok (2010) categorizes these instructional materials into two groups: printed ones such as course-books, workbooks, teacher’s guides etc.; and non-printed ones such as computer-based materials, videos etc. correspondingly, tomlinson (2012) classifies the language teaching materials in terms of instructional purposes as follows: informative (informing the learning about the target language); instructional (guiding the learner in practicing the language); experiential (providing the learner with experience of the language in use); eliciting (encouraging the learner to use the language); and exploratory (helping the learner to make discoveries about the language). in other words, instructional materials in an efl context have an indispensable role as they facilitate language learning (tomlinson, 2008), draw learners’ attention, motivate learners towards foreign language learning, provide comprehensible input (krashen, 1985) and authentic language (lee,1995), make learning more concrete and more meaningful, and guide the learner in practicing the language. instructional materials commonly employed / çakır 71 from a general perspective, instructional materials can be categorized in three groups: (1) traditional materials; (2) audio and visual materials; and (3) information communication technologies (ict) based materials. traditional materials that can be regarded as classic or old usually include board, course-book, worksheet, chart, realia, flashcard, teacher made materials, etc. as for audio and visual materials, which are regarded as new for many teachers, are video player, audio materials, video camera, computer, projection device, and so on. the third category for the instructional materials, widely acknowledged as the newest, is ict based materials such as interactive white boards (iwb), materials providing mobile assisted language learning (mall) such as web 2.0, tablet computers, the internet, podcasts, smartphone, web 3.0 etc. of all the materials listed above, particularly, course-books are the only materials that most of the foreign language teachers use in the efl classrooms, and they are automatically expected to include all aspects of language (çakır, 2010). course-books play a pivotal role in language classrooms in all types of educational institutions – state schools, colleges, language schools – all over the world (rahimpour & hashemi, 2011). sheldon (1988) regards the course-book as the visible heart of any elt program and she puts forth a number of reasons to justify the widespread use of course-books in the teaching of english as a foreign or second language throughout the world. they are the universal elements of language teaching (hutchinson & torres, 1994) for many teachers as they have ready-made materials and syllabi designed by the specialist in the field. in cunningsworths’ (1995) words, “course-books provide a resource, a source of activities, a reference, a syllabus, a great support for inexperienced teachers, and serve as a source for self-directed learning or self-access work” (p.7). the other issue that should be focused on regarding the use of course-books in english language teaching (elt) is its role in raising learner’s motivation. in attempt to figure out whether there is a positive relationship between the course-book and language learning, rahimi and hassani (2012) conducted a study on iranian foreign language learners. the result revealed that there was a positive relationship between iranian students’ positive attitudes towards efl course-books and their attitudes towards learning english as a foreign language. on the other hand, richards and renandya (2002) claim that many of the course-books as preplanned teaching materials have some possible following disadvantages (as cited in kayapınar, 2009): (a) they fail to present appropriate and realistic language models; (b) they propose subordinate learner roles; (c) they fail to contextualize language activities; (d) they foster inadequate cultural understanding; (e) they fail to teach idioms. in consistent with this claim, tsiplakides (2011) believes that among the main negative effects of the use of course-books is that they may contain inauthentic language, may distort content, may not satisfy students’ needs, and may be expensive to buy. educationally speaking, course-books are not a magical tool, and they have both advantages and disadvantages for efl learners and teachers. it is widely accepted that presenting the target language only through ready-made printed materials does not always contribute to comprehensive and meaningful understanding. thus, apart from the course-books or textual materials, teaching should also be supported by other visual materials such as pictures, flashcards, posters, tables, charts, etc. in this context, mayer (2009) claims that visual materials play an important role in assisting instruction in order to clarify, define and explain the related teaching point. therefore, it is safe to say that in order to create a meaningful learning atmosphere and to offer a comprehensible input, word and pictures need to be presented simultaneously. in this way, learners have a chance to construct both verbal and pictorial mental schemas and build connections between them (as cited in kuzu, akbulut & şahin, 2007). furthermore, words and visual aids brighten up the classroom and bring more international electronic journal of elementary education vol.8, issue 1, 69-82, 2015 72 variety and interest into language lessons, as well as help to provide the contexts which illuminates the meaning of utterances, and helps in giving information of one kind or another about life in the foreign country concerned (lee & coppen, 1971). the other contribution of visual materials to language learning is that they assist teachers by giving them time for necessary classroom activities like drills, exercises, instructions and explanation (abebe & davidson, 2012). it is true that with the rapid advancements in technology, new facilities and devices such as ict based materials have been brought into the efl classroom. new trends in teaching are increasingly technologically interactive, which precedes the instructional materials at the service of learners and teachers. in this digital age, particularly foreign language teachers must cope with new challenges, responsibilities, and duties. as shamylee and phill (2012) agree, the tradition of english teaching has been drastically changed with the entry of technologies. in this respect, language teachers cannot disassociate themselves from new technology, which allows for the application of scientific knowledge to practical tasks (çakır, 2006). in other words, technology satisfies both auditory and visual senses of the students in learning environments (shyamlee & phil, 2012). it is undoubtedly true that audio-visual materials offer paramount effects on understanding of both written and spoken language. some of the basic audio-visual aids that are frequently used in foreign language teaching include audio books, mp3, sound files, podcasts, the internet, cds, songs, videos, etc. ali, ghani and ali (2010) maintain that the history of the audio-visual aids can be traced back to the greek period, however, the technological advancements have brought new innovations in the form of computer assisted programs and accessories. that is to say, technology is seen that it is not a new issue concerning only modern teachers. this long history also explains the effectiveness of audio-visual materials (rao & jyoti, 2012). foreign language teaching, in particular english language instruction has been conducted with primarily the most available instructional materials, such as course-books, flashcards, audio recordings, video etc., despite their countless benefits, instructional materials have not been utilized properly to assist language learning. in many cases, unfortunately, it can be observed that a great amount of teachers resist using innovative materials due to limited knowledge and time. however, it is widely accepted that a single extracurricular material or activity brings a wide variety into the classroom. in this regard, it would be wise to cite the famous chinese proverb “a picture is worth a thousand words”, which clearly and simply explains the function of a single material to clarify the meaning of something better. therefore, it is essential for foreign language teachers to utilize a variety of instructional materials so that language teaching can turn out to be fun and motivating for learners. the study reported here aimed to identify the types of the instructional materials that were employed or overlooked by the foreign language teachers. to this end, the following research questions were formulated to guide the present study: 1. what kinds of instructional materials do foreign language teachers use at elementary schools? 2. what are the possible reasons for utilizing certain instructional materials in the efl classrooms? method participants participants included 38 teachers of english working at elementary schools in kayseri, turkey and they were informed about the observations to be conducted. the age range of instructional materials commonly employed / çakır 73 the participants was 24-48 with an average age of 30.97 and standard deviation is 4.783. there were 25 male and 13 female participants with the average working experience of 7.73 years. they were observed by 68 students majoring in english language teaching at a state university. the student teachers, who were selected randomly on a voluntary basis, were assigned to take notes of instructional materials specified in the questionnaire that the participants used to promote teaching english. the student teachers were all fourthyear students and had a compulsory training program in which they were required to complete the related procedure of school observation. that is, they attended the related schools and observed the participants in fulfilment of the requirements of their course of school observation course. apart from this course, the student teachers also took the course of materials development and evaluation in foreign language teaching, in which all kinds of instructional materials were introduced and evaluated. research instrument in order to gather data about the types of instructional materials used to teach english, a questionnaire was designed by the researcher. in the development phase of the survey, the instrument was forwarded to four faculty members and an outside expert in the same field. upon receiving the positive feedback, the survey was first piloted with ten prospective elt teachers in order to determine whether or not the content of the questions were valid. the comments taken during this process were considered in redesigning some of the items in the questionnaire. the questionnaire consisted of likerttype questions using a three-point scale seeking the frequency of the usage of 15 instructional materials that language teachers are supposed to use in efl classrooms. it was assumed that in order to motivate and draw learners’ attention teachers should vary the types of materials used. data analysis to conduct this descriptive study, the questionnaire was distributed to 68 elt department students. they were asked to evaluate 38 teachers that they had been observing two times a week for three months. they were requested to evaluate objectively and choose the correct usage frequency of materials by teachers. the data obtained from the questionnaire were analysed through spss 17 program using descriptive statistics and frequencies. the researcher also interviewed with five randomly selected teachers during and at the end of the research to explore the reasons behind employing or not employing the instructional materials specified in the research instrument. the questions in the interview were spontaneous to cross-check the data in the questionnaire. findings the aforementioned survey included 15 instructional materials that might be utilized in efl classes by foreign language teachers. the findings of this study were analysed in four groups. materials were classified with regard to their functions in teaching process, such as: main sources, visual aids, audio-visual aids, technological aids, and other materials. the items associated with the main sources included course-books, workbooks, printed materials (worksheets, photocopies etc.), and board; the second category, visual aids, to investigate was composed of visual, audio, and audio-visual aids such as flashcards, pictures/posters, audio/recording materials (speakers, etc.), songs or podcasts, and videos and films. the third group of instructional materials under research was composed of information communication technologies (ict), interactive white boards and projectors, and internet based materials such as dyned, morpa campus etc. the last part of the research instrument included the materials of real objects, authentic materials, and games. international electronic journal of elementary education vol.8, issue 1, 69-82, 2015 74 table 1. the percentage of the basic materials use instructional materials always sometimes never f % f % f % 1 course-book 35 92.1 3 7.9 0 0 2 workbook 27 71.1 10 26.3 1 2.6 3 printed materials (worksheets etc.) 13 34.2 22 57.9 3 7.9 4 board 32 84.2 6 15.8 0 0 as can be estimated, the materials employed by the participants are those that can be easily available such as course-book, workbook, printed materials, and the board. the results revealed that participants in the study heavily depended on the printed materials listed in table 1. it is clearly indicated above that almost all of the participants (always= 92.1% and sometimes= 7.9%) regarded course-books as the fundamental element of teaching. thus, it is obvious that course-book remains as the main component of english teaching settings as every teacher in this study, without exception, uses it. in addition, almost all of the respondents (97.4%) stated that workbook would help them to practice the target language. apart from the course-book and workbook, additional printed materials were also among the most frequently used ones by the participants to enhance classroom instruction. being a traditional material, the board (always= 84.2%, sometimes= 15.8%) was the other most frequently employed material by the participants. even if it seems to be a traditional or outdated material, it still serves as a useful tool in teaching settings for many classroom teachers. reasons for too much reliance on the course-book and board in a semi-structured interview conducted with five teachers, the reasons for overemploying the course-book and board to assist teaching were investigated. in this respect, following questions were asked. “what are the advantages and disadvantages of coursebooks?”, “why do you think many teachers are heavily dependent on course-books and boards to assist teaching the target language?”, “what makes you feel secure about using the course-book?”. it needs to be clarified here that due to the small sample size, the statements below cannot be overgeneralized for all the teachers of english working at elementary schools in turkey. the following is a selection of representative interview quotes from teachers talking about their heavy reliance on the course-book and board:  they are ready-made materials and everything is provided. (t1, t3, t4)  when we have a course-book we feel more secure. (t2, t4, t5)  we don’t have enough time to prepare extra materials.(t2, t3)  they are enough to teach the target language. (t4, t5)  we need to follow the syllabus set by ministry of education (moe). (t1, t2, t3, t5)  when we use extracurricular materials, students cannot follow them easily. (t3, t5)  we feel that we need to write everything on the board. (t3, t4)  students are used to copying everything on their notebooks. (t2, t4)  it is easier to explain the teaching point on the board. (t3, t5)  students do not learn when we don’t use the board. (t1, t4)  when i don’t write the rules/sentences on the board, i feel something missing in my teaching. (t3, t5) instructional materials commonly employed / çakır 75 as is clearly seen course-book keeps its popularity in the eyes of the teachers. in this regard, prodromou (1999) points out that teachers do not often introduce any extraneous material into the lesson as they regard the course-book as a holy book which is not tampered with or questioned. according to the quotes obtained from the participants, a general view is that students don’t need to be offered other instructional materials when the course-books are accompanied with the board. additionally, they think that these two close friends would be sufficient for both themselves and students as long as they are exploited properly. table 2. the percentage of visual, audio, and audio-visual aids use instructional materials always sometimes never f % f % f % 5 pictures /posters 4 10.5 15 39.5 19 50 6 flashcards 2 5.3 12 31.6 24 63.2 7 audio/recording materials (speakers, etc.) 2 5.3 10 26.3 26 68.4 8 songs/podcasts 4 10.5 9 23.7 25 65.8 9 videos/films 3 7.9 12 31.6 23 60.5 table 2 indicates that half of the participants (50%) prefer pictures/posters as visual aids to facilitate teaching. the reason why they are used more than others might be due to their availability. despite their effectiveness and availability, it is seen that more than half of the participants (never= 63.2%) do not utilize flashcards as instructional materials. the other type of instructional materials that were investigated in this study were audio/recording materials. research proves that listening materials have a great benefit for learners to acquire the target language as they provide linguistic and communicative input. however, the results indicate that 68.4% of the participants do not pay much attention to the use of audio/recording materials. this fact is also supported with the results obtained from other audio visual materials such as songs and podcasts (never= 65.8%) and videos and films. the percentage of those who never use video in their efl classrooms is 60.5%. it seems that less than half of the participants (always f= 2, sometimes f= 10) are eager to use audio and recording materials as technological tools. however, the use of audio materials would always be beneficial for non-native speakers as such kinds of tools bring the real language into the classroom. the productive role of video and audio materials in efl setting is undeniable. they enable learners to be exposed to the target language in its real context. it is not always possible for non-native english teachers to otherwise create realistic language-usage situations in the classroom. furthermore, recording materials, when applied appropriately, can be used for multiple purposes such as listening to audio recordings, recording learners’ own voices, conducting a podcast activity, listening to songs and podcasts etc. in other words, the use of podcasts as a new technology has huge potential in enhancing second language learners’ listening and speaking skills (jarvis, 2004; stanley, 2005). in this study, the results show that the percentage of those who use audio/recording materials is very low (always= 5.3%, never= 68.4%). to maintain an enjoyable and attractive teaching atmosphere, songs, and more recently podcasts are regarded as two important materials that deserve to be utilized by teachers. particularly the use of songs which are now available in many course-books delivered by the ministry of national education carries crucial importance in teaching some language patterns and chunks. educationally speaking, young learners, even adults, enjoy learning through songs. however, this research proves that many of the teachers (never= 65.8%) are not particularly willing to make use of songs. international electronic journal of elementary education vol.8, issue 1, 69-82, 2015 76 table 3. responses to the use information communication technologies (ict) instructional materials always sometimes never f % f % f % 10 projector/interactive white board (iwb) 2 5.3 13 34.2 23 60.5 11 internet based materials 1 2.6 18 47.4 19 50 12 courseware programs (dyned, morpa campus, etc.) 2 5.3 6 15.8 30 78.9 the data obtained from the class observations and semi-structured interviews demonstrate that teachers rarely use ict materials. it was noticed that teachers were reluctant to use projection device and iwb although many schools had either projection device or iwb. according to the results, 60.5 % of the participants do not use it whereas only 5.3% of the participants adopt it for educational purposes. when it comes to internet based materials, it is seen that half of the participants use them (always= 50%). the other type of instructional material that is freely offered to the use of students at state elementary schools is dyned courseware. apart from dyned, some other online e-course software programs such as morpa campus, dahiders, etc. are also available for many foreign language teachers. as it is indicated in table 3, most of the teachers (78.9%) do not use the freely offered courseware as a supplementary material for learners. on the other hand, when utilized appropriately, mobile assisted language learning (mall) based materials entail myriad of benefits for foreign language learners in that they are fast, effective, flexible, authentic, personal, motivating, portable, and sometimes informal. furthermore, as they are easy to use anytime and anywhere, the effective use of technology in and beyond the walls of the classroom would help learners gain autonomy. table 4. responses to the use of other materials instructional materials always sometimes never f % f % f % 13 real objects 4 10.5 11 28.9 23 60.5 14 authentic materials 1 2.6 10 26.3 27 71.1 15 materials for games 5 13.1 18 47.4 15 39.5 the positive role of real objects and authentic materials in foreign language settings is undeniable and they should be utilized in every stage of language teaching. in this way, it would be possible for learners to study and acquire the target language meaningfully and communicatively, rather than through mechanical and rote learning. however, the results of this study demonstrate that such crucial materials are not employed as required. as shown in table 4, while one third of the participants (28.9%) used authentic materials, the percentage increased to 38.9 % for real objects. on the other hand, 60.5% of the participants prefer games for instructional purposes in the classrooms, which supports the belief that the participants are aware of the benefits of games in elementary education. thus, it is highly suggested by the researcher that young learners should be introduced to the target language by using concrete objects efficiently. reasons for not employing instructional materials effectively to figure out the reasons for not using a range of instructional materials sufficiently and efficiently, five randomly selected teachers were requested to express their opinions. therefore, the following spontaneous questions were asked to the interviewees. “do you use additional materials in the class?”, “what sort of problems do you encounter when you use extra instructional materials?”, what do you think about utilizing instructional materials in teaching english?”. based on the results, the following statements clearly reveal that employing extracurricular activities and materials might bring extra workload instructional materials commonly employed / çakır 77 on many teachers. for that reason, they tend to avoid such efforts. some of the prominent results that the teachers stated are listed as follows:  we have to cover the content of the curriculum in a given period. (t1, t2, t3, t4, t5)  my students don’t understand when i use extra material. (t3, t4)  once i tried to use audio material, the class didn’t react positively. they didn’t understand and lost their motivation. (t2)  i have too much workload. i can’t spend much time on the use of materials. (t3, t4, t5)  i have a course-book. i think that is enough for my class. (t2, t4, t5)  students need to learn grammar first. (t3)  my students won’t go abroad. so i don’t need to focus on communicative activities that require additional materials. (t4)  when i attempt to use an instructional material apart from the book, i need to prepare beforehand. however, the course-book entails all the activities for teaching english. (t1,t3,t4)  it is not my way of teaching. it is not easy to get used to novelty. it takes time. (t1, t4) the statements above clearly reveal that utilizing extracurricular instructional materials is not quite possible for the participants due to some factors. possible factors include the content of the curriculum, time constraints, time allocated to the course, adapting the new materials etc. the participants also believe that a course-book itself would be efficient to cover the units. in general, it can be concluded that utilizing instructional materials apart from the course-book is not appreciated adequately by the participants. discussion the current study aimed to determine the types of instructional materials that the foreign language teachers employed to teach english in the state elementary schools in a city in the centre of turkey. to do this, 38 teachers were observed for fourteen weeks in terms of the instructional materials they preferred to supplement their english teaching. in this descriptive study, five of the teachers who were observed were also interviewed during the term to validate the data and to explore their reasons for using or not using some of the materials. a general finding in this study is that most of the teachers are not so very willing to take part in teaching activities that require additional material, effort and time. when analysed carefully, it is possible to come across some factors such as burnout, not attending to professional development activities, limited technological knowledge, limited time to prepare and apply the related materials, workload, etc., one of the important factors that lead teachers to avoid implementing the innovative techniques and materials in the efl classes would be to suffer from burnout (cephe, 2010). for that reason, it is suggested that the teachers need to update themselves and their knowledge regularly in order to keep up with the innovations in the field. depending on the results obtained in this study, it is highly suggested that teachers should participate in the conferences, workshops, symposiums and in-service training programs. to support this view, akyel (1999) collected some data from the semi-structured interview with foreign language teachers to find out the reasons for not participating in the professional development activities. the results that she obtained are consistent with the findings of the present international electronic journal of elementary education vol.8, issue 1, 69-82, 2015 78 study, including too much workload, low salary, not enough enthusiasm, administrative problems, etc., professionally speaking, it can be stated that foreign language teachers should attend professional development activities regularly in order not to lose their motivation and to stay up-to-date. based on the results in this study, it can also be stated that course-books still remain the indispensable teaching tool for many teachers. sheldon (1988), hutchinson and torres (1994), and richards (1990) believe that course-books lie at the heart of any english language teaching situation. furthermore, they offer advantages for teachers and language students alike, and they constitute a useful resource for both teachers and learners. moreover, they are irresistible tools in elt contexts because, as indicated in this study, it is difficult for teachers themselves to create their own teaching materials. not only for teachers, but also for students course-books are an unavoidable component of language instruction since their progress and achievement can be measured concretely when they are used in the english classroom (haycraft, 1998; hashemi, 2011). the interviewees in this study also generally stressed that course-books allowed them to guarantee a degree of consistency in the course taught by a number of different teachers. although the participants in this study seem to depend heavily on course-book and the board, it is clear that they have tendency to use some other audio visual materials to facilitate language learning in the classroom. pictures (50%) and flashcards (36.8%) are two sorts of visual materials that the participants employ in the classroom. the percentage of the ones who always and sometimes use video in efl classrooms is 39.5% with 5.3% use of audio recording materials, it can be concluded that the participants are not technologically friendly enough to cover the courses as required. on the other hand, it is a fact that use of audio-visual materials makes learning more long-lasting than the use of traditional textbooks (craig & amernic, 2006). findings in this study indicated that teachers, unfortunately to a great extent, tend to use the traditional instructional materials which do not require too much preparation for the teachers despite the existence of many other current instructional materials. one of the reasons for not utilizing some instructional materials in the class is that they are not always possible for the participants to reach. however, although many schools have either a projector device or an iwb and they offer a lot of advantages for language learners, the results (never= 60.5%) show that they are not adopted as required. the results of this study support the finding of turel and varol (2012) in that teachers are reluctant to use technology due to limited knowledge of technology. additionally, the results of the study prove that authentic materials, real objects and materials used for games aren’t exploited efficiently. in order to expose learners to the target language in its real contexts authentic materials such as radio programs, magazines, news, songs, etc. need to be introduced to the learners. the result (28.9%) in this context is not satisfactory for the benefits of foreign language learners. as brinton (1991) asserts, authentic materials and media bring the outside world into the classroom and thus students can build the direct relationship between the language classroom and the outside world. the number of those who prefer to use real objects is a lit bit higher (38.9%). in foreign language teaching contexts, real objects usually have a crucial role considering the fact that the more learners are exposed to concrete objects the faster they learn. conclusion and recommendations this study sought to determine the types of instructional materials employed by foreign language teachers at elementary schools. being a hard task for foreign language teachers, teaching should be supported with multiple instructional materials. to this end, teachers should also be well-prepared, well-qualified, and well-motivated. in the current century, instructional materials commonly employed / çakır 79 due to the rapid advancement of technology several technological devices have appeared to be utilized for the benefit of language teachers. most of the learners, who are called as digital natives today (prensky, 2001), expect to be provided with these instructional materials as well as the traditional ones. therefore, the choice and appropriate use of instructional materials appropriately has a vital importance to increase the effectiveness of instruction. these findings suggest that that it is necessary to refine the traditional foreign language teaching classrooms. to create an interactive and effective learning environment, learners should be provided with audio and video supported materials. if possible, learners should be able to access language learning materials easily, and also be able to communicate with their teachers and peers beyond the walls of the classroom. as for the teachers at elementary school, the use of technology needs to be encouraged and they should know how to integrate technology into the curriculum. it is crucial to know how to utilize materials at hand meaningfully and responsibly. the research proves that heavy reliance on the course-book does not lead the teachers to successful and satisfactory results. rather, they need to consistently vary type of materials; use correct, natural, and current and standard english; and adapt and develop appropriate materials, both commercial and non-commercial, all the time. it can be concluded that apart from the ready-made and printed materials, other instructional teaching materials, which are most likely to draw learners’ attention, are also overlooked by many of the foreign language teachers. even though most of the classrooms at the elementary schools are equipped with projection devices and iwbs, it is observed that some of the teachers remain reluctant to utilize them. it is safe to say that this study has highlighted the continuous importance of professional development of foreign language teachers through seminars, in-service training programs, conferences, and workshops. thus, it is suggested that these development activities make it possible for teachers to gain personal growth, to familiarize themselves with innovative techniques in the field, to have professional enthusiasm, and to better exploit the materials at hand for the benefit of learners. the present study has some limitations such as types of research methods (questionnaire, semi-structured interview), number of the participants (n= 68), types of instructional materials researched (15 types), date of the research (2012 fall term), and analysis of the data (descriptive). further study with more sampling groups using other research methods is therefore recommended. however, the present findings of this study may offer insights for educational administrations, decision and policy makers, and, of course, for many of the teachers of english working actively at schools today. • • • acknowledgements i would like to thank tuğba birdal and my colleagues for their invaluable contributions to this study for collecting the related data and proofreading the paper. international electronic journal of elementary education vol.8, issue 1, 69-82, 2015 80 references abebe. t. t.,& davidson. l. m. (2012). assessing the role of visual teaching materials in teaching english vocabulary. language in india, 12(3), 524-552. akyel, a. (1999). teachers’ attitudes towards their job and professional development in turkey. an efl perspective. mediterraneal journal of educational studies, 4(1), 101-110. ali, z., ghani, f., & ali, a. (2010). the use of instructional technology in private schools and national education policy. the dialogue, 6(2), 161-172. brinton, d.m. (1991). the use of media in language teaching. in m. celce-murcia (ed.), teaching english as a second or foreign language, (pp. 454-472). boston, ma: heinle and heinle publishers. brown, j. d. (1995). the elements of language curriculum: a systematic approach to program development. boston: heinle & heinle publishers. cephe, p. t. (2010). a study of the factors leading english teachers to burnout. hacettepe university journal of education. 38, 25-34. chomsky, n. (1988). language and problems of knowledge: the managua lectures. cambridge, ma: mit press crawford, j. (2002). the role of material in the language classroom: finding the balance. in richards, j. c. & renandya, w. a. (eds.) methodology in english teaching. an anthology of current practice. (pp.80-89) cambridge: cambridge university press. cunningsworth, a. (1995). choosing your course-book. oxford: macmillan heinemann elt. çakır, i̇. (2006). the use of a video as an audio-visual material in foreign language teaching classroom. the turkish online journal of educational technology, 5(4), 67-72. çakır, i̇. (2010). the frequency of culture-specific elements in the elt course-books at elementary education in turkey. novitas royal, 3(2), 136-146. craig, r. j., & amernic, j. h. (2006). powerpoint presentation technology and the dynamics of teaching. innov high educ, 31, 147-160. hashemi, r. (2011). textbook selection and evaluation in efl context. world journal of education, 1(2), 62-68. haycraft, j. (1998). an introduction to elt. harlow: longman. hutchinson, t., & torres, e. (1994). the textbooks as agent of change. elt journal, 48(4), 315-328. jarvis, h. (2004). investigating the classroom applications of computers on efl courses at higher education institutions. journal of english for academic purposes, 3(2), 111-137. jones, s. (2009). a retrospective evaluation of an elt course-book for a korean university conversation course. ma tesol/tefl, module 3. retrieved from http://www.birmingham.ac.uk. kayapınar, u. (2009). course-book evaluation by english teachers. inönü university journal of faculty of education, 10(1), 69-78. kitao, k., & kitao, s. k. (1997). selecting and developing teaching/mearning materials. retrieved from http://iteslj.org/articles/kitao-materials.html krashen, s. (1985). the input hypothesis. torrance, california: loredo publishing co. kuzu, a., akbulut, y., & şahin, m. c. (2007). application of multimedia design principles to visuals used in course-books: an evaluation tool. the turkish online journal of educational technology, 6(2), 8-14. lee, w. r., & coppen, h. (1971). simple audio-visual aids to foreign language teaching. london: oxford university press. instructional materials commonly employed / çakır 81 lee, w. (1995). authenticity revisited: text authenticity and learner authenticity. elt journal, 49(4), 323-328. littlejohn, a. (2012). language teaching materials and the (very) big picture. electronic journal of foreign language teaching, 9(1), 283–297. retrieved from: http://e-flt.nus.edu.sg/ v9s12012/ littlejohn.pdf. mayer, r.e. (2009). multimedia learning (2nd ed). cambridge: cambridge university press. mcdonough, j., shaw, c., & masuhara, h. (2013). materials and methods in elt: a teacher’s guide (3rd edition). oxford: wiley-blackwell prensky, m. (2001), digital natives, digital immigrants part 1. on the horizon, 9(5), 1-6. http://dx.doi.org/10.1108/10748120110424816 prodromou, l. (1999). how to be boring teacher? english teaching professional, 12, 16-18. rahimi, m., & hassani, m. (2012). attitude towards efl textbooks as a predictor of attitude towards learning english as a foreign language. procedia-social and behavioral sciences, 31, 66-72. rahimpour, m., & hashemi, r. (2011). textbook selection and evaluation in efl context. world journal of education, 1(2), 62-68. rao. t.s., & jyoti, a. a. v. (2012). utilization of audio-visual aids in teaching english at primary level. international journal of multidisciplinary educational research, 1(2), 311-318. richards, j. c. (1990). the language teaching matrix. cambridge: cambridge university press. richards, j. (2001). curriculum development in language education. cambridge: cambridge university press. richards, j. (2010), curriculum approaches in language teaching: forward, central and backward design. relc journal. 44(1), 5–33. richards, j. c. & renandya, w. a. (eds.) (2002). methodology in language teaching: an anthology of current practice. usa: cambridge university press. sheldon, l.e. (1988). evaluating elt course-books and materials. elt journal, 42(4), 237-246. shyamlee, s. d., & phil, m. (2012). use of technology in english language teaching and learning: an analysis. international conference on language, medias and culture ipedr. 33. (pp. 150-156). retrieved from http://www.ipedr.com/vol33/030-iclmc2012-l10042.pdf solak, e. & çakır, e. (2015). exploring the effect of materials designed with augmented reality on language learners' vocabulary learning. journal of educators online 12(2). retrieved from http://www.thejeo.com/archives/volume12number2/solakcakir.pdf stanley, g. (2005). podcasting for elt. retrieved january 17, 2015, from http://www.teachingenglish.org.uk/think/resources/podcast.shtml tok, h. (2010). tefl textbook evaluation: from teachers’ perspectives. educational research and review, 5(9), 508-517. retrieved from http://www.academicjournals.org. tomlinson, b. (2008). language acquisition and language learning materials. in b. tomlinson (ed.), english language teaching materials (pp. 3-14). london: continuum. tomlinson, b. (2012). materials development for language learning and teaching. language teaching. 45(2), 143-179. tsiplakides, i. (2011). selecting and english course-book: theory and practice. theory and practice in language studies, 1(7), 758-764. turel, y.k., & varol, f. (2012). pre-service teachers’ choice of physical versus electronic instructional materials. contemporary educational technology, 3(1), 17-35. international electronic journal of elementary education vol.8, issue 1, 69-82, 2015 82 www.iejee.com this page is intentionally left blank http://www.iejee.com/ international electronic journal of elementary education, 2016, 8(4), 167-181. issn:1307-9298 copyright © iejee www.iejee.com what teachers should know about why these students perform so well: an examination of korean-american achievement through student perspectives of east asian parenting beliefs, styles and practices wendi j. otto a  a claremont graduate university, usa received: 2 february 2016 / revised: 24 june 2016 / accepted: 12 august 2016 abstract it has been widely reported that korean-american students as a group outperform most other groups of students in terms of academic achievement due to having parents with especially high academic expectations due to traditional confucian values. to examine this achievement, this study examines the common factors across the indigenous east asian parenting ideologies, styles, and practices based on the perceptions of high-achieving, korean-american primary school students through the east asian parenting model. perceptions about child development and learning, the mother-child relationship, authoritarian parenting style, and specific east asian parenting practices are patterns explored in this study. these findings may inform teachers, pre-service teachers, and others involved in the education of korean-american students about differentiating factors (i.e., specific east asian parental and/or teaching practices) that may be influential for explaining and improving the academic achievement for the children in this group. keywords: korean achievement, asian achievement, korean parenting, east asian parenting model introduction research on parenting styles and children’s developmental outcomes have been prominent in the parenting and achievement literature for decades (baumrind, 1971; (coleman, 1966; darling & steinberg, 1993; grusec & goodnow, 1994). more recent research shows that parenting style is not consistent across families, especially from diverse ethnic and socioeconomic backgrounds (grusec, goodnow, & kuzcynski, 2000; spera, 2006; steinberg, lambom, dornbusch, & darling, 1992). though parenting style is most influenced by cultural value orientation and socialization goals such as educational  wendi j. otto, 150 e 10th st, claremont graduate university claremont, ca 91711, usa. e-mail: wendiottophd@gmail.com http://www.iejee.com/ international electronic journal of elementary education vol.8, issue4, 167-181, 2016 168 achievement in asian-american families (chao, 1994), data disaggregation on diverse asian (i.e., east, southeast) parenting practices is needed. further, east asian students and korean-american students in particular have emerged as high performers in the u.s., consistently outperforming other groups including other asian-american subgroups (nclb, 2011; nces, 2013). given this track record, what is lacking in the literature on this subject is a body of work for educators which looks at parenting styles and practices for high-achieving korean-american students as they relate to academic achievement. unfortunately, literature for teachers which examines the parenting styles and practices for high-achieving, korean-american students is negligible. purpose of the study the purpose of this study is to examine the common perceptions and differentiating factors regarding east asian parenting ideologies (parental beliefs about child development and learning and the mother/father-child relationship), styles (parental behaviors that affect the emotional climate of the home), and practices (behaviors defined by specific contextual socialization goals, such as techniques of discipline) across highachieving, korean-american primary school children via the east asian parenting model developed by the author of this study. by examining these second-generation children’s perceptions across these parenting ideologies, styles, and practices, it may be possible for teachers, pre-service teachers, and others involved in the education of korean-american students to learn of the differentiating factors (i.e., specific east asian parental and/or teaching practices) that may be influential for explaining and improving the academic achievement for children in this group. background and literature review to begin, pre-service teachers and teacher educators must understand the role that culture plays in the asian family in order to contextualize the family practices within asian families compared to those within western ones. along these lines, there is tremendous diversity within the asian race as identified by the subgroups in this category according the u.s. census bureau (u.s. census bureau, 2012). despite this diversity, however, western scholarship on asian families continually stresses the sociocultural asian parenting characteristic of interdependence among family members as a predominant asian family trait which begins early in childhood, particularly in the east asian family, where it has been reported that children as young as infants and toddlers learn to become oriented towards this interdependence (chao r., 1994; chao & tseung, 2002; lee s., 1996; takaki, 1998). the concept of filial piety must also be understood, defend as a set of unifying principles which underlie the interdependence n confucian-based societies such as in east and parts of southeast asia and act as guiding principles governing patterns of socialization and rules about intergenerational conflict, (lee, 1996). historically, this philosophy began in china and later spread to japan, korea, and vietnam where it continues to strongly influence parenting in the east asian family through how to admonish and teach children to behave or conduct themselves towards their parents and even their ancestors (kim & hong, 2004; koh & shin, 2005). as a result, filial piety does not just extend to one’s own parents but to the entire family in terms of respecting and honoring the family and the family home. this principle thus emphasizes familial interpendence, as the children’s actions do not just simply have ramifications for themselves, but their actions can bring honor and pride or dishonor and shame to the entire family (kim & park, 2006; koh & shin, 2005; larson & lee, 2000). next, role relationships and reciprocity are the complimentary characteristics to the interdependence described within the asian family, through which authority and what teachers should know about why these students perform so well / otto 169 responsibility are divided and deferred to among certain family members. essentially, different family members fulfill different roles with an overall family system of reciprocity, which is defined by caring and mutual obligation. in turn, parents and other elders within the family hold considerable authority and responsibility and are then treated with great respect by the children (lee, 1996). yet, the parental authority that parents and various other family members hold within the asian family is misunderstood in western thought. in other words, the asian cultural frameworks which emphasize interdependence among family members suggest that the parental authority that asian parents exert over their children reflects a strong family system and proves strong parental love and concern, opposite of some western frameworks and literature on the topic which characterize this parental authority as harsh and stifling (baumrind, 1971; kang, 2002; larson & lee, 2000). unfortunately, as many pre-service teachers have limited cultural competence, the picture of school success is reserved for the asian and asian-american stereotype of the quiet, obedient, superior student above all others. this characterization stems from the “model minority” stereotype of asians-americans, developed as a result of asian-american political behavior in the 1960s as well as the group’s longitudinal national educational successes and attainment rates (chao & tseung, 2002; lee, 1996). despite this oversimplified stereotype, the parental concern in many asian families is indeed over the educational achievement of the children; that is, that the children’s school is seen as the primary responsibility of the parent. in fact, it is commonplace for the efficacy of asian parents to be based upon the educational achievements of their children and that the importance placed on educational achievement for parents in the asian family lay within the broader cultural beliefs and attitudes about child development and learning, parental educational expectations and aspirations, as well as the involvement of the parents in their children’s schooling (chao r., 1994; oh-hwang, 1993; schneider & lee, dec., 1990; shin & yang, 2008). parenting style as context: east asian parenting it was baumrind (1971) who documented and developed the four distinct different styles of parenting the authoritarian, the authoritative, the permissive, and the neglectful of which the authoritative style of parenting has been accepted as the ideal means for socializing children for the internalization of parental values and goals in the united states. this seminal work posted that the parents who develop positive relationships with their children by being firm, not overly controlling, and relying more on reasoning and persuasion than by use of power will have more success in the socialization of their children in any particular given context, particularly educational settings (baumrind, 1971). research on this topic was later extended in the 1980s and 1990s to examine parenting practices along with parenting style on the achievement of children from various ethnic groups (steinberg et al., 1992). this latter seminal work posted that children of different ethnic groups may respond positively to different parenting styles and practices which affect achievement, thus beginning a new academic discourse regarding the influence of parenting on the achievement of diverse groups in the american educational system. this discourse continues in the form of achievement literature utilizing theoretical models which incorporate parenting beliefs, parenting style, and parenting practices within the home environment as influential factors affecting the academic achievement of children from diverse backgrounds (chao, 1994; chao & arque, 2001; darling & steinberg, 1993; spera, 2006). as such, parental ideologies (parenting beliefs, values and goals) are critical factors influencing the home environment and in the korean culture, the uncompromised value of academic achievement appears to be highly influenced by the confucian and international electronic journal of elementary education vol.8, issue4, 167-181, 2016 170 familial values of their korean parents, as well (akiba & zhao, 2009; baumrind, 1971; chao, 1994; darling & steinberg, 1993; grusec & goodnow, 1994; grusec, goodnow, & kuzcynski, 2000; paik, 2008; park, 2007; shin & yang, 2008; sorensen, 1994). as a result, the high academic outcomes for students of korean ethnicity on u.s. national exams suggest that the parenting ideologies which guide the east asian child-rearing experience for korean parents (parental beliefs and practices about child development and learning and the mother/father-child relationship) strongly affect achievement (chao, 1994). theoretical framework the theoretical framework for this study is based on the earlier work of baumrind (1971), chao (1994), and darling and steinberg (1993), whose models were selected to serve as the basis for the formulation of the theoretical framework for this study because each framework alone is insufficient to fully explain korean-american achievement to educators within the context of parenting. therefore, the model for this study builds upon these seminal frameworks by extending chao’s (1994) chinese parenting style model (who extended baumrind’s (1971) western parenting conceptualizations) and darling and steinberg’s (1993) constructs of parenting styles and practices to form the eapm (east asian parenting model), a new framework which seeks to explain korean-american educational success through the interactions between east asian parenting ideologies (beliefs), styles, and practices. the eapm builds upon the chinese (or east asian, hence korean) cultural notion of training developed by chao (1994) in response to the “authoritarian” (controlling and restrictive) parenting style concept developed by baumrind (1971) and completely misunderstood in the recent, yet controversial, best-selling book entitled battle hymn of the tiger mother popularized in western educational circles (chua, 2011) . the eapm also incorporates the contextual parenting style model developed by darling and steinberg (1993) to investigate the lack of understanding of ethnic differences in western europarenting research. thus, the eapm posts that the east asian parenting ideologies of chao shun and guan (child-rearing beliefs and practices about child development and the mother/father-child relationship) from chao’s (1994) model influence both the parenting style (the emotional climate in which parental behaviors are expressed) and parenting practices (behaviors defined by specific contextual socialization goals, such as techniques of discipline) from darling and steinberg’s model (1993), all which directly affect the developmental outcomes of the children (the academic achievement of korean-american primary school students). drawing from chao’s (1994) eastern model of child-rearing through the chinese concepts of chiao shun (teaching or training children in the proper and expected societal behaviors) and guan (control and governance) and their impact on the academic success of chinese students, the eapm presents an extended framework for east asian school success while rejecting baumrind's (1971) parenting style conceptualizations (authoritarian, authoritative, permissive and neglectful) as ethnocentric in the same fashion as chao (1994) and darling and steinberg’s (1994) parenting models do and posts that the western concepts of authoritative and authoritarian do not capture the important features of east asian (including korean) parenting ideologies, styles, and practices, especially when explaining for school success (chao, 1994). as such, the eapm draws upon the simultaneous parental utilization of chiao shun and guan as complimentary values as the foundation to academic success for korean-american students and argues that the chinese concepts of chiao shun and guan must be understood by educators from an indigenous (eastern) perspective in order to fully understand the what teachers should know about why these students perform so well / otto 171 workings of the eapm (chao, 1994; darling & steinberg, 1993; ho & kang, 1984; wu, 1985). as this is a qualitative study, a “broad” interpretation will be used for the cause and effect relationship in the model. therefore, the model will consider four individual factors: 1) student’s perceptions of their parents’ east asian ideologies about child development and learning (chiao shun), 2) students’ perceptions of their parents’ east asian ideologies about the mother/father-child relationship (guan), 3) students’ perceptions of east asian parenting style, and 4) students’ perceptions of east asian parenting practices. the developmental outcome is korean-american academic achievement. see figure1. = developmental outcomes east asian parenting model (eapm) (korean-american academic achievement) figure 1. conceptual model of the east asian parenting model (eapm) research questions the overarching research question guiding this study, therefore, is: what are the common factors (east asian parenting ideologies, styles, and practices) across high-achieving, korean-american primary students? specifically, what are the common factors across east asian ideologies about child development and learning (chao shun) based on the perceptions of high-achieving, korean-american primary school students? what are the common factors across east asian ideologies about the mother/father-child relationship (guan) based on the perceptions of high-achieving, korean-american primary school students? international electronic journal of elementary education vol.8, issue4, 167-181, 2016 172 what are the common factors across east asian parenting style based on the perceptions of high-achieving, korean-american primary school students? what are the common factors across east asian parenting practices based on the perceptions of high-achieving, korean-american primary school students? what are the common factors across east asian parenting ideologies, styles, and practices based on the perceptions of high-achieving, korean-american primary school students? methodology & data analysis methods & design method: this study utilized a qualitative design to explore the common factors across highachieving, second-generation korean-american primary school students based upon the perceptions of their parents’ east asian parenting ideologies, styles, and practices. creswell (2009) states that a qualitative methodology is appropriate where the research looks into the complexity of participants’ experiences and how the meanings they make of these experiences (perceptions) shape their beliefs and actions. krathwohl (2009) further argues that a qualitative approach is most appropriate in research that explores phenomena unable to be captured through quantitative measures. thus, this study provides insight into the indigenous east asian parenting ideologies, styles, and practices as they affect the high academic achievement of korean-american primary school students sample. a purposive sample of second-generation, korean-american, high-achieving, primary students (n=40-total) was utilized for this study. second-generation students were selected because these children were the first generation in their family born in the united states and their selection allowed for the perceptions about their first-generation parents’ east asian parenting ideologies, styles, and practices as they affect their academics. children’s perceptions, as opposed to parental ones, were critical for this study as “the perception of parent style by the child is more important than actual parenting” (steinberg, lambom, dornbusch, & darling, 1992, p. 32). primary students in grades four through six were selected to be used for this study because pre-adolescent children are primarily influenced and controlled by family at this age rather than by outside influences such as negative peers (moon & morash, 2004; morash & moon, 2007; noyot-corbitt & moon, 2010). the students were selected from highly-populated korean-american cities in orange county, ca. study participants were recruited at several after-school tutoring organizations and tae kwon do studios in orange county and asked privately to participate in this research examining korean-american academic achievement. a sample size of 40 was chosen because this number will allow for patterns and trends that affect high-achieving korean-american students to emerge. study participants were all also in grades four through six, had middle-class ses as a “control” variable for success, spoke some degree of english and categorized as “high-achieving” based on their most recent state testing or cst english/language arts and mathematics test score results. high achievement was verified for each study participant via score report copies provided to the pi by parents. snowball sampling was also utilized, as some initial recruits were also asked to recommend acquaintances to the study until all participants were found. protection of human subjects. this study received full irb approval. data & data analysis data. the measurement tool for this qualitative study is one semi-structured interview (appendix a). questions from parts i and ii were adapted from the methods section of chao’s (1994) chinese parenting model. questions from parts iii and iv were adapted what teachers should know about why these students perform so well / otto 173 from the parental authority questionnaire (buri, 1991) which measures parenting style and practices based on baumrind’s (1971) parenting conceptualizations: authoritarian, authoritative, permissive and neglectful. the authoritarian style and practices statements were identified and modified to fit the eapm framework. data analysis. data underwent a content analysis process. as a first step, participant statements were separated by question as the pi read each transcript line by line and wrote down short, descriptive statements or quotes from each statement into microsoft word tables. with this accomplished, summarizing statements were created to best interpret their meaning after which each summary was coded by theme according to the constructs of the eapm. then, all codes across participants were read together for emergent patterns and themes on student perceptions about their parents’ east asian parenting beliefs about child development and learning, the east asian mother-child relationship and east asian parenting styles and practices, culminating in a total number of 17 results tables for sections i, ii, iii, and iv of the structured interview (appendix a). contact the author to request copies of such data tables. results this study purports to provide a theoretical and empirical basis for understanding the common factors across east asian parenting ideologies, styles, and practices which inform high-achieving, korean-american primary school children via the east asian parenting model framework developed by the author, which proposes an intersection of achievement (the developmental outcomes) between these constructs for koreanamerican, primary school students. see figure 2. figure 2. the east asian parenting model in terms of the research questions as the constructs overlap and form an intersection of achievement for korean-american academic success. rq #1: what are the common factors across east asian ideologies about child development and learning based on the perceptions of high-achieving, korean-american primary school students? the findings suggest that the participants’ perceptions of their parents’ east asian parenting ideologies about child development and learning, as they relate to academic rq #5 international electronic journal of elementary education vol.8, issue4, 167-181, 2016 174 achievement, are consistent with the indigenous east asian concept of chao shun in that the overwhelming majority of parents are perceived to believe that children must be trained (92.5%) by age 5 (39% of the 92.5%) in the proper behaviors (29% of the 92.5%) and that children can improve in almost every situation (95%) through effort (31% of the 95%) and hard work (46% of the 95%). though not an overwhelming majority, the majority (58%) of the study participants perceive their parents to prefer their children be in the company of adults rather than in the company of other children to monitor or teach proper behavior (41% of the 58%) and to limit exposure and/or the influence of negative peers (41% of the 58%). though the majority of the parents do spank (72% of the 54 total responses), less consistent are the students’ perceptions about the reasons for parental spanking, reporting to be spanked for behavioral reasons (31%) and/or moral shortcomings (30%). a solid 19% of study participants report their parents to not believe in spanking under any circumstances. rq #2: what are the common factors across east asian ideologies about the mother/fatherchild relationship based on the perceptions of high-achieving, korean-american primary school students? results of the findings suggest that the participants’ perceptions are consistent with the indigenous east asian concept of guan in that the majority of study participants perceive a physically and emotionally close relationship with their mother through multiple responses about emotional support, physical affection, health and wellness (24%, 23%, and 22% of 97 responses, respectively) and sleeping arrangements, with 33 out of the 40 study participants (83%) reported to having slept with their mothers at least up through the ages of three to seven. results further suggest that the participants’ perceptions of their parents’ east asian parenting ideologies about the mother/father-child relationship, as they relate to academic achievement, are less consistent for the care-giving and sacrificial roles of the mother as defined in the indigenous east asian concept of guan. these parenting ideologies are perceived by many of the study participants’ as not being solely reserved for the mother but also for immediate family (62% out of 42 responses) and many participants disagree with (45% out of 40 participants) or are unsure of (18% out of 40 participants) whether mothers should sacrifice everything for their children’s education, with three participants vehemently opposed to this belief and interjecting this opposition into their responses to this interview question. as for fathers, results suggest that they are perceived by the study participants to assist with their children’s education through active participation (74% of the 57 responses). rq #3: what are the common factors across east asian parenting style based on the perceptions of high-achieving, korean-american primary school students? results of the findings suggest that the participants’ perceptions of their parents’ east asian parenting style, as it relates to academic achievement, is consistent with the east asian parenting and achievement literature in that the overwhelming majority of the study participants perceived themselves as not having any say in parental educational decisions (88% of responses) though the majority of study participants perceived a positive emotional climate in their homes (55% of responses). also consistent with this same literature are the reports by study participants who perceived negative parental responses if opposition to parental decisions is voiced (55% of responses). the responses also include swift and firm reactions from their parents when parental expectations are not met (74% of responses). see tables 10-13. what teachers should know about why these students perform so well / otto 175 rq#4: what are the common factors across east asian parenting practices based on the perceptions of high-achieving, korean-american primary school students? results suggest that the participants’ perceptions of their parents’ east asian parenting practices, as they relate to academic achievement, are consistent with the east asian parenting and achievement literature in some ways yet vary in others. over one-third of the study participants perceived their parents as utilizing educational practices on them far beyond any other practices done “for their own good” (39% of responses), all the while expecting the study participants to comply with parental requests within five minutes (67.5% of responses). a large number of study participants also reported that they perceived their parents teaching them how to behave institutionally as well as publicly through explicit behavioral directions (57% of responses) and over one-half of the responses reported by study participants (54% of responses) perceived their parents to have and utilize social capital for their educational needs through the utilization of available resources and friends (31% and 23% of responses, respectively). rq #5: what are the common factors across east asian parenting ideologies, styles, and practices based on the perceptions of high-achieving, korean-american primary school students? answers which yielded over 50 percent of responses for any of the semi-structured interview questions for each construct within the theoretical model are considered to be the common factors across high-achieving korean-american children. these common factors, as they affect academic achievement, are consolidated into figure 3. the common themes found to affect the intersection of high achievement for this group within the parenting ideology about child development and learning were that the majority of the study participant perceived their parents to (1) believe in children’s early training (learning to be self-controlled and behave properly) by the age of 5, (2) the effort model for learning, (3) the preference for children to be in the company of adults for purposes of behavior modeling, and (4) spanking for behavioral reasons or moral misbehavior is appropriate, suggesting that the indigenous east asian parenting ideology of chao shun currently informs the intersection of achievement for korean-american highachieving students. more common themes found to affect the intersection of high achievement for this group within the parenting ideology about the mother/father-child relationship were that the majority of the study participants commonly perceived their parents to believe in mothers having extremely close emotional and physical relationships with their children including, but not limited to, sleeping in their mother’s bed beyond the age of early childhood. the current study also found that fathers have now been included in this construct, differentiating the eapm from other east asian achievement and developmental frameworks, as fathers are commonly perceived by the majority of the study participants as assisting with school through active participation and to be present in their lives, suggesting that at least part of the indigenous east asian parenting ideology of guan also currently informs the intersection of achievement for these children. furthermore, the eapm proposed that the parenting ideologies of chao shun and guan would influence the parenting styles and practices utilized on the study participants which would then form the intersection of high achievement for their children. the common themes found to affect this intersection regarding east asian parenting style and practices were that the study participants commonly perceived their parents to believe in providing a happy and comfortable home for them where the parents served as primary decisionmakers, whose expectations regarding their children were to be met as best they could and they also commonly perceived their parents to provide explicit institutional and international electronic journal of elementary education vol.8, issue4, 167-181, 2016 176 public behavioral directions in addition to expected parental requests to be responded to immediately, respectively. the care-giving and sacrificial roles of the mother, as they relate to the east asian ideology about the mother/child relationship, were not found to be common factors affecting achievement along with parental practices done for the child’s “own good” and parental social capital utilization patterns, as all four of these factors yielded no clear majority response from the study participants. therefore these factors were not included in the intersection of achievement and were not included in figure 3. figure 3. student perceptions of the 13 common factors across high-achieving, koreanamerican primary school students within the eapm that form an intersection of achievement. discussion and conclusions the east asian parenting model predicts that when all the common factors presented in figure 3 regarding student perceptions of their parents’ east asian parenting ideologies, styles and practices are present, achievement will be high for korean-american students. as the results suggest, this model serves well to explain the high achievement of this particular group, as 13 out of the original 17 eapm factors explored in the interview what teachers should know about why these students perform so well / otto 177 protocol were found to be common to form an intersection of achievement among the students. overall results from this study suggest that the eapm serves well to comprehensively explain to teachers, pre-service teachers, and educators in general about the common factors across highachieving, korean-american primary school students from the perspectives of the students themselves. these common perceptions, which form the intersection of achievement, can inform teachers and pre-service teachers as to an explanation of why the korean-american students in their classrooms perform so well, independent of the pedagogical and curricular structures within school districts in the public k-12 system. thus, what american teachers need to understand is that the findings from the current study strongly suggest that the east asian cultural and confucian beliefs about child development and learning (chao shun) and the close dynamics of the east asian motherchild relationship, combined with the emerging, active relationship of the “new” korean father (evolving definition of guan), serve as developmental foundations for the child; foundations on which the child is both physically and mentally “trained” to accept through both the controlling yet supportive east asian parenting style and the strict behavioral practices designed to result in korean-american academic achievement. adding to the field of asian-american achievement literature in addition to informing teachers and pre-service teachers about the home-school achievement connection for korean-american primary students, these results also suggest that the indigenous east asian guan parenting ideology is adapting to american acculturation via changing and evolving paternal roles for the korean mother as well as for the korean father within the present-day korean-american family. this is an important new piece of research, as the adaptation of the guan ideology to american acculturation is a clear departure from chao’s (1994) findings a generation ago. implications for future research results of the current study suggest that the study participants are achieving at high levels despite the guan ideology (including the role of the korean father) adapting to acculturation. future research could benefit from a further exploration of an alternate definition of guan as one that substitutes the sacrificial and care-giving roles of the korean mother for the active participation of the father and other immediate and extended family members and the role this alternative definition plays in the academic achievement of korean-american students; possibly in terms of an east asian american parenting model (eaapm) which utilizes this new definition of guan as a foundational ideology for koreanamerican academic success. further, additional research on an alternate definition of guan may help inform educators working within school settings (including tutors and hagwon) about ways to strengthen paternal roles in the education of korean-american children. international electronic journal of elementary education vol.8, issue4, 167-181, 2016 178 for teachers and teacher education this study can also bring about an awareness and appreciation of the cultural and structural factors within eastern achievement frameworks that influence the education of korean-american students from the children’s perspective to teachers and pre-service teachers, as most curriculum and teacher education programs primarily focus on the cultural and structural factors within western achievement frameworks to explain the high and/or low academic achievement for minority students within the current educational system. last, this study may serve to culturally broaden the students within such teacher education programs, as instructing teachers and pre-service teachers about korean-american children’s perceptions of east asian parenting beliefs (including the role of east asian fathers), styles, and practices as they affect achievement can diversity their perspectives to increase their awareness of the common, if not predictive, factors affecting the academic achievement of the children in this group. • • • references akiba, m., & zhao, h. (2009). school expectations for parental involvement and student mathematics achievement: a comparative study of middle schools in the us and south korea. compare, 39(3), 411-428. baumrind, d. (1971). current patterns of parental authority. developmental psychology monographs, 4(1, pt. 2). bissell-havran, j. m., loken, e., & mchale, s. m. (2012). mothers’ differential treatment of adolescent siblings: predicting college attendance of sisters versus brothers. youth adolescence, 41, 1267– 1279. buri, j. r. (1991). parental authority questionnaire. journal of personality assessment, 57(1), 110119. chao, r. k. (1994). beyond parental control and authoritarian parenting style: understanding chinese parenting through the cultural notion of training. child development, 65, 1111-1119. chao, r. k., & arque, c. (december 2001). extending research on the consequences of parenting style for chinese americans and european americans. child development, 72(6), 1832-1843. chua, a. (2011). battle hymn of the tiger mother. new york: the penguin press. coleman, j. (1966). equality of educational opportunity (coleman) study (eeos). icpsr06389-v3. ann arbor, mi: inter-university consortium for political and social research [distributor], 200704-27. doi:10.3886/icpsr06389.v3. darling, n., & steinberg, l. (1993). parenting style as context: an interactive model. psychological bulletin, 113(5), 48-496. goodenow, c., & grady, k. e. (1993). the relationship of social belonging and friends' values to academic motivation among urban adolescent students. the journal of experimental education, 62(1), 60-71. grusec, j. e., & goodnow, j. j. (1994). impact of parental discipline methods on the child's internalization of values: a reconceptualization of current points of view. developmental psychology, 30(1), 4-19. grusec, j. e., goodnow, j. j., & kuzcynski, l. (2000). new directions in analysis of parenting contributions to children's acquisition of values. child development, 71(1), 205-211. lim, h. (2007). a religious analysis of education fever in modern korea. korean journal, 71-98. what teachers should know about why these students perform so well / otto 179 moon, b., & morash, m. (2004). adaption of theory for alternative cultural contexts: agnew's general strain theory in south korea. journal of international and comparative criminal justice, 28, 77-104. morash, m., & moon, b. (2007). gender differences in the effects of strain on the delinquency of south korean youth. youth & society, 38(3), 300-321. nces. (2013). the condition of education 2013. washington d.c.: u.s. department of education. nclb. (2011). no child left behind. retrieved 2011. 20-september from ed.gov: http://www2.ed.gov/nclb/landing.jhtml noyot-corbitt, c., & moon, s. s. (2010). multifaceted reality of juvenile delinquency: an empirical analysis of structural theories and literature. child adolescent social work journal, 24, 245-268. paik, s. j. (2008). altering the curriculum of the home: learning environments for u.s. and korean students. marriage and family review, 43(3/4), 289-307. park, s.-j. (autumn 2007). educational manager mothers: south korea's neoliberal transformation. korea journal, 187-213. piaget, j. (1952). the origins of intelligence in children. new york: international university press. shin, c. s., & yang, s. (2008). parent attitudes towards education: what matters for children's wellbeing? children and youth services review, 30, 1328-1335. sorensen, c. w. (feb., 1994). success and education in south korea. comparative education review, 38(1), 10-35. spera, c. (2006). adolescents' perceptions of parental goals, practices, and styles in relation to their motivation and achievement. the journal of early adolescence, 26, 456. steinberg, l., lambom, s., dornbusch, s., & darling, n. (1992). impact of parenting practices on adolescent achievement: authoritative parenting, school involvement, and encouragement to succeed. child development, 63(5), 1226-1281. wu, d. (1985). child training in chinese culture. in w. s. tseng, & d. wu (eds.), chinese culture and mental health (pp. 113-134). orlando: academic press. yeung, w.-j. j. (2013). asian fatherhood. journal of family issues, 34(2), 143–160. young, n. f. (1972). independence training from a cross-cultural perspective. american anthropologist, 74(3), 629-638. international electronic journal of elementary education vol.8, issue4, 167-181, 2016 180 appendix a semi-structured interview protocol: student perceptions of the east asian parenting model this interview is to be administered by the researcher in english. this interview will cover four sections. sections i and ii are derived from the concepts of chiao shun and guan: "ideologies on child development and learning" (derived from the literature on east asian parenting) and "ideologies on the mother-child relationship" (derived from the same literature and involves such ideas as the child being in constant care of the mother). parts iii and iv cover east asian parenting style and practices questions adapted from the parental authority questionnaire by buri (1991), darling and steinberg’s constructs of parenting style and parenting practices, and a previous pilot study conducted by the author. opening statement: “i am going to ask you some questions. please respond to the following questions as you think your parents believe. your answers can be long or short. just answer the best you can.” part i: east asian parental ideologies about child development and learning (based on chao, 1994) 1. in what ways do your parents feel that a child should be taught as soon as possible to behave properly and have self-control? (i.e., in a restaurant or with adults)? 2. in what ways do your parents feel that a child can improve in almost anything if they work hard? (i.e., school) 3. in what ways do your parents feel that being around other adults instead of other kids helps children to behave? (i.e., with adults instead of daycare) 4. under what circumstances do your parents feel that spanking a child is acceptable? (i.e., disobeying parents)? part ii: east asian parental ideologies about the mother-child relationship (based on chao, 1994) 5. in what ways does your mother show her love for you? 6 how do your parents feel about only mothers taking care of the children in a family?] (i.e., making all meals, transportation arrangements, no babysitters or daycare)? 7. in what ways do your parents feel that mothers should sacrifice everything for a child’s education? (i.e., no outside career or all money earned spent only for tutors or tuition)? 8. how do your parents feel about children being allowed to sleep in the mother’s bed? (i.e., whenever they want and/or at any age)? 9. in what ways does your father play a role in your education? (i.e., parent conferences, homework?) part iii: east asian parenting style (based on buri, 1991 and darling and steinberg, 1994) 10. what is the emotional climate (mood) of your home? (i.e., stressful, tense, loving)? 11. what happens when your parents make any decisions in the home which affect you? (i.e., discipline)? 12. what happens when you disagree with your parents? (i.e., house rules)? 13. what happens if you do not meet your parents’ expectations for you in any given situation? (i.e., behavior or grades)? part iv: east asian parenting practices (based on buri, 1991 and darling and steinberg, 1994) 14. how have your parents made you do things “for your own good”? 15. how quickly are you expected to do something when your parents ask you to do it? 16. what parenting practices do your parents use to teach you how to behave in and outside of school? 17. if you need something for your education (i.e., information, knowledge, calculator), and your parents cannot get it for you, how do your parents get you what you need? what teachers should know about why these students perform so well / otto 181 www.iejee.com this page is intentionally left blank http://www.iejee.com/ international electronic journal of elementary education, 2015, 7(3), 423-440. issn:1307-9298 copyright © iejee www.iejee.com assessing preschool teachers’ practices to promote self-regulated learning  fahretdin hasan adagi̇deli̇  istanbul university, turkey seda saraç institute of creativity, turkey engin ader boğaziçi university, turkey received: 13 march,2015 / revised: 1 may 2015 / accepted: 27 may 2015 abstract recent research reveals that in preschool years, through pedagogical interventions, preschool teachers can and should promote self-regulated learning. the main aim of this study is to develop a self-report instrument to assess preschool teachers’ practices to promote self-regulated learning. a pool of 50 items was recruited through literature review. items, then, were formulated as statements, to which the teachers could respond on a likert-scale. in line with the expert and teacher opinions, twenty statements were removed from the original pool and some statements were reformulated. the latest version of the scale consisted of 21 statements. the participants were preschool teacher (n=169) from istanbul. empirical testing at item and scale level showed that t-srl is a reliable and a valid instrument to assess preschool teachers’ classroom practices promoting self-regulated learning of their children at the age of 3-6. keywords: self-regulated learning, teacher practices, preschool education. introduction today’s rapidly changing societies with the emerging new forms of socialization and new models of economic development where knowledge is the main asset required educational systems to modify themselves. the development of these necessary skills and competencies is one of the most important aims of education. however, the skills and competencies needed for this new world is different from the ones that were required by the industrial mode of production of the past century. with the deseco (the definition and selection of competencies)  paper presented at the 6th biennial meeting of the earli special interest group 16.  corresponding author: fahretdin hasan adagi̇deli̇, istanbul university, faculty of hasan ali yücel education, müşküle sokak no:1 vefa/i̇stanbul, phone: +90 532 697 3113, e-mail: fahretdin.adagideli@istanbul.edu.tr http://www.iejee.com/ international electronic journal of elementary education vol.7, issue 3, 423-440, 2015 424 project, which was carried out in collaboration with a wide range of scholars, experts and institutions, oecd developed a framework of necessary key competencies (rychen & salganik, 2003). reflectiveness is the underlying concept in this framework. being reflective requires individuals to reach a level of social maturity that allows them to distance themselves from social pressures, take different perspectives, make independent judgments and take responsibility for their actions, that is, to use metacognitive skills (thinking about thinking), creative and critical abilities (ananiadou & claro, 2009). in another study carried out in the context of the oecd/ceri project on new millennium learners (nml), oecd aimed at developing a framework for century competencies for the new generation of learners in the light of the requirements of the 21st century (ananiadou & claro, 2009). according to wolters (2010) core competencies in the 21st century framework appear nearly synonymous with the dimensions of selfregulated learning. self-regulated learning the term self-regulation is used to depict individuals’ deliberate and effective use of metacognition, motivation, and strategic action in order to attain goals (butler & winne, 1995; perry & winne, 2006; schunk & zimmerman, 2007). self-regulated individuals exercise metacognition by engaging in and monitoring reflective, analytical forms of thinking. motivation involves goal setting, attributions, and self-efficacy that effect individuals’ commitment to and pursuit and attainment of goals. strategic action is the external manifestation of individual’s metacognition and motivation (perry & vandekamp, 2000). every child is born with the capacity to self-regulate and this capacity for selfregulation develops with age. although biological factors like temperament and predisposed reactivity underpin the development of self-regulation in children (bodrova & leong, 2007; berk, mann, & ogan, 2006), early experiences play an important role on this development (boekaerts, 1997). recent investigations demonstrate that development of effective self-regulation during preschool years is a prerequisite for school readiness and success (e.g. denham, warrenkhot, bassett, wyatt, & perna, 2012; mcclelland, acock, & morrison, 2006). in fact, selfregulation predicts children’s success in school more powerfully than iq tests or math and reading skills upon school entry (blair & razza, 2007; diamond, barnett, thomas, & munro, 2007). self-regulated learning is a special type of self-regulation pertaining to learning that takes place in school or classroom contexts. according to zimmerman (1998) selfregulated learning is the self-directive process through which learners transform their mental abilities into academic skills. self-regulated learning process involves academically effective forms of learning involving metacognition, intrinsic motivation, and strategic action (zimmerman, 1989, 1990, 2002; winne & perry, 2000). research studies on self-regulated learning emerged in the 1980s, gained prominence in the 1990s and has been growing since then (dinsmore, alexander, & loughlin, 2008). according to whitebread et al. (2009), self-regulated learning has three main components; metacognitive knowledge (mk), metacognitive regulation (mr); and emotional and motivational regulation (emr). metacognitive knowledge refers to one’s knowledge about cognition related to person, tasks and strategies. metacognitive regulation refers to some procedural verbalization and behaviours including planning, monitoring, control and evaluation that enable to perform activities in a more structured way. emotional and motivational regulation refers assessing preschool teachers’ practices / adagideli, saraç & ader 425 to monitoring and controlling of motivational and emotional experiences about activities being carried out by children. table 1 shows components and subcomponents of the model along with the descriptions. self-regulated learning of preschool children research on self-regulated learning emerged more than two decades ago to answer the question of how students become masters of their own learning processes (zimmerman, 2008). unfortunately, due to the long-held view that children under the age of 10 have difficulty in coordinating the cognitive and metacognitive processes required to complete complex, multifaceted tasks (winne, 1997; zimmerman, 1990) and very young children are not capable of self-regulated learning in any formal way (zimmerman, 1989, 1990), most research on self-regulated learning has involved learners in upper-elementary grades through college (perry, phillips, & dowler, 2004). however, over the last decade, various indications have been found for suggesting traces of self-regulated learning earlier than expected. according to whitebread, bingham, grau, pino pasternak and sangster (2007), studies in laboratory settings and studies based on children’s self-report data have been underestimating young children’s abilities. studies in which children have been observed in their natural settings and/or while performing familiar tasks showed that young children can and do engage in self-regulated learning (e.g; annevirta & vauras, 2006; istomina, 1975; perry, 1998; perry et al., 2004; robson, 2010; sperling, walls, & hill, 2000; whitebread & coltman, 2010; whitebread et al., 2007). table 1. description of components and subcomponents of self-regulated learning (adapted from whitebread et al. (2009)) components subcomponents descriptions metacognitive knowledge knowledge of persons knowledge about cognition in relation to; self: refers to own capabilities, strengths, weaknesses, academic/task preferences, comparative judgements about own abilities others: refers to others' processes of thinking, universals: refers to universals of people’s cognition knowledge of strategies refers to own knowledge in relation to strategies used or performing a cognitive task, where a strategy is a cognitive or behavioural activity that is employed so as to enhance performance or achieve a goal. knowledge of tasks refers to own long term memory knowledge in relation to elements of the task metacognitive regulation planning refers to the selection of procedures necessary for performing the task, individually or with others monitoring refers to the on-going on-task assessment of the quality of task performance (of self or others) and the degree to which performance is progressing towards a desired goal control refers to a change in the way a task had been conducted (by self or others), as a result of cognitive monitoring evaluation refers to reviewing task performance and evaluating the quality of performance (by self or others). motivationalemotional regulation monitoring refers to the assessment of current emotional and motivational experiences regarding the task control refers to the regulation of one’s emotional and motivational experiences while on task. international electronic journal of elementary education vol.7, issue 3, 423-440, 2015 426 preschool teachers’ practices supportive of self-regulated learning how teachers’ practices affect students’ self-regulated learning have been researched on various educational levels. for example, at the elementary level, hammann, berthelot, saia and crowley (2000) investigated how often teachers coach their students’ learning and the relation of this coaching to students’ strategic learning. the researchers videotaped 11 teachers during daily classroom instructions on 3 occasions. then, the students responded to a questionnaire assessing use of learning strategies. the results showed that only in 9% of the videotaped segments, the teachers coached their students’ learning (e.g. describing cognitive processes, suggesting strategy use, etc.). the mostly recommended strategies by the teachers were; using learning aids, engaging in metacognitive activity and using elaboration strategies for remembering. results also indicated that students’ strategic learning is significantly related to teachers’ coaching of learning. at the secondary level, ader (2013) developed a framework for teachers’ promotion of students’ self-regulated learning. in this ethnographic study with three secondary school mathematics teachers, the researcher focused on the metacognitive component of selfregulated learning. data were collected via observations of the classrooms, audio recording of various lessons and interviews with the teachers. also students’ work and the materials used by the teachers were collected. the researcher showed that there are differences in the teachers’ emphasis on metacognition throughout the stages of the lessons and the activities they used, and during their interactions with the students of different achievement levels and progress with the activities. during the introduction and early stages of the lessons, the students were urged to reflect on their initial work and their knowledge regarding the mathematical concepts involved. other times, due to the teachers’ adoption of a didactic approach to teaching, a lack of emphasis on metacognition was evident. when it comes to earlier levels of education, due to the findings from studies indicating that children show signs of self-regulated learning earlier than previously thought, researchers have been motivated to study the features of the preschool teaching and learning contexts that are conducive to promoting self-regulated learning of young children in preschool years. in stipek, feiler, daniels and milburn (1995) study, children in child-centered preschools and kindergartens were compared to children in didactic, highly academic programs. a total number of 227 children, including children from poor, minority and middle ses families participated. the results showed that children in child-centered classrooms were more willing to cooperate with their classmates and were able to choose from different activities and materials that are interesting and meaningful. on the other hand, the children in teacher-centered classrooms were observed to be more dependent learners, seek for more adult support and be more worried about school. perry and vandekamp (2000), in their observational study in five classrooms (kindergarten to 3rd grade), identified features of classroom environments that promote self-regulated approaches to reading and writing in young children. they found that nonthreatening evaluation practices, involvement in complex reading and writing activities, the provision of autonomy related to what students read and write, and the ability to modify learning tasks to control challenge are all contextual features that improve self-regulated learning in these classrooms. whitebread and colleagues investigated the extent to which different learning contexts (e.g., working individually, in a small group, with an adult) appear to afford differential assessing preschool teachers’ practices / adagideli, saraç & ader 427 opportunities for children to experience and practice their metacognitive skills. the results of this observational study showed that pair work and small group work along with challenging tasks and teachers’ warm approach have been found to be among the practices most supportive of self-regulated learning in preschool settings (whitebread et al., 2007; whitebread & coltman, 2010). despite this growing interest on the features of teaching and learning contexts, due to the time-consuming nature of observational studies, it’s difficult for researchers to carry out a large-scale study to investigate how much teachers promote self-regulated learning in their classroom. a workable instrument is a need. the only workable instrument for carrying out a large study was developed by lombaerts, engels and athanasou (2007), which was developed for primary education context only. thus, the aim of this study is to develop a self-report instrument to assess preschool teachers’ practices promoting selfregulated learning in their classrooms. method participants participants were 169 preschool teachers in istanbul. all teachers participated voluntarily in the study. after initial descriptive analysis, 10 teachers were removed from the original sample as these subjects were outliers for normal distribution on several items. for further analysis were carried out with 159 teachers. the majority of the participants were female (96.2%) and worked with children (83%) aged from 4 to 6. these results are consistent with the population means according to ministry of education statistics (2014). table 2 shows the main sample characteristics of the final sample. table 2. sample characteristics: participants' demographic and professional background (n= 159) characteristic/category % gender male 3.8 female 96.2 year of experience 0-5 years 41.5 6-10 years 27.7 11 years and above 30.8 class size 1-10 children 10.7 11-20 children 67.3 21-30 children 22.0 age of children 3-4 year-old 17.0 4-5 year-old 41.5 5-6 year-old 41.5 types of school public 67.3 private 32.7 international electronic journal of elementary education vol.7, issue 3, 423-440, 2015 428 scale development for the scale development, whitebread et al. (2009) model of self-regulated learning for young learners was adopted. as previously mentioned, according to this model, selfregulated learning has three main components: metacognitive knowledge, metacognitive regulation, and emotional-motivational regulation. under each component, there are also subcomponents. the metacognitive component has three subcomponents, namely, knowledge of person (kop), knowledge of task (kot) and knowledge of strategy (kos). total number of 17 items (kop= 7 items; kot= 4 items; kos= 6 items) was formulated in order to assess to what extent preschool teachers provide opportunities for children to be aware of their own and their peers’ cognition as well as of their knowledge about task and strategies. for the metacognitive regulation (mr) component, a total number of 24 items was formulated under four subcomponents, namely: planning (7 items), monitoring (4 items), control (6 items) and evaluation (7 items) in order to assess to what extent preschool teachers provide opportunities for children, while working on tasks, to plan, monitor, control, and evaluate their cognitive processes while working on tasks. 9 items related to the emotional-motivational regulation (emr) component were formalized under two subcomponents, namely, monitoring of emotions-motivation (5 items) and control of emotions-motivation (4 items) in order to assess to what extent preschool teachers provide opportunities for children to monitor and control their emotional and motivational states. total number of items for three components was 50. the numbers of items mainly reflect the proportionality in the number of subcomponents within each component of self-regulated learning. these items were structured as statements, to which the teachers could respond on a likert-scale ranging from 0 = “never” to 4 = “always”. testing and refining 50 statements for three components were emailed to four researchers from u.k, canada, belgium, and turkey who are experts both in self-regulated learning and preschool education. the experts rated each statement on four dimensions; whether it was clear; whether it was supportive of self-regulated learning; whether it was suitable for preschool context and whether it was reflective of its given self-regulated learning component. the expert opinions were collected to ensure the face validity of the scale. in the light of feedback from the experts, 20 statements were removed from the original pool and some statements were reformulated. the latest version of the scale consisted of 30 statements (mk= 10, mr= 13, emr= 7). although one of the experts was turkish who was knowledgeable in turkish preschool context and curriculum and since the scale’s cultural appropriateness was an important concern, to further ensuring the scale’s appropriateness for turkish preschool education context, five preschool teachers examined the statements in terms of clarity and suitability for turkish context. these teachers rated all the items as suitable for turkish context. however, following the teachers’ suggestions, some terms were changed (e.g. using the “activity” rather than using “task” in the items). subsequently, a scale with 30 statements was formulated as a four point likert-scale, with 0=‘never’, 1=‘sometimes’, 2=‘often’, and 3=‘always’ as possible responses. metacognitive knowledge, metacognitive regulation, and emotional and motivational regulation comprise 10, 13 and 7 items respectively. numbers of items for each subcomponent were shown in table 3. since knowledge of person subcomponent of mk assessing preschool teachers’ practices / adagideli, saraç & ader 429 has various dimensions (metacognitive knowledge of oneself and others) it has more items in this subcomponent in comparison with the other subcomponents. table 3. numbers of items for each subcomponent components/subcomponents # of items metacognitive knowledge person 6 task 2 strategy 2 metacognitive regulation planning 3 monitoring 3 control 3 evaluation 4 emotional and motivational regulation monitoring 4 control 3 results this section involves the report of empirical instrument testing at item and scale level. first, for ensuring the construct validity of the scale, the underlying factor structure of the scale was examined. internal consistency of the subscales and correlations between them were also examined. exploratory factor analysis to investigate the possible underlying factor structure of 30 items, exploratory factor analysis (efa) was used. before running efa, data was tested for the assumptions of efa statistics. according to the assumptions of efa, the determinant of the correlation matrix indicating singularity in the data should be bigger than .00001 and kaiser–meyer–olkin (kmo) measure should be bigger than .80 to assure adequacy of sample size. moreover, barlett's test of sphericity should be significant which indicates that correlation matrix is not an identity matrix. although the data satisfied the assumptions of kaiser–meyer– olkin test (kmo= .879) and barlett's test of sphericity (χ²=1472.344, p<.001), one of the items, i.e. item 27, did not meet the requirement of normal distribution and the determinant of the correlation matrix was too small. therefore, item 27 was removed. 10 participants were also removed because their responses were outliers for normal distribution on several items. in order to reach a determinant value of required magnitude, correlation matrix was examined. 8 items were removed (item 4, 5, 6, 11,16,17,23, 28) according to correlation matrix results since their correlation with other items was not sufficient. therefore, the determinant of the correlation matrix was increased to a new value of 5,519e-005 (> .00001) and a normal distribution was ensured. thus, efa was carried out with 159 participants and 21 items. principal axis factoring method of factor and promax with kaiser normalization method of rotation was used and factors were rotated by promax with kaiser normalization. the results of factor analysis suggested that there were five factors international electronic journal of elementary education vol.7, issue 3, 423-440, 2015 430 underlying structural framework of the t-srl. the eigenvalues were 7.8, 1.8, 1.3, 1.2 and 1.1 respectively. the three-factor model accounted for 51.6% of the common variance. the items with high loadings on the first factor reflected emotional and motivational aspects; the second factor reflected metacognitive regulation during task (planning, monitoring and control); while items loading high on the third factor were representing the metacognitive knowledge of task and strategy. the fourth factor represented the metacognitive regulation after task (evaluation) while the last factor’s items were loaded by the metacognitive knowledge of person. corresponding item loadings within the three factor model are presented in table 4 (see appendix a for the english version; appendix b for the turkish version of the instrument). table 4. results of the principal axis factoring factor analysis (numbers in the parenthesis correspond to item numbers of the 21-item scale) factor i ii iii iv v item 26 (#19) .913 item 30 (#21) .661 item 25 (#18) .660 item 29 (#20) .631 item 24 (#17) .624 item 14 (#10) .729 item 13 (#9) .707 item 19 (#13) .646 item 18 (#12) .524 item 12 (#8) .503 item 15 (#11) .410 item 9 (#6) .881 item 8 (#5) .602 item 7 (#4) .507 item 10 (#7) .492 item 20 (#14) .845 item 21 (#15) .635 item 22 (#16) .482 item 2 (#2) .725 item 1 (#1) .615 item 3 (#3) .359 assessing preschool teachers’ practices / adagideli, saraç & ader 431 the first factor which was labelled “t-srl emotional and motivational regulation” (emr; 5 items) determines to what degree teachers allow children to monitor and control their emotion and motivation in classroom context. the second factor was labelled “t-srl metacognitive regulation during task” (mrdt; 6 items) and determines to what extent teachers provide opportunities to children to plan, monitor and control their tasks while they are involved in tasks. the third factor, labelled “t-srl metacognitive knowledge of task and strategy” (mkts; 4 items) concerns teachers’ efforts to make children aware of characteristics of several tasks and strategies. the fourth factor, labelled “srl metacognitive regulation after task” (mrat; 3 items), aims to determine whether teachers create a classroom context where children evaluate their tasks. the fifth factor labelled “tsrl metacognitive knowledge of person” (mkp; 3 items) assess to what extend teachers provide opportunities to children to be aware of their own cognition. cronbach's alpha for the total scale with 21 items was 0.91. the subscales also had good internal consistency scores separately: .842 for the emotional and motivational regulation; .807 for the metacognitive regulation during task; .787 for the metacognitive knowledge of task; .753 for the metacognitive regulation after task; .718 for the metacognitive knowledge of person. besides, item-to-subscale correlations ranged from 0.67 to 0.86 over five subscales. due to acceptable internal consistency scores for the scale and all subscales (a > 0.70), items of the t-srl emotional and motivational regulation, t-srl metacognitive regulation during task, t-srl metacognitive knowledge of task and strategy, t-srl metacognitive regulation after task and tmetacognitive knowledge of person scale can be considered as a scale, with scores ranging from a minimum of 0 to a maximum of 84. pearson correlations between subscales ranged from 0.37 to 0.61 (p<.001) and can be considered as important in practice (see table 5). in order to examine whether there is a significant difference between scores of teachers who scored at top 27% and bottom 27% on scale, t-test was used. this analysis was accepted as a way of ensuring validity of the scales in the literature (karakelle & saraç, 2007) table 5. correlations between the subscales of the t-srl practice scale emr mrdt mkts mrat mkp emr 1 .580** .479** .549** .523** mrdt .610** .483** .534** mkts .425** .535** mrat .371** mkp 1 **. correlation is significant at the 0.01 level (2-tailed). and results showed that there is a significant difference between scores of top 27% and bottom 27% teachers on the scale. this result provides evidence for the validity of the scale. the scores obtained by the top and bottom 27% of teachers according to their scores from the scale was examined on the subscales to provide evidence for consistency international electronic journal of elementary education vol.7, issue 3, 423-440, 2015 432 between scale and subscales. this analysis was accepted as a way of ensuring validity of the scales in the literature (moore & foy; 1997). the findings verify that each subscale discriminate between those who score high and low on the scale (p<.001; see table 6). that is an indication that there is a consistency between scale and each subscale and it is evidence of validity of t-srl. in order to examine the item-total correlation and the discrimination of items, pearson correlation analysis and t-test was used. these analysis were accepted as a way of ensuring reliability of the scales in the literature (onat & otrar, 2010). results showed that each item in the scale had positive statistically significant relation with the total score (p<.001). moreover, there are significant differences between teachers who got highest scores (top 27%) and those who got lowest scores (bottom 27%) for each item (p<.001). these findings assured that all items belonged to the structure of the scale and each item has discrimination power (see appendix c). table 6. discrimination analysis of the subscales among the top and bottom scorers from the total scale t df p emr -15.549 84 .000 mrdt -13.716 84 .000 mkts -13.474 84 .000 mrat -11.329 84 .000 mkp -11.504 84 .000 conclusion results of the present study showed that t-srl is a reliable and a valid instrument to assess preschool teachers’ classroom practices promoting self-regulated learning of their children at the age of 3-6. in the present study, a relatively small yet diverse group of preschool teachers participated voluntarily. therefore, further research with larger groups is needed. moreover, this study was conducted in the turkish context. cross-cultural studies in which the t-srl will be administered would show the usability of the instrument in different cultures. besides, these studies would reveal comparative results showing how practices of teachers diverse across different culture. whitebread et al. (2009) suggested three main components describing self-regulatory abilities of young children: metacognitive knowledge, metacognitive regulation and emotional-motivational regulation. while metacognitive knowledge has 3 subcomponents as knowledge of person, strategy and task. furthermore knowledge of person is divided into 3 subcomponents; knowledge of self, other and universals. metacognitive regulation consists of 4 subcomponents; planning, monitoring, control and evaluation. lastly, monitoring and control of emotions and motivations formalized as subcomponents of emotional-motivational regulation. preliminary results of validity and reliability analysis of t-srl showed that the factor structure of the instrument was different from the assessing preschool teachers’ practices / adagideli, saraç & ader 433 structure presumed by whitebread et al. (2009). however, structure of t-srl made sense considering the distribution of factors. considering the similarities between the initial theoretical framework and what was found as a result of the analysis conducted, emotional-motivational regulation and metacognitive knowledge of person are the common factors. the major difference found between framework of whitebread et al. (2009) and the factor structure found in this study, appeared in the metacognitive regulation component. while this component comprises 4 subcomponents in framework of whitebread et al., the results of this study suggested a two-phase factor structure, i.e. metacognitive regulation during and after tasks. planning, monitoring and control, which could be considered as metacognitive regulation activities while working on a task, formed the metacognitive regulation during task subscale of t-srl. evaluation, which could be considered as regulatory activities after task performance appeared as metacognitive regulation after task in the present instrument. this might be resulting from teachers’ focus on the sequential progress of students’ activities in the classroom, rather than emphasising the orchestration of multiple subcomponents within metacognitive regulation. this difference between how researchers have in mind and how teachers interpret may be due to the level of understanding on how children self-regulate their learning. since the researchers focus more on self-regulatory activities of children, they have a more detailed conceptualisation. preschool teachers that took part in this study did not have a specific training about self-regulation and metacognition. hence, manifestations of their conceptualisation of metacognitive regulation can be less sophisticated in its dimensionality. items presumed as metacognitive knowledge of others were eliminated according to factor analysis. this could once again be resulting from preschool teachers not having awareness regarding the promotion of metacognitive knowledge of other persons. alternatively, there can be a problem with the wording of these items (item 4, 5, 6). metacognitive knowledge of task and strategy are also separate factors of the aforementioned framework. however, in the present factor distribution, these two factors aggregated in the same factor. as suggested in the literature, the scale with items left should be further analysed with confirmatory factor analysis to assure factor structure of the scale. however, there is another alternative for further improving the scale. the 30 item-scale can be administered to teachers again after editing the removed items’ wording. the advantage of this latter approach would be to acquire the same factor structure presumed in whitebread et al. (2009). there is a need for instruments to assess classroom practices of preschool teachers for promoting self-regulated learning. although there are instruments appropriate for primary classrooms, there is a lack of such an instrument at the preschool level. although there are concerns regarding the use of self-report measures, a self-report instrument is a practical measurement tool for teachers to evaluate their classroom practices, thus allowing researchers to conduct large-scale studies. therefore, t-srl presented in this study would be the first step to fill the gap in research efforts towards developing such a measurement instrument. yet metacognitive and self-regulatory researchers highlight the importance of checking construct validity of such self-report instruments since participants could reflect intentionally or unintentionally a distorted reflection of their actions in their responses to self-report instruments (e.g. veenman, 2005). checking the consistency of data from teachers’ self-reports and data from experts’ observations of teachers’ practices would provide further evidence for validity of the instrument. international electronic journal of elementary education vol.7, issue 3, 423-440, 2015 434 implications the scale developed in the present study, would be a useful instrument not only for researchers in self-regulation and preschool education but also for practitioners in preschools. as mentioned earlier in this paper, there is a lack of instruments to assess practices of preschool teachers to promote self-regulation although there are studies and instruments assessing primary education teachers’ practices (lombaerts et al., 2007; perry & vandekamp, 2000). moreover, the scale would help teachers to develop understanding and awareness of self-regulated learning, therefore, to learn how to create classroom context enriching self-regulated learning and to acknowledge the levels of their children’s self-regulation abilities. it would also be a worthwhile effort to see whether changes occur in teachers’ reports of their teaching practices and the factor structures of their responses with differential levels of training given to teachers about self-regulation and practices to promote students’ self-regulated learning. • • • references ader, e. (2013). a framework for understanding teachers' promotion of students' metacognition. international journal for mathematics teaching & learning. retrieved june 10, 2014 from http://www.cimt.plymouth.ac.uk/journal/ader.pdf ananiadou, k. and claro, m. (2009). 21st century skills and competences for new millennium learners in oecd countries. retrieved from paper prepared for organization for economic cooperation and development: http://www.oecd-ilibrary.org/education/21st-century-skillsand-competences-for-new-millennium-learners-in-oecd-countries_218525261154. annevirta, t., & vauras, m. (2006). developmental changes of metacognitive skill in elementary school children. journal of experimental education, 74, 197–225. berk, l.e., mann, t.d., & ogan, a.t. (2006). make-believe play: wellspring for the development of self-regulation. in d.g. singer, r. golinkoff, & k. hirsh-pasek (eds.), play = learning: how play motivates and enhances children’s cognitive and social-emotional growth (pp. 74 – 100). new york: oxford university press. blair, c., & razza, r. p. (2007). relating effortful control, executive function, and false belief understanding to emerging math and literacy abilities in kindergarten. child development, 78, 647–663. bodrova, e. & leong, d. j. (2007). tools of the mind: the vygotskian approach to early childhood education (2nd ed.). columbus, oh: merrill/prentice hall. boekaerts, m. (1997). self-regulated learning: a new concept embraced by researchers, policy makers, educators, teachers, and students. learning and instruction, 7(2), 151–86. butler, d. l., & winne, p. h. (1995). feedback and self-regulated learning: a theoretical synthesis. review of educational research, 65(3), 245-281. denham, s. a., warren-khot, h. k., bassett, h. h., wyatt, t., & perna, a. (2012). factor structure of self-regulation in preschoolers: testing models of a field-based assessment for predicting early school readiness. journal of experimental child psychology, 111, 386-404. diamond, a., barnett, w.s., thomas, j., & munro. s. (2007). preschool program improves cognitive control. science, 318, 1387-1388. assessing preschool teachers’ practices / adagideli, saraç & ader 435 dinsmore, d. l., alexander, p. a., & loughlin, s. m. (2008). focusing the conceptual lens on metacognition, self-regulation, and self-regulated learning. educational psychology review, 20(4), 391-409. hamman, d., berthelot, j., saia, j., & crowley, e. (2000). teachers' coaching of learning and its relation to students' strategic learning. journal of educational psychology, 92(2), 342. istomina, z. m. (1975). the development of voluntary memory in preschool age children. social psychology,13, 5–64. karakelle, s., & saraç, s.(2007). çocuklar için üst bilişsel farkındalık ölçeği (übfö-ç) a ve b formlari: geçerlik ve güvenirlik çalışması.türk psikoloji yazilari, 10(20), 85-103. lombaerts, k., engels, n., & athanasou, j. (2007). development and validation of the self-regulated learning inventory for teachers. perspectives in education, 25(4), 29-47. mcclelland, m. m., acock, a. c., & morrison, f. j. (2006). the impact of kindergarten learning-related skills on academic trajectories at the end of elementary school. early childhood research quarterly, 21(4), 471 – 490. ministry of education (2014). national education statistics 2013-2014. ankara: milli eğitim bakanlığı. moore, r. w., & foy, r. l. h. (1997). the scientific attitude inventory: a revision (sai ii). journal of research in science teaching, 34(4), 327-336. onat, o., & otrar m. (2010). bilişsel duygu düzenleme ölçeğinin türkçeye uyarlanması: geçerlik ve güvenirlik çalışmaları [adaptation of cognitive emotion regulation questionnaire to turkish: validity and reliability studies], marmara üniversitesi atatürk eğitim fakültesi eğitim bilimleri dergisi, 31, 123-143. paris, s. g., & paris, a. h. (2001). classroom applications of research on self-regulated learning. educational psychologist, 36(2), 89-101. perry, n. e. (1998). young children’s self-regulated learning and contexts that support it. journal of educational psychology, 90, 715–729. perry, n., phillips, l., & dowler, j. (2004). examining features of tasks and their potential to promote self-regulated learning. the teachers college record,106(9), 1854-1878. perry, n. e., & vandekamp, k. o. (2000). creating classroom contexts that support young children’s development of self-regulated learning. international journal of educational research, 33, 821– 843. perry, n. e., & winne, p. h. (2006). learning from learning kits: gstudy traces of students’ selfregulated engagements with computerized content. educational psychology review, 18(3), 211228. robson, s. (2010). self-regulation and metacognition in young children's selfinitiated play and reflective dialogue. international journal of early years education, 18(3), 227-241. rychen, d. s. & salaganik, l. h. (2003). introduction. in d. s. rychen, & l. h. salaganik (eds.), key competencies for a successful life and a well-functioning society (pp. 1-12). göttingen: hogrefe & huber. schunk, d. h., & zimmerman, b. j. (2007). influencing children's self-efficacy and self-regulation of reading and writing through modeling. reading & writing quarterly, 23(1), 7-25. sperling, r.a., r.t. walls, and hill l. a. (2000). early relationships among self-regulatory constructs: theory of mind and pre-school children’s problem solving. child study journal 30(4), 233-252. veenman, m. v. j. (2005). the assessment of metacognitive skills: what can be learned from multimethod designs. in c. artelt, & b. moschner (eds.), lernstrategien und metakognition: implikationen für forschung und praxis, (pp. 1-30). berlin: waxmann. international electronic journal of elementary education vol.7, issue 3, 423-440, 2015 436 whitebread, d., bingham, s., grau, v., pino pasternak, d., & sangster, c. (2007). development of metacognition and self-regulated learning in young children: the role of collaborative and peerassisted learning. journal of cognitive education and psychology, 3, 433–455. whitebread, d., & coltman, p. (2010) aspects of pedagogy supporting metacognition and mathematical learning in young children; evidence from an observational study. zdm the international journal on mathematics education, 42 (2), 163-178. whitebread, d., coltman, p., pino pasternak, d., sangster, c., grau, v., bingham, s., et al. (2009). the development of two observational tools for assessing metacognition and self-regulated learning in young children. metacognition and learning, 4(1), 63–85. winne, p. h. (1997). experimenting to bootstrap self-regulated learning. journal of educational psychology, 89, 397-410. winne, p. h., & perry, n. e. (2000). measuring self-regulated learning. in p. pintrich, m. boekaerts, & m. zeidner (eds.), handbook of self-regulation, (pp. 531-566). orlando, fl: academic press. wolters, c. a. (2010). self-regulated learning and the 21st century competencies. retrieved from paper prepared for the nrc planning meeting on 21st century competencies: http://www.hewlett.org/uploads/self_regulated_learning__21st_century_competencies.pdf zimmerman bj (1989). a social cognitive view of self-regulated academic learning. journal of educational psychology, 81(3), 329-339. zimmerman, b. j. (1990). self-regulated learning and academic achievement: an overview. educational psychologist, 25(1), 3-17. zimmerman bj 1998. developing self-fulfilling cycles of academic regulation: an analysis of exemplary instructional models. in d. h. schunk, & b. j. zimmerman, (eds). self-regulated learning. from teaching to self-reflective practice (pp.1–10). new york: guilford press. zimmerman bj (2002). becoming a self-regulated learner: an overview. theory into practice, 41(2), 64-70. zimmerman, b. j. (2008). investigating self-regulation and motivation: historical background, methodological developments, and future prospects. american educational research journal, 45(1), 166-183. assessing preschool teachers’ practices / adagideli, saraç & ader 437 appendix a dear teachers, we are conducting a research study on early year teachers' classroom practices. please read the following statements and indicate how frequently these teaching activities occur in your classroom regarding the 2013-2014 academic year. thank you. classroom practices 0= never 1=rarely 2=usually 3=always 1. i provide opportunities for my children to be aware of how they learn. 0 1 2 3 2. i provide opportunities for my children to be aware of their task preferences. 0 1 2 3 3. i provide opportunities for my children to develop awareness about their strengths and weaknesses in learning. 0 1 2 3 4. i draw my children’s attention to various strategies they can use for classroom tasks. 0 1 2 3 5. when i talk about strategies, i draw attention to similarities and differences among various strategies. 0 1 2 3 6. i provide opportunities for my children to develop their awareness that there are various types of tasks 0 1 2 3 7. i provide opportunities for my children to detect similarities and differences across tasks. 0 1 2 3 8. i encourage my children to identify what resources they will need to complete a task before they begin working. 0 1 2 3 9. i let my children make decisions about how to work. 0 1 2 3 10. while working on tasks, i encourage my children to stop and look back on what they did. 0 1 2 3 11. i teach my children how to check their progress. 0 1 2 3 12. i teach my children how to seek help appropriately. 0 1 2 3 13. i provide opportunities for my children to apply a previously learned strategy. 0 1 2 3 14. i want my children to evaluate the quality of their work. 0 1 2 3 15. i teach my children how to evaluate their learning. 0 1 2 3 16. i provide opportunities for my children to evaluate the quality of their peers’ performances. 0 1 2 3 17. i help my children to develop awareness about their emotional reactions while working on tasks. 0 1 2 3 18. i teach my children to monitor their friends’ emotional reactions while working on tasks. 0 1 2 3 19. i help my children to develop awareness about their motivational level regarding the task. 0 1 2 3 20. i teach my children various attention focusing strategies. 0 1 2 3 21. i teach my children how to resist distraction. 0 1 2 3 international electronic journal of elementary education vol.7, issue 3, 423-440, 2015 438 appendix b değerli öğretmenim, okulöncesi öğretmenlerinin öğrencilerine sağlayabildikleri öğrenme-öğretme ortamlarının niteliğini belirlemek amacıyla bir çalışma yürütmekteyiz. bu çalışma kapsamında sizden iki ölçek doldurmanızı rica ediyoruz. her iki ölçeği de 2013-2014 öğretim yılına ait deneyimlerinizi göz önünde bulundurarak cevaplayınız. katkınız için teşekkür ederiz. sinif i̇çi̇ etki̇nli̇kler 0= hiçbir zaman 1=nadiren 2=sık sık 3=her zaman 1. öğrencilerime, nasıl öğrendiklerini fark etmeleri için fırsatlar sunarım. 0 1 2 3 2. öğrencilerime, ne tür etkinlikleri tercih ettiklerini anlamaları için fırsatlar sunarım. 0 1 2 3 3. öğrencilerime, öğrenmedeki güçlü ve zayıf yönlerinin farkında olmaları için fırsatlar sunarım. 0 1 2 3 4. öğrencilerime, etkinliklerde kullanabilecekleri farklı farklı yöntemlerin olduğuna dikkat çekerim. 0 1 2 3 5. bir etkinlik için kullanılabilecek çeşitli yöntemler arasındaki farklılık ve benzerliklere dikkat çekerim. 0 1 2 3 6. öğrencilerime, farklı etkinlik türleri olduğunun farkına varabilmeleri için fırsatlar sunarım. 0 1 2 3 7. öğrencilerime etkinlik türleri arasındaki farklılık ve benzerlikleri görmeleri için fırsatlar sunarım. 0 1 2 3 8. öğrencilerimi, bir etkinliğe başlamadan önce etkinlikte ihtiyaç duyacakları materyalleri belirlemeye teşvik ederim. 0 1 2 3 9. öğrencilerimin, nasıl çalışacaklarına kendilerinin karar vermelerine izin veririm. 0 1 2 3 10. öğrencilerimi, bir etkinlik üzerinde çalışırken geriye dönüp yaptıklarına bakmaya teşvik ederim. 0 1 2 3 11. öğrencilerime, bir etkinlik üzerinde çalışırken kendi ilerlemelerini nasıl kontrol edeceklerini öğretirim. 0 1 2 3 12. öğrencilerime, ne zaman ve ne şekilde yardım istemeleri gerektiğini öğretirim. 0 1 2 3 13. öğrencilerime, önceden öğrendikleri yöntemleri kullanmaları için fırsatlar sunarım. 0 1 2 3 14. öğrencilerimden, kendi öğrenmelerini değerlendirmelerini isterim. 0 1 2 3 15. öğrencilerime, öğrenmelerini nasıl değerlendireceklerini öğretirim. 0 1 2 3 16. öğrencilerime arkadaşlarının performanslarını değerlendirmeleri için fırsatlar sunarım. 0 1 2 3 17. öğrencilerime, etkinlikler üzerinde çalışırken verdikleri duygusal tepkilerin farkında olmaları için yardımcı olurum. 0 1 2 3 18. öğrencilerime, arkadaşlarının etkinlikler üzerinde çalışırken verdikleri duygusal tepkilerini izlemeyi öğretirim. 0 1 2 3 19. öğrencilerime, etkinliğe ilişkin motivasyon düzeylerinin farkında olmaları için yardımcı olurum. 0 1 2 3 20. öğrencilerime dikkat toplama yöntemlerini öğretirim. 0 1 2 3 21. öğrencilerime dikkat dağıtıcı şeyler karşısında nasıl direneceklerini öğretirim. 0 1 2 3 assessing preschool teachers’ practices / adagideli, saraç & ader 439 appendix c: the discrimination of items and the item-total correlations item-total correlation discrimination of items n r p t df p item1 159 .589 .000 -8.305 84 .000 item2 159 .543 .000 -7.682 84 .000 item3 159 .597 .000 -8.733 84 .000 item4 159 .601 .000 -9.796 84 .000 item5 159 .571 .000 -8.764 84 .000 item6 159 .593 .000 -8.688 84 .000 item7 159 .635 .000 -9.991 84 .000 item8 159 .563 .000 -8.085 84 .000 item9 159 .544 .000 -5.842 84 .000 item10 159 .640 .000 -8.751 84 .000 item11 159 .669 .000 -10.362 84 .000 item12 159 .554 .000 -8.032 84 .000 item13 159 .672 .000 -9.890 84 .000 item14 159 .552 .000 -8.303 84 .000 item15 159 .670 .000 -10.510 84 .000 item16 159 .515 .000 -6.598 84 .000 item17 159 .623 .000 -9.823 84 .000 item18 159 .557 .000 -7.345 84 .000 item19 159 .645 .000 -9.771 84 .000 item20 159 .677 .000 -10.856 84 .000 item21 159 .757 .000 -13.790 84 .000 international electronic journal of elementary education vol.7, issue 3, 423-440, 2015 440 www.iejee.com this page is intentionally left blank http://www.iejee.com/ international electronic journal of elementary education, 2014, 8(2), 175-178. introduction to the special issue: contemporary irish education kathy hall coláiste na hollscoile corcaigh, éire university college cork, ireland mary horgan coláiste na hollscoile corcaigh, éire university college cork, ireland this special issue is devoted to various aspects of education in the republic of ireland. we thank the general editors for the invitation to guest edit this special issue and for the opportunity to put this collection of papers together for the readership of iejee. all the papers assembled are based on very recently conducted research in ireland and none have been published already. all the authors have tailored their papers for an international audience and throughout have sought to explain ‘local’ terms and contexts, a feature that was emphasised in the invitation to submit a paper for the special issue and a point that was noted by the peer reviewers in their feedback. as such we hope that the articles included here have relevance for the wider educational research community. ireland has featured prominently in global affairs over recent decades in relation to its economics – in the 1990s and early 2000s for its unprecedented economic growth captured in the phrase ‘celtic tiger’ and more recently because of the its banking crisis and the rapid and radical move to austerity policies. although a pan european phenomenon, the impact on ireland of the recession has been especially severe, touching all aspects of social policy and practice, including education. in addition, the historical context, the values underlying practices, and the pressures and influences from the wider social, cultural and economic international sphere all contribute to how practice is experienced by key players such as teachers and learners. collectively, the papers in this volume offer a contemporary perspective on some key policies and practices in irish education. the first paper in the collection, jointly authored by t.j. o ceallaigh and aine ni dhonnabhain, is about the promotion of the irish language in the school system. starting from the premise that the language we choose to use on a daily basis conveys something of our identity as a nation and a people, and from the fact that irish as a language is unique to ireland, their paper critically analyses the complex and controversial relationship across the irish language, the state, its people, and the education system itself. they explain how the education system came to be tasked with ensuring the revival and health of the language. the paper is detailed, constructive and optimistic about what can be done to ensure the health of irish as a living language while recognising the very real barriers and challenges that have to be addressed. though specifically about the irish language, the paper can also be read as an historical and contemporary account of politics, policies and http://www.iejee.com/ 176 practices that extend beyond the language itself to what it means to be educated as an irish citizen. thus, the paper offers insights into what constitutes appropriate curriculum and assessment in irish schools along with the values underpinning these important dimensions of the education system. the next paper analyses an aspect of secondary education in ireland that is also unique internationally – the programme known as ‘transition year’ (ty). described as a ‘bridge that connects young people in the classroom with adult life and the world of work’, the author, joseph moynihan, takes us through the origin of, rationale for, and experience of ty in the education system. his own empirical research and that of others endorses ty as hugely successful in its impact on young people’s overall development, and in particular, on their subject choices towards the end of their secondary schooling, what they study at third level, and their subsequent career choices. as he notes in the paper, it has even been recognised in the popular press (not typically prone to reporting ‘good news’ stories in education) as an outstanding feature of the irish education system. understandably the author expresses surprise that this practice has not been taken up by other countries, given its success in ireland. it is likely that much of its success is down to its close fit with research about how people learn, especially its fit with sociocultural learning theory and activity theory, as reflected in the work of engestrom at the university of helsinki. the next two papers are about ‘voice’ and its role in policy and research. the first, by domhnall fleming, theorises and problematises the notion of student voice in irish education and in post-primary schooling in particular. he distinguishes between the emphasis on student voice in the interests of rights and democracy and the emphasis on student voice in the interests of accountability and school improvement. this is a crucial and helpful distinction as it enables the author get at the heart of how policies geared to the enhancement of student voice actually function in practice. in reviewing international literature, he, inter alia, makes links with national policy in ireland. of particular note in the paper is the author’s analysis of the origin and role of school councils which were designed to inject greater democracy into schools from the perspective of students and to prepare students for their roles as active, participatory citizens. the empirical case study of the functioning of three school councils highlights the limitations of that construct and initiative as a way of empowering students as citizens. in sum his paper calls for ‘a more person-centred dialogic interaction within an inclusive classroom’. the next paper on voice takes a different angle. here the context is the ‘researcher voice’. it is noteworthy that over the last decade in ireland there has been strong growth in the study of educational research as a process. this has been at least partially facilitated by the movement to ‘cohort’ doctoral programmes in universities where groups of students come together to study and share research and perspectives on matters educational. most of the papers in this special issue are the fruits of such research programmes. the author of this paper on researcher voice, dan o’sullivan, offers a very interesting ‘telling case’ for not only do we get some insights into the concerns of newly qualified teachers in their first year of teaching but we get a problematizing of the researcher as instrument in the research process seeking to understand the transformations in identity accompanying induction into the teaching profession. beginning with a brief but revealing account of the author’s own induction into professional life, the paper explores the notion of researcher voice, unpacking its layers to reveal ‘authoritative’, ‘supportive’ and ‘interactive’ stances, and how they collide and overlap in the teasing out of the data collection and analysis process. over the last several decades research on leadership has directed us to its distributed nature and challenged us to question the more traditional notion of the ‘good’ leader as always some charismatic, powerful figure who leads by virtue of formal position or introduction / hall & morgan personality or exclusive claim to power. an exploration of the ‘why’, ‘what’, and ‘how’ of leadership in irish schools is offered by margaret o’donovan in the next paper. recognising the changed economic reality where school leadership is characterized by increased workload and an ever-expanding role-definition, the paper describes and analyses how school leaders (re)construct a form of leadership in times of austerity along with the associated challenges. interestingly, the author’s evidence shows how the concept of distributed leadership does not explicitly form part of the discourse of practitioners, posing challenges for the development of teacher-leadership capacity. like the first paper, it can also be read as illustrating the legacy of history on how irish schools are governed and led, and more particularly how the notion of ‘ethos’ is a highly relevant theme in the discourse of school leaders in ireland. the place of the arts in irish education has rarely been guaranteed or without controversy in the primary and post-primary curriculum. its value remains contested in practice even if the rhetoric is positive. despite the irish government’s endorsement of the arts in and out of school, siobhan dowling long’s paper highlights the lack of investment in the arts and the inadequate recognition by policy makers in ireland of the life-enhancing nature of the arts and their role in lifelong learning. the increasing emphasis on literacy and numeracy in both compulsory phases of education, along with the role of formal assessment, especially now in primary school, militate against the promotion of aesthetic experiences for learners. in post-primary school, the study of the arts is not obligatory and by the end of that phase only a minority of irish students have studied the arts in any depth. in contrast to the situation in finland where students get an indepth understanding and appreciation of their finnish culture and heritage, the author illustrates the paucity of the experience afforded the irish primary and post-primary student. the author concludes that the lack of investment in the arts is an area of grave concern. in recent years in ireland early childhood care and education has risen to prominence in the media, among politicians and in the public domain generally, while concern about standards in some early years services is a particular focus. as rosaleen murphy in the next paper explains, services for the youngest children (i.e. before age 4) are largely the responsibility of the department of health while children in the formal school system are the responsibility of the department of education and skills. crucially, while pre-school care and education is seen as the responsibility of parents, and is among the most expensive in europe, primary education is state-funded. the paper argues that this split is highly problematic for conditions of work of those who work with the youngest children, for investment in that sector, and for the quality of service in the very earliest years of a child’s life when the foundations of learning are laid. the author calls for parity of esteem to be given to the pre-school years and challenges the current policy and practice context in ireland where the divergence between both is untenable given the significance of the early years for the lifelong wellbeing of the individual and society more broadly. the last two papers in this special issue offer detailed accounts of aspects of irish education that to varying degrees point to a growing neoliberalisation in education policy and practice, thus illuminating the impact of global trends that privilege an economic imperative over other purposes of human development in our school system. drawing on stephen ball’s theorization of policy, kevin cahill’s paper offers a critical policy analysis of intersections between social class inequality and education in twenty-first century ireland. he takes several key irish policy documents (including the important document, ‘an action plan for educational inclusion’) that inform the practice of education and he subjects them to critical interrogation and review. his argument is that the refusal in irish 178 policy and legislative discourses to name social class as a significant issue in understanding and addressing educational opportunity, experience and outcomes is a fundamental absence that militates against equality. he argues too that social class inequality is not just an issue for education to address but is an equity and social justice issue especially in a country that so explicitly claims democracy and inclusion as fundamental tenets. the final paper is a critical engagement with the discourse of reform in ireland with particular reference to the subject, mathematics. in this paper liz kirwan meticulously tracks the ‘genetic imprint’ of pisa and its instrumentalist view of mathematics on the development of a new mathematics syllabus for schools (project maths). she shows how the irish government in reforming the mathematics education policy was strongly influenced by the conceptual framework of pisa and thus argues that project maths does indeed bear the genetic imprint of pisa. however, she goes on to show how project maths is not a mini-pisa since it still retains much of the more traditional content of abstract, symbolic mathematics. she argues that the twin emphases of realistic mathematics on the one hand and the more abstract, traditional emphasis on the other is problematic, all the more problematic as the emphasis on real life mathematics is abundantly evident in the high stakes assessment system. one might anticipate that the power of the assessment may well reign supreme over other official endorsements of particular content and particular pedagogies. will teachers teach to the test? further research will have to answer this question. overall, the papers assembled pinpoint some of the issues that are challenging the irish education system. national issues, such as the irish language, and international issues, such as the impact of pisa, combine to give some insights into globalization in action in ireland. the search for a good fit across multiple local/national heritages and international pressures means that irish education as a system is not fixed but is changing to try to keep pace with international developments and national values, especially as society changes and becomes more diverse. it is clear from this small sample of papers that gaps remain between policies and practices and that greater attention needs to focus on those gaps to ensure a fairer educational experience for all citizens of the state. kathy hall is professor of education and head of school in university college cork ireland. she previously held professorships in the uk, at the open university and leeds metropolitan university, and she was a senior lecturer in canterbury christ church university. her research seeks to develop sociocultural understanding of learning and pedagogy and she has published widely in these areas. recent books include networks of mind: learning, culture, neuroscience (with curtin and rutherford) published by routledge, 2015 and research methods for pedagogy (with nind and curtin) published by bloomfield january, 2016. mary horgan is a senior lecturer in the school of education university college cork where she played an active role in developing the first degree programme in ireland in early childhood studies in 1995. she is patron of omep irl. and was vice chair of the national council for curriculum and assessment’s early years technical working group from which aistear (2009) the irish early years curriculum framework emerged. her research interests lie in childhood education with particular reference to curriculum, voice, agency and identity. international electronic journal of elementary education, 2015, 7(3), 355-370. issn:1307-9298 copyright © iejee www.iejee.com some reflections from pre-service elementary teachers’ practice teaching on the area of understanding data in the math-teaching course1 özlem doğan temur  dumlupınar university, türkiye serap akbaba dağ dumlupınar university, türkiye sedat turgut dumlupınar university, türkiye received: february 2015 / revised: may 2015 / accepted: may 2015 abstract with developing technology statistical information and data sources become a very important issues and from primary school it has become necessary to gain the skills for making interpreting and making sense of data. these skills consist of collecting information, arrangement and analysis of collected data and the interpretation of the results. the duty of guiding students in their process of making statistical information meaningful falls upon teachers. this study, whose aim was to investigate prepared course content for sub-learning area in grade 1-4 math course and obtained experiences by pre-service elementary teachers in the schools they went as a part of teaching practice course, was conducted with nine fourth-year students attending an undergraduate program of elementary teaching in a state university during 2013-2014 academic year. pre-service teachers were each asked to prepare and conduct a lesson plan suitable for the lesson outcomes and the level of the classes that they were to teach. their applications were assessed by semistructured observation form about data teaching developed by the researchers. it was observed that pre-service teachers could not reflect given lesson outcomes on the topic of data to the lessons they prepared to teach during their teaching practice. in the implementations, it was noted that pre-service teachers could not effectively include students in both collecting and arrangement as well as interpretation processes of the information and that they taught in teacher-centered manner although they prepared a correct activity. it was also noted that pre-service teachers could not well enough differentiate category and concept of variable in table and graph activities. keywords: data instruction, pre-service teacher, teaching practice 1 this paper was presented as the proceedings of the 13th national symposium of elementary teacher education  özlem doğan temur, dumlupınar university, kutahya, turkey. phone: +90 (274) 422-4602. email: ozlem.dtemur@dpu.edu.tr http://www.iejee.com/ international electronic journal of elementary education vol.7, issue 3, 355-370, 2015 356 introduction math is not only a field about numbers and calculations but also a system that finds a place in many circumstances encountered in daily life. developing technology offers a very rich visual world mankind. maybe, primary school students are the group that spends the most effort trying to explain that visual world. it is because seeing and explaining many elements like figures, numbers, data, words etc. together is a process that have just been learnt. primary school students will come across graphs and data not only in math lessons but also in science and social sciences and try to set some relationships. therefore, the topic of data and its teaching is very important from an elementary teachers’ point of view. topıc of a graph in primary school provides an introduction to statistics, another branch of mathematics. statistics, to answer a question, includes important skills like collecting data, summarizing data, making sense of data, interpreting, concluding for the future and deciding. acquisition of the skills of data collection, data summation, making sense of data and inference are established as goals in primary schools (olkun & toluk ucar, 2004). primary school students should be active in the problem-solving processes (forming questions and their answers, collecting data and its presentation, data analysis, and data inferences) about data. teachers should encourage students to collect data and interpret them. studying questions like “let’s assume that,” and “what if…happens,” provides students to better define the data analysis period and its nature (franklin & mewborn 2008). real world is full of data and its sources. children need to ask and answer real world questions like ‘what?’, ‘how?’, ‘when?’, ‘where?’, ‘who?’, ‘why?’ to collect data, to organize the data they collected, and to interpret. therefore, it could be said that data analysis has more significance than in just forming and reading data. children need to make judgments to collect data. when, at the first stage, facing prompting questions asked by their teachers, children feel a need to collect data. when a teacher says, “i think, plain ice cream would be the most loved ice cream in this class,” the children will want to find out what kind of ice cream is the most loved one (cathart, pothier, vance & bezuk, 2006). primary school students need to have developed some set of skills to find out the answers of the questions that they are curious about. acquisition of these sets of basic skills since preschool constitutes an important section of the primary and secondary school mathematics education programs. children should come across, since preschool, activities that are aimed to develop the skills of ranking, sequencing, and analyzing. they label the properties of the objects by using characterizations like red, hot and circles. this kind of activities increases the skills of classification and comparison of groups with similarities and differences (van de walle, 2010). the first experiences of the students in the topic of data are their encounter with the objects whose properties are easily noticed. these kinds of objects and qualification cards are easy to produce or to obtain by the teachers, too. some students start by classifying only one property, some can classify according to different properties, too. teachers should help their students think in different ways when the students classify objects. venn diagrams are one of the ways that facilitates students’ job in classifying multiple properties of the objects. overlapping circles simplify classifying multiple properties of the objects. objects that fall outside of the categories will stay outside of the circles (bahr & garcia, 2010). primary school mathematics education program sees forming problems that could be answered by table or by summarizing in the form of graphs as fundamental purpose of teaching the topic of data. giving precedence to activities of data, data analysis, simple classification for understanding statistics, comparison, and counting, the mathematics teaching program emphasizes that students form questions, which are meaningful in and some reflections from pre-service elementary teachers / doğan-temur, akbaba-dağ & turgut 357 of themselves and determine the answers given to those questions, that the students should be directed to organize given answers; and then, that students can present data they collected in both tables and graphs. examining the data teaching period in the mathematics teaching program, it is seen that reading tables and basic skills about the topic of data are included for first grade; object graphs and table formation in the second grade; figure charts, in which each picture represents one object only or a picture multiple objects in the third grade; formation of bar graphs, organization of data in a table or a graph and data analysis in the fourth grade. the program states that it needs to proceed from pictures to symbols or to more tangible presentations over time [mathematics (1-5 grades) curricula, 2009]. it is attention drawing that the program emphasizes the skills advancing from the abstract to the concrete. the approach of the mathematics teaching program on the topic of data can be seen as an approach that featuring the development of the children’s thinking about statistics. children’s thinking in the topic of statistics can be investigated on four levels. the first level of those is to define data, the second to organize data, the third to present data, and the fourth to analyze and interpret data (jones, thornton, langrall, mooney, perry & putt, 2000). stating that statistical thinking develops in four periods, biggs and collis (1991) define these periods as subjective, transitional, quantitative and analytical. while focusing on their own personal data and trying to make relationships with given data, first-level thinkers are not yet ready to draw conclusions, second-level thinkers start to notice importance of quantitative thinking and are not completely successful, even if they are able to use numbers when performing measurements and trying to make sense of data. the third-level thinkers start to use quantitative thinking, foundation of statistical reasoning, start to acquire the concepts about measures of central tendency, and start to, occasionally though, make relationships between the data and context in which the data are found. the fourth-level thinkers can find the data in their context and make relationships between the data and their context. when investigating the presentations children use in their demonstrations, five common forms, parallel to development of statistical thinking, can be said to be found. these can be named as dynamic, pictorial, iconic, written and symbolic presentations. while dynamic representations are those with children’s movements or acting which children perform live with the object itself, pictorial representations are those formed with the images of real things. iconic presentations can be considered on the basis of using a sign for every counted unit. iconic presentations can be considered as tables, but usages of tables are not the only way. it could be said that written presentations like words and sentences that we encounter everywhere all times are also among iconic representations. children mostly prefer presenting data in written form, for instance, like two grapes as two pieces, four grapes as four pieces. usages of standard presentations of numbers and signs are now the best examples for iconic presentations (carruthers & worthington, 2006). acquisition of basic skills on the topic of data will be the foundation to table and graph studies that they encounter in primary schools. tables are the first tools to be used for organizing data for graphical studies. tables like frequency tables can be used for organizing data before graph formation activities. it should be primarily decided that what table is needed before graph presentation ( friel, curcio & bright, 2001). tables are very handy techniques for recording the data. children should be offered opportunities so that they discover ways of organizing data with table activities (cathart, pothier, vance & bezuk, 2006). international electronic journal of elementary education vol.7, issue 3, 355-370, 2015 358 children’s graph experiences generally advance in four stages. the first period that starts with object graphs is the concrete stage; the one that which object and figure chartss are used together is concrete figure stage; following figure chart studies, the one in which symbolic figures are used instead of real picture of the object is figure-abstract stage; the last one, in which figures in tables and graphs are matched with multiple situations or objects rather than single situation or object is abstract stage (cathart, pothier, vance & bezuk, 2006). graphs that are formed with real objects are the first stage of graph studies. object graph activities should first be performed by using two objects and later more than two objects or situations. after performing these activities, graph studies can be done on the structures formed from lines and columns with the similar structures that can be used instead of real objects (charlesworth, 2000). children use some basic skills like counting, comparing, pairing, and classifying. for that reason to give graph activities a place, since preschool, is very important (aktaş, 2006). the number of categories to be compared is, too, as important as the number of object used to form a graph in object graph activities. for example, if the teacher makes his students face a graph about kinds of ice-cream that they like the most, he should primarily pay attention to choose graphs with two categories. this point, too, needs attention for figure chart activities following the activities conducted with object graphs and with two or three categories. kinds of graphs to be used and number of categories are hierarchical elements that teachers should pay attention about in children’s studies of reading and organizing graphs (baratta-lorton 1995; cited in bahr & garcia, 2010). this hierarchy is shown in terms of kinds and numbers of groups. table 1. kinds of graphs and number of groups compared kind of graph number of groups compared object graph two categories are compared three categories are compared figure chart two categories are compared three categories are compared object graph four categories are compared figure chart four categories are compared symbolic graph two categories are compared three categories are compared four categories are compared being more tangible, figure charts could be a proper starting point for introducing graphs to children. figures, instead of numbers, are shown in graphic representation. it is used to compare the sizes of various categories. each figure used shows only one parameter or one group. a bar graph is a more concrete form of figure chart because numbers, now, start being used instead of figures. bar graphs, too, are used for comparison as figure charts are (olkun & toluk uçar, 2004). at the end of a study with 121 fourth grade and 127 sixth grade students on reading twelve different bar graphs, data interpretation, and estimation, pereiramendoza and mellor (1991) observed that 95 percent of fourth grade students and 98 percent of sixth grade students succeeded in reading data from bar graph. it is very important, from the perspective of quality education, for teachers to know about the points that children have difficulty and make mistakes in the data teaching. these mistakes could be named as misreading, misunderstanding, carrying out incorrect math operations in translation procedures used for tables and graphs such as doing some reflections from pre-service elementary teachers / doğan-temur, akbaba-dağ & turgut 359 multiplication instead of addition, making mistakes during operations and misinterpretations in problems related with data (ryan & williams, 2007). koparan and güven (2013) emphasizes that consistency in statistical thinking, relating, multidimensional thinking and presentation increase towards higher grades in their study in which they aim to define differences between class levels of students of various grades. from that it could be said that statistical thinking is related to cognitive development. on the other hand, it is seen that some sixth grade students show advance thinking in statistical thinking. therefore, value of experiences students have had in development of statistical thinking skills is supported by the finding of the study. also, one of the findings of the study is that seventh and eighth grade students could generally read tables, figures, and graphical presentation and recognized pieces of data but that almost half of the sixth grade students generally appeared at the first level. this situation brings it to mind that sixth grade students have had some problems in defining the data. it could be said that a successful data teaching period is possible by designing instructional environment and by instructing teachers who, depending on scientific sources, can develop this period. in primary schools, the foundation of the data analysis and its interpretation stands on the activities that are directed to students’ being able to read presentations and being able to be aware of presentations. from this point of view, in the study, it is aimed to describe pre-service elementary teachers’ experiences about data teaching process. method this research is a descriptive study that aims to investigate lesson contents about first through fourth grade math lessons prepared by pre-service elementary teachers and experiences they obtained in primary schools they went as a part of teaching practice. descriptive studies are researches in which obtained data are organized, interpreted, and presented to the readers and in which physical conditions or groups are described (yıldırım & şimşek, 2008; büyüköztürk, çakmak, akgün, karadeniz & demirel, 2010). descriptions of pre-service teacher experiences are targeted in the study. research group table 2. teaching practice groups, practice classes for groups, sub-learning areas and lesson outcomes for the topic of data pre-service teachers and practice classless sub-learning area outcomes 1a-1b table the learners will be able to read tables 2a-2b object graph 1. the learners will be able to collect data about a problem and form a figure chart. 2. the learners will be able to interpret object graph. table learner will be able to organize the data into a table. 3a-3b figure chart 1. the learners will be able to collect data about a problem. 2. the learner will be able to create a figure chart. 3. the learner will be able to interpret a figure chart. table the learners will be able to create tally and frequency tables. 4a-4b-4c bar graph 1. the learners will be able to create a bar graph. 2. the learners will be able to interpret a bar graph. probability the learners will be able to use words indicating probability in proper sentences. nine fourth-year students attending a state university during the academic year 20132014 constitute the research of the study. these students were grouped so that they would practice on data-teaching topic in all classes of the primary schools they went. pre international electronic journal of elementary education vol.7, issue 3, 355-370, 2015 360 service teachers were coded as a, b, and c according to the grade they practiced. these groups, the classes in which groups practiced, sub-learning areas of the topic of data and its outcomes are shown in table 2. colectıng data and theır analysıs data collecting tools are consisted of semi-structured observation form prepared by the researches and lesson plans prepared by the pre-service teachers. the observation form about data teaching is composed of four sub-sections. these sections are named as research question and the data (9 items), table (8 items), figure and bar graph (10 items), and supplementary explanations (6 items). the sections have contributed to create themes of the descriptive analysis. the themes have been dealt with under four titles as following and the findings have been presented in the way that each pre-service teachers’ actions will be in these themes’ coverage. 1) research question and the data, 2) table and graph, 3) supply usage 4) the points at which difficulties have been experienced during the application or that explained incorrectly and incompletely in the process, pre-service teachers were observed during one-hour-lesson by a researcher, and it is recorded by means of application period observation forms. at the end of the application period, lesson plans prepared by the pre-services teachers were collected from them to analyze. during the data analysis, observed experiences of each pre-service teacher were separately coded according themes defined in the observation form. all documents were analyzed by three researches separately for the reliability of the coding and after all researches reanalyzed the documents for the codes mismatched with each other, a consensus about the coding was reached by the researches. direct quote were utilized to ensure the external reliability of the study. findings pre-service teacher: 1/a research question and the data. doing his introduction of the lesson, the pre-service teacher had difficulty and failed to make a proper introduction. he had difficulty in drawing students’ attention by saying “i will explain tables. you have already learned it before, so i start right now”. he directly asked the research question to the students. instead of taking answers of the questions, “do you like colors?” and “what color do you like the most” from the students, he wrote four names of colors on the board and collected students’ answers into these categories. meanwhile, some children did not want to take part by saying “but i don’t like these colors”. after this activity, he asked, “what animals do you like the most?” and he brought four pıeces of cardboards with anımal pictures on them. passing out smiley faces to the children, he told them to stick them on to the opposite side of the animal they liked the most. table and graph. after passing out smiley faces to the children, he told them to stick smiley faces on the opposite side of the animal they liked the most. at this stage he encouraged students to collect data, but he drew the table himself. he did not make students do a table-reading study on the table drawn. supply usage. he passed out study papers about old macdonald’s farm. the study papers he had chosen had features of ready-made materials. they were photocopied material. the students had difficulty in following the material because they were first graders and some reflections from pre-service elementary teachers / doğan-temur, akbaba-dağ & turgut 361 because reading-writing activities were still in progress. study papers were not proper for first graders in terms of spacing and the font used. the points at which difficulties have been experienced during the application or that explained incorrectly and incompletely. the pre-service teacher had difficulty in keeping class order because the students were at first-grade level. he conducted activities out of the lesson outcomes of the topic of data. the students struggled to understand. in the study paper activity, ready-made materials were used and it was not proper for the level of the class. creating table activity was chosen instead of table-reading. he had difficulty in keeping the class in order during the activity of table creating because he did not include the students. he himself answered the questions on the study papers and then he asked the students to answer the questions individually. however, the students struggled to answer. it resulted in failure because the students did not know what and where to write. the lecturer of the teaching practice had to warn the pre-service teacher to go more slowly. pre-service teacher: 1/b research question and the data. he made an introduction to the lesson with a question gathered under three categories. he said to the students, “let’s find out who likes honey and milk in this class” and he did not pose it as the research question. he carried on the activity with questions like “who likes milk? who likes honey? how many of you like milk?” he did not do an activity like organizing the data and reflecting them to the tables. table and graph. a table activity was presented about seasons. the pre-service teacher showed a table named “students who like seasons” by means of an overhead projector. he asked the students to analyze the table and made an introduction to the lesson by asking, “what season do you like the most? what season are we in now?” afterwards, he carried on the activity with question like “according the table, what seasons are liked the most, what seasons are liked the least?” he used tables of maximum four categories. in the “my farm” activity, he presented a table with three categories by the overhead projector. supply usage. he presented activities as study papers and by overhead projectors. the tables that he used in the table-reading activities were correctly prepared. however, during his instruction on tables, the pre-service teacher carried on activities with routine questions rather than putting forward the relationships between the contexts of the data and tried to squeeze a lot of activity in one lesson. he could not execute the activities on time and the students did not want to take part in table-reading activities. the points at which difficulties have been experienced during the application or that explained incorrectly and incompletely. showing the pictures of lions, elephants and horses, he said, “these are animals living in my farm” one of the students said, “lions and elephants don’t live in farms, sir” he presented some table activities like “the most loved colors” and “the most loved animals”. he tried to fit all activities in one lesson. after the first 20 minutes, he lost control of the class. the students did not take part in the lesson. he asked proper questions about the tables but the students were lost in the lesson because he did not give any feedback and did not make any relationships between the contexts of the data. pre-service teacher: 2/a research question and the data. he drew a smiley face for each male and female student and asked the number of girls and boys in the class. asking, “i wonder what fruits are liked the most in this class”, he tried to state the fruits liked. both questions attracted the students’ attention. the children’s most loved fruits were gathered under six categories. these are apples, bananas, water melons, coconuts, strawberries and grapefruits. he international electronic journal of elementary education vol.7, issue 3, 355-370, 2015 362 ignored some preferred fruits. the students asked, “why weren’t ours picked?” the students raised their hands and stated the fruits they liked but he could not determine the number of fruits correctly because there were too many categories. the students did not collect data about the research question given. table and graph. during the determination of the most loved fruits, he did not show the data on the table. he asked the students some questions on the data, which were not recorded. his questions were directed to conducting operations on questions. the questions like “how many of you do you think likes grapefruit and strawberries?” confused the students. he created a figure chart, not an object graph, about the most loved fruits. he drew a picture of the every fruit that every student liked. the students did not take part in graph creating activities. he asked a few questions like, “what fruit is liked the most? what fruit is liked the least?” on the graph he created. he did not elaborate enough on the interpretation of the graphs. supply usage. no materials were used. the points at which difficulties have been experienced during the application or that explained incorrectly and incompletely. figure charts were used although object graphs were mentioned for learning outcomes of grades. he did not appoint any duties in collecting data and creating graphs. for that reason, along the lesson, students stayed inactive and he, himself, had difficulty to control the class. he did not do any activity for the lesson outcome: “the learners will be able to organize the data”. and, for that reason, the students struggled to make sense of and to organize the data. pre-service teacher: 2/b research question and the data. he made an introduction to the lesson with the question: “what subject do you like the most?” he himself wrote the names of the lessons without having students’ answers. he asked them to name the ones they liked the most among math, social science and turkish. stating that the students confuse due to many categories, he limited, as a result, the categories to three. yet some students persistently stated different names of the subjects too. he made a forecast graph for three days for the same reason. the students reacted by saying, “a week has seven days, not three. why are we doing it like that, sir?” table and graph. the students themselves did not collect the data and did not organize it into the tables. the pre-service teacher first drew the table for the most loved subject and, about the table he drew he asked the questions, “what subjects are the most loved? and what subject are the least liked?” he did not ask any question about the table he drew about the forecast. he wanted the students only to take notes in their notebooks. he never dwelt on concepts of variable and category on the table. the pre-service teacher brought blue, green, and blue beads. he asked the students to fix the beads with the color they like to the stick they belonged. some of the students said they did not like these colors and they did not want to fix them to the sticks. yet, the preservice teacher stated that they had to choose a color. the students mostly chose blue and there were no more blue beads to fix. due to being not well-structured, the activity failed. nevertheless, it was the only activity reflecting the outcomes of the figure charts and allowed the students to do classification. supply usage. the beads and the sticks to which beads were fixed were used. the points at which difficulties have been experienced during the application explained incorrectly and incompletely. the pre-service teacher obliged the students to choose three some reflections from pre-service elementary teachers / doğan-temur, akbaba-dağ & turgut 363 categories and to study them. he did not give any opportunity for them to collect data and to submit their proposal for research question. he did not allow for the interpretation of the graphs by using directing questions. in the activities with research question given, the students thought they had to determine the categories first because the determination of category activities was done with giving research questions and not giving an opportunity to answer. pre-service teacher: 3/a research question and the data. the pre-service teacher made an introduction by putting the table he had prepared beforehand the lesson on the board. instead of making the students notice the research questions the table reflected, he started the lesson by saying, “let’s see what animals are there in old macdonald’s farm”. the students did no activity about data collecting. table and graph. the pre-service teacher asked the students, “who drinks milk every day” and wrote the names of six students on the board. he showed them how to show with the tally method as an example. however, the students had difficulties because reading tallytable activities had not been performed. correcting the wrong presentation, the preservice teacher, himself, showed the correct presentations. some students wanted to correct, but the pre-service teacher did not give them permission to speak. tally presentations were emphasized, not the tally chart. he affixed some of the animal pictures on the cardboard he had brought. stating that each picture he had affixed showed three pictures, he asked, “what do you seen on the board?” however, the students struggled to answer because the question was not clear. later, displacing circles with the animal pictures, he followed a way from the concrete to the abstract. changing the number of both animals and circles, he asked question like, “how many animals….? what is the most numerous animal? what is the least numerous animal?” and gave some students permission to answer. he called some students to the board and handed them some beads. he said every bead represented five beads and he asked the class to determine how many beads the students at the board had. two students took part in the lesson and could calculate it, but the rest of the class could not work it out. supply usage. a pre-prepared cardboard on which animal pictures were to be stuck with the name of old macdonald’s farm were used as the material. the points at which difficulties have been experienced during the application explained incorrectly and incompletely. the pre-service teacher asked them to make a bar graph with the beads they had in their hand. not having created any figure chart well enough and not having seen bar graphs before, the students did not took part in the activity and the teacher dropped the activity. presenting data on the tables was not elaborated well enough. he tried to cover all four lesson outcomes about tables and graphs in a single lesson. the students participated in the table creating activities by using their preknowledge. activities about the outcomes were not covered completely. pre-service teacher: 3/b research question and the data. the pre-service teacher asked the questions: “what is tally chart? what is figure chart?” he wanted to check their pre-knowledge but the students had not seen figure charts and tally charts before. they could not answer. he showed the students a zoo and asked them to find out the number of animals. thus, the students played a role in the process of collecting data. however, it was not stated as the research question. table and graph. moving from the visuals, the students worked out the number of animals. the pre-service teacher explained what the tally is and gave the students some international electronic journal of elementary education vol.7, issue 3, 355-370, 2015 364 information about the presentation. he asked the students to write the names of the animals and to show the number of animals next to animal names as tally. the students successfully completed the tally activities. following the tally chart activities, the preservice teacher did not ask any question. the students did not name the table they created. moving from the table the students created about the zoo, he asked the students to create a figure chart as if the students had known the topic of figure chart. the students did not know what to do. then the pre-service teacher, personally, drew it on the board. he did not give any information about graphs. when copying the table on the board to their notebooks, most of the students did not write the title of the table, not even record the names of the variables. showing the figure chart of the liked football teams he prepared and asking questions on the graph, he did some table-reading activities. he proceeded graph reading activities with question like “what football team is the most supported? what football team is the least supported?” each figure in the graph represented two figures. however, the pre-service teacher forgot to share this with the students. supply usage. a zoo picture and a figure chart named “supported football teams” were use as materials. the points at which difficulties have been experienced during the application explained incorrectly and incompletely. the students behaved as if they did not know figure and tally charts. the pre-service teacher did not dwell on the concept of category. he started figure chart creating activities without conducting any activities like collecting data and creating graphs activities. because he did not sufficiently emphasize the properties that graphs should bear, the students forgot the details such as naming the created tables and graphs etc. the visual of zoo was not clear. the students counted differently because it was a photocopy. pre-service teacher: 4/a research question and the data. the pre-service teacher asked, “what comes to our mind when we hear the word graph?” the students replied, “tally charts, columns, bars, lines, figure charts”. the pre-service teacher said “we use tally charts when creating graphs”. he passed out candies with three different colors. without stating the research question, he said, “let’s count the candies in terms of their colors”. table and graph. making the students count the candy in terms of colors, he himself drew a tally chart on the board. calling a student to the board, he asked to the student to write the number of pieces of candy next to the colors according to the tally graph. created tables were named. tally charts and frequency tables were created. the pre-service teacher made an introduction to graph creating activities without conducting any graph reading activities. he asked about the hobbies of the students. he wrote the hobbies of the students under five categories on the board. the pre-service teacher drew a bar graph in both horizontal and perpendicular appearance and asked the students to interpret the data according to the questions he would ask from the graph. he did graph-interpretation activities by asking questions like “how many of you are attending guitar course? what hobby is the most preferred one? what hobby is the least preferred one?” supply usage. he did not use any material. from a ready-made source, photocopied study papers were passed out to the students. the points at which difficulties have been experienced during the application explained incorrectly and incompletely. the pre-service teacher wrote how many students there are in each category opposite to the hobbies in the graph about the hobbies he created. axis names were not written. the title of the graph was not written either. a histogram was some reflections from pre-service elementary teachers / doğan-temur, akbaba-dağ & turgut 365 drawn instead of bar graph. saying, “this is incorrect”, he corrected it. one of the students said, “let’s unite the columns, sir”. the pre-service teacher replied, “next time we will do it that way”. copying from the board to their notebooks, the children, too, made histogramlike drawings. pre-service teacher: 4/b research question and the data. the pre-service teacher asked, “how many of you are attending weekend courses arranged in your school?” the students gave answers; he himself recorded the data and created a table. he asked the students, “on what else subject shall we collect data?” the students replied, “let it be about games and supported football teams”, but the pre-service teacher replied, “no not them, let’s collect data about the months you were born in”. it is because he had prepared for that. the pre-service teacher had difficulty in determining a research question as an in-class activity. table and graph. collecting the data, himself, about the children attending weekend courses, he drew the tally chart and the frequency table himself. afterward, he collected the data about the months the students were born in by asking them to raise their hand. one of the students wanted to draw the table. the student themselves drew the tally chart and the frequency table in accordance with the data. the pre-service teacher asked some question for table-reading. he asked the students, “in what month were the most of your friends born in?” there were four mounts in which the most births took place. one of the students said, “we haven’t named the tables”. the pre-service teacher named the tables. during table-reading, the students had difficulties because the categories on the table had not been named and because there were too many categories on the tables about the months in which the students were born in. yet, they correctly answered the questions the pre-service teacher asked. bar graph activities were not done for those tables. the pre-service teacher had prepared a bar graph with the name “the most loved colors”. he presented it by the help of an overhead projector. he had prepared a graph with four categories. he asked questions like “what is the most loved color? what is the least liked color? how many times bigger is the number of those who like yellow color than the number of those who like black color?” he explained the relationship between lines and columns on the bar graph. the students had difficulty in copying it from the board to their notebooks. they made some incorrect drawings. there were some students who drew bar graphs like a histogram. a table was drawn with the help of the data collected from the students about the football teams supported in the class. one of the students went to the board and drew a bar graph. he correctly set the relation between the columns and the lines. the students sitting in their desks examined what their friends did on the board and copied it in their notebooks. he asked, “what football teams are supported by the most and the least people?” he did not ask anymore questions directed for interpretation. the graphs were drawn horizontally. supply usage. he presented the activity named “liked color” by the help of a overhead projector. the points at which difficulties have been experienced during the application explained incorrectly and incompletely. the pre-service teacher drew a table named busra’s academic success with four categories and he asked the students about the grades that busra had got from the subjects. then he said, “i will teach you the mean”. he calculated the mean with the grades that büsra had earned on the subjects. he explained how to work out the mean. he said, “we add the grades up and divide it by four”. he told the student to calculate by changing the grades. the students struggled to calculate. he called a student to the board and explained how to work it out step by step. one of the students asked, “what is it for?” he said, “you will learn later”. international electronic journal of elementary education vol.7, issue 3, 355-370, 2015 366 pre-service teacher: 4/c research question and the data. the pre-service teacher asked the students, “what should be eaten to get some vitamins in cold weathers?” the students replied, “fruits and vegetables”. the pre-service teacher asked “who likes what fruits”. after that, he showed the graph he prepared about the favorite fruits. table and graph. he asked the students, “who regularly brushes their teeth?” then he said, “let’s make a table about it”. one of the students asked, “did you ask as days?” the pre-service teacher replied, “yes”. one the students added, “let it be weekly; it could be difficult to show on the table otherwise”. the pre-service teacher said, “okay”. the students told how many times they brush their teeth and the pre-service teacher himself drew the tally chart and the frequency table according to the data. he asked, “who are those who brush the teeth regularly with respect to the table?” the students could not decide how to answer. the students asked the pre-service teacher, “how will we know if it is done regularly?” before the bar graph activity, the pre-service teacher showed the students the figure chart he had prepared about the most loved fruits. he added that each fruit represents three fruits. then he started graph reading activities. he asked few questions like “what fruit is the most loved and the least liked ones? how many more fruits should banana lovers eat to catch apple lovers?” the per-service teacher took a different number of beads, in five colors, in his hands. he said, “we will draw a bar graph with respect to the number of these beads”. he wrote on the board that each bead represents three numbers. the pre-service teacher drew a bar graph considering the number of beads in his hands. he explained how to draw the graph. he asked the students to take some beads in their hands and to draw a bar graph with respect to the number of the beads. the students struggled to align the numbers on the perpendicular axis because the teacher had not explained the relationship between lines and columns in the graph. the pre-service teacher called a student to the board and instructed him to draw the graph. he emphasized only the points at which the student at the board had difficulty in. he did not check the graph of the students who were sitting. supply usage. the most loved fruits board prepared by the pre-service teacher was used as the material. the points at which difficulties have been experienced during the application explained incorrectly and incompletely. not naming the tables and graphs, the pre-service teacher sometimes forgot to name perpendicular and horizontal axes. most of the time, the students reminded him. instead of encouraging the students to create graphs, he asked them to answer the questions for which they can work out in their head on ready-made tables. conclusion and discussion when analyzing the lessons pre-services teachers prepared and the data about application samples, it could be said that pre-service teachers are not knowledgeable about the topic of data teaching. during the lesson presentations, it was observed that the pre-service teachers struggled, got bored, and asked for help from the class teacher. pre-services teachers have difficulty in realizing an effective math period with the increase in their level of concern (swars, daane, & giesen, 2006). it has been noticed that especially preservice teachers who did teaching practice in grade 3 and grade 4 were sometimes criticized and their mistakes were corrected, again, by the students. for example, when one of the students asked, “what is it for?” the teacher coded as 4b could not answer the question and said, “you will learn later”. when analyzing the pre-services teacher’s some reflections from pre-service elementary teachers / doğan-temur, akbaba-dağ & turgut 367 practice teaching, one of the attention-drawing points is that it has been observed that the pre-service teachers dealt with outcomes that were outside of the class outcomes. although the arithmetic mean was not a topic of the grade-four level, the pre-service teacher presented an example about calculating mean and pre-service teacher 4/b, directly giving the rules for the solution of question by saying, “add up the grades and divide it by four”, carried out the solution of the problem. it was seen that pre-service teachers are not completely knowledgeable about the concepts of category and variable in table and graph activities. in some examples, too many categories, over the class level, for the variable were determined and for that reason the students had difficulty in organizing the data. again, the pre-service teachers did not name the tables, graphs and the axes in the table and graph activities. some pre-service teachers, either, did not ask about the students’ ideas in the activities of category determination and so, the students showed reluctance for taking part in the activities. sometimes, the students warned the pre-service teacher or told the activities that could be done and the pre-services teachers did activities in the same direction accordingly. for example a pre-service teacher who, using too many categories, struggled shared his experiences with another pre-service teacher named 2/b and 2/b limited the number of categories in the activities. teaching practice opportunities for pre-service teachers becomes more effective with their natural sharing experiences in teacher-student and teacher-teacher interactions. the pre-services teachers’ positive opinions that they can effectively teach the topics in the first grade level show that experiences acquired in real class contexts have very important effect on their pedagogical development. at the stages of collecting and organizing the data, it was observed that pre-service teachers themselves did the works in the class instead of instructing the students to do. posing the research question to the class, after collecting the data of the class about the research question from the students, some pre-service teachers organized the data either by themselves or with one or more students. the students who were sitting at their desks took part in the lesson by observing and copying what was done on the board made mistakes. it is thought that pre-service teachers see themselves, not the students, at the center and for that reason they could not do a lesson in accordance with constructivist approach. when the number of teaching practice and their duration increase, it could be said that pre-service teachers will realize lessons in which student are more active and that the way they do their teaching practice will change. in the lessons that elementary teachers will do about the topic of data, they should try to take care to: a) use real data; b) present good examples about the topic of data; c) ensure that students are active in all stages in the topic of data (franklin et al., 2007). during a single lesson more than one table and graph activities were done and, as a result, the lesson outcomes that wanted to be acquired was overlooked. it was seen that pre-service teachers did not well enough emphasize table and graph reading activities. defining the data is something beyond roughly-reading the information exists on the tables and graphs. reading the data is to see the information on the data representation without having any difficulty, to understand the given graph, and to make sense of the data in its context (curcio, 1987). koparan and guven (2013) highlighted that since activities which are directed to primary school students’ being able to be aware of the data presentations and being able to read some certain data from data presentation will set the foundation for the analysis and the interpretation of the data, these activities need to be done. for example, during table activities, activities in which one figure represented a few international electronic journal of elementary education vol.7, issue 3, 355-370, 2015 368 figure were carried out in the class but the pre-service teachers did not dwell on these activities in both table and graph activities. in table-reading, and table and graph interpretation activities, they rather preferred to instruct the student to study with questions directed to making operations instead of questions that would direct the students to interpret. in the conversations with the pre-service teachers after the teaching practice lessons, the pre-service teachers stated that they did not know what to ask, that there were exercise questions in the sources as routine so they mostly preferred to use these kinds of questions and that they became quite anxious. in the lessons, the pre-service teachers used the overhead projector, study papers, ready-made cardboards for presentations, and the board as materials. especially in collecting data and object graph activities, real objects, too, like beads, sticks, and candies are among the examples of material usage. pre-service teachers did not do any activity in the computer environment although there were computers in the classes they did teaching practice. it has drawn attention that in figure chart activities in the second-grade level, the preservice teachers perceived figure charts as object graphs. doing figure chart activities instead of object graph activities, the pre-service teacher 2/a did not include the students in the data collecting process and carried out the lesson with ready-made activities. 2/b brought blue, green, and red beads and wanted the students to fix the beads with the color they like to the sticks. saying,“are there only these colors? i don’t like these colors”, some students said they did not want to fix the beads, but the pre-service teacher said they had to choose a color. nevertheless, the activity worked out successfully because a natural figure chart activity was realized and the students were active in the activity. without having done figure chart activity well enough, the pre-service teacher 3/a attempted to do a bar graph activity in, again, grade 3 level but the students had difficulty in placing the number in sequence on the perpendicular axis and in naming the axes in the activity. one of the hardships experienced in the graph activities was encountered in bar graph activities in grade 4 level. difficulties were experienced in graph creating activities because of not having adequately done bar graph activities and because of the conducted activities in which teachers were active while the students passive. for example, copying already-drawn bar graphs on the board to their notebooks, those who made drawings similar to histograms drew attention. meanwhile, it was seen that the pre-service-teachers did not have any information about the differences between bar graphs and histograms. for example, in bar graph activities, the pre-service teacher named as 4/a drew horizontal and perpendicular axes and created the graph according the data given. however, names of the axes and title of the graph were not written either. he drew a histogram for a bar graph and later corrected it by saying, “this wouldn’t do”. one of the students said, “let’s unite the columns” but the pre-service teacher replied, “in the next one, we will do it”. copying from the board to their notebook, the children, too, made some drawings similar to histograms. that the pre-services teachers did not know about the differences between bar graphs and histograms caused the students to incorrectly learn and to make incorrect drawings. bar graphs and histograms are very similar in appearance. columns are used to show frequencies belonging to the categories in both kinds of graph. the difference between bar graphs and histograms is the kind of data being used. if the numbers representing categories are continuous, or if the numbers are able to be regrouped in different intervals, histograms can be used. if the data are non-continues, bar graphs can be used (musser, burger & peterson, 2008). that pre-service teachers know basic concepts in the topic of data is important from the point of view that they should be able to prevent students from incorrectly learning. some reflections from pre-service elementary teachers / doğan-temur, akbaba-dağ & turgut 369 completing the process with favorable and useful experiences, very few pre-service teachers who worked with a lecturer and a teaching practice teacher who was enthusiastic, eager, and conscious of his duties mentioned the experiences they had in the real learning environment and about the feeling that they felt themselves like a real teacher (eraslan, 2008). in this study, too, in the assessment study that pre-service teachers did with the teaching practice teachers after the lesson, they stated that they remembered the explanations done in the scope of “math teaching” course and that, preparing to teach the students the topic of data in the practice school, they revised what they learnt in the lessons but they struggled, failed to put their knowledge into application, and, made mistakes during the lesson in the class environment. the pre-service teachers expressed that they benefitted from the activities about the topic of data along the teaching practice process and from the assessments done after the lessons. moving from these results, it could be said that “math teaching” course has an important place in preservice teachers’ pedagogical development and that, for that reason, the lecturers should show sensitivity in the execution of the lesson. in the education of elementary teachers who will set the foundation for math topics and concepts, the importance of “math teaching” course and the lecturer executing the lesson are obviously seen. pre-service teachers’ practice teaching processes should be analyzed in not only the topic of data but also other math topics too, and the results, meeting the deficiencies, should be shared with pre-service teachers. • • • references aktaş, y. (2006). okul öncesi dönemde matematik eğitimi (3. baskı). adana: nobel kitabevi. bahr, d. l., & de garcia, l. a. (2010). elementary mathematics is anything but elementary.belmont: wadsworth, cengage learning. biggs, j. b., & collis, k. f. (1991). multimodal learning and intelligent behavior. in h. rowe (eds.), intelligence: reconceptualization and measurement (pp. 57-76). hillsdale, nj: erlbaum. büyüköztürk, ş., kılıç çakmak, e., akgün, e. a., karadeniz, ş., ve demirel, f.(2010). bilimsel araştırma yöntemleri. ankara: pegem akademi. cathart w. g., pothier y. m., vance j. h., & bezuk n. s. (2006). learning mathematics in elementary and middle schools: a learner centered approach (fourth edition). usa: pearson education. carruthers, e., & worthington, m. (2006). children's mathematics, making marks, making meaning (second edition). london: sage publications. charlesworth, r. (2000). experiences in math for young children. delmar: thompson learning. curcio, f. r. (1987). comprehension of mathematical relationships expressed in graphs. journalfor research i̇n mathematics education, 18, 382-393. eraslan, a. (2008). fakülte-okul i̇şbirliği programı: matematik öğretmeni adaylarının okul uygulama dersi üzerine görüşleri. hacettepe üniversitesi eğitim fakültesi dergisi 34, 95-105. franklin, c., kader, g., mewborn, d. s., moreno, j., peck, r., perry, m.& scheaffer, r. (2007). guidelines for assessment and instruction in statistics education (gaise) report: a pre-k-12 curriculum framework. american statistical association. franklin c. a.& denise s. m. (2008). “contemporary curriculum issues: statistics in the elementary grades: exploring distributions of data.” teaching children mathematics. (14), 10–16. international electronic journal of elementary education vol.7, issue 3, 355-370, 2015 370 friel, s. n., curcio, f. r. , & bright, g. w. (2001). making sense of graphs: critical factors influencing comprehension and instructional implications. journal for research in mathematics education. 32(2), 124–158. hacıömeroğlu, g. (2012). pedagojik gelişim ölçeğinin türkçe’ye uyarlaması: sınıf öğretmeni adaylarının matematik öğretimine i̇lişkin gelişim düzeyi. dicle üniversitesi ziya gökalp eğitim fakültesi dergisi,18, 48-68. jones, g. j., thornton, c. a., langrall, c. w, mooney, e. s, perry, b., & putt, i. j. (2000). a framework for characterizing children’s statistical thinking. mathematical thinking and learning, 2(4), 269–307. koparan, t. ve güven, b. (2013). i̇lköğretim i̇kinci kademe öğrencilerinin i̇statistiksel düşünme seviyelerindeki farklılaşma üzerine bir araştırma. i̇lköğretim online, 12(1), 158-178. matematik dersi (1-5. sınıflar) öğretim programı (2009). 11.02.2015. retrieved from http://ttkb.meb.gov.tr/www/ogretim-programlari/icerik/72 musser, g. l, burger, w. f., & peterson, b. e. (2008). mathematics for elementaryteachers: a conteporary approach (eighth edition). usa: john wiley&sons. olkun, s., ve toluk uçar, z. (2004). i̇lköğretim etkinlik temelli matematik öğretimi (3.baskı). ankara: anı yayıncılık. pereira-mendoza, l., & mellor, j. (1991). students’ concepts of bar graphs: some preliminary findings. in d. vere-jones (ed.), proceedings of the third international conference on teaching statistics: vol. 1 (pp. 150-157). the netherlands: international statistical institute. ryan j. and williams j. (2007). children’s mathematics 4-15:learnıng from errors and mısconceptıons. (first published). berkshire england: open university press swars, s. l., daane, c. j., & giesen, j. (2006). mathematics anxiety and mathematics teacher efficacy: what is the relationship in elementary preservice teachers? school science and mathematics, 106(7), 306–315. van de walle, j., karp, k. s., bay-williams, j. m. (2010). elementary and middle school mathematics; teaching developmentally, (7th ed.).boston, ma: allyn ve bacon yıldırım, a., ve şimşek, h. (2008). sosyal bilimlerde nitel araştırma yöntemleri. seçkin yayıncılık. international electronic journal of elementary education, december 2016, 9(2), 405-418. issn:1307-9298 copyright © iejee www.iejee.com the effects of physical activity on the on-task behavior of young children with autism spectrum disorders shane k. h. miramontez a * ilene s. schwartz a a university of washington, usa received: september, 2016 / revised: october, 2016 / accepted: november, 2016 abstract the purpose of this study was to examine the effects of adult-directed physical activities conducted during circle time on the on-task behavior of students during a journal-writing activity held immediately after circle. the participants of the study were three male students with autism spectrum disorder (asd) who attended a full day inclusive kindergarten program. during the last five minutes of morning circle the entire class, including the target participants, engaged in one of three activities that required different amounts of physical exertion (yoga, dance party, or book reading) that were selected randomly by the teacher. on-task behavior data was collected for each participant during the first five minutes of the subsequent journal writing activity. results found that, in general, movement (yoga and dance party) prior to journal led to higher levels of on-task behavior during journal writing, with different activities yielding different outcomes for individual participants. implications are that students with asd would benefit from participating in multiple bouts of planned physical activity across the instructional day. keywords: physical activity, on-task behavior, autism introduction autism spectrum disorder (asd) is a lifelong neurological disorder characterized by deficits in social-communication and restrictive or repetitive behaviors (american psychiatric association, 2013). the center for disease control and prevention (cdc) estimates the prevalence of autism to be 1 in 68 (cdc, 2016). this means that in the united states over 1.5% of school-aged children are students with asd, increasing the likelihood that every teacher in the united states will have a student with asd in his or her classroom. while some of these students are still educated in segregated special education classrooms, many are being educated in the general education classroom, where appropriate behavior, attending skills, and engagement in group and individual tasks are building blocks for school success. due to the range of social and communication deficits, however, students with asd are often less engaged during instruction than their typically developing peers (carnahan, musti-rao, & bailey, 2009) and do not perform as well in * corresponding author: shane k.h. miramontez, department of special education, university of washington, 102 miller, box 353600, seattle, wa 98195-3600. e-mail: skh20@uw.edu http://www.iejee.com/ international electronic journal of elementary education vol.9, issue 2, 405-418, december, 2016 406 school as one would expect based on their scores on norm-referenced assessments (estes, rivera, bryan, cali, & dawson, 2010; goodman & williams, 2007). these challenges and discrepancies in students with asd have resulted in the need for an expansion of classroom interventions that are both effective and practical for teachers to implement. evidence-based interventions targeting engagement and on-task behavior – skills fundamental to learning – are especially valuable. interventions that are easy to implement, involve all students, and may have benefits for everyone in the classroom will be more likely to be adopted and implemented with fidelity by busy classroom teachers. the national research council (nrc) defines engagement as, “sustained attention to an activity or person” (p. 160) and recognizes engagement as a key component for effective programming for students with asd (national research council, 2001). students who are more engaged with peers, teachers, and educational materials are better positioned to learn academic and social-emotional skills needed to be successful in school and their communities (marks, 2000; wang & holcombe, 2010). there are several ways in which researchers have increased the academic engagement of students with asd including arranging the instructional/classroom environment (e.g., carnahan, hume, clarke, & borders, 2009; koegel, singh, & koegel, 2010; schilling & schwartz, 2004), visual supports (e.g., bryan & gast, 2000; carnahan, hume, clarke, & borders, 2009; delano & snell, 2006; macduff, krantz, & mcclannahan, 1993; spriggs, gast, & ayres 2007), self-monitoring (e.g., holifield, goodman, hazelkorn, & heflin, 2010), and systems of reinforcement (e.g., koegel et al., 2010). while effective, maintaining a scripted intervention for only one or two students in the classroom may not be feasible for some teachers. therefore, one challenge facing researchers and practitioners as they attempt to develop and use strategies to increase the on-task behavior of children with asd in the classroom, is the feasibility and thus sustainability of the intervention. sustainability includes features such as how the intervention is viewed by the classroom teachers, how the intervention fits into the social context and actual schedule of the school, and how much effort and/or cost is required for the intervention to be implemented. the sustainability of an intervention, often measured by social validity assessments in behavioral research, is essential to consider. if a classroom teacher is unwilling or unable to implement an intervention, students with asd may not have access to the same learning opportunities as their peers (schwartz & baer, 1991). a promising intervention for increasing appropriate on-task behaviors in children, including those with asd, is physical activity or exercise (wong, et al., 2014). physical activity interventions may be particularly appealing because many require little to no specialized training (jogging or dancing, for example) and fit well into the ecology of a school day. this feasibility, combined with the added impact of health and well-being (national center for chronic disease prevention and health promotion, [nccdphp]), and the overwhelming evidence that young children – especially those with autism, whose overweight and obesity rates were higher than the national averages (egan, dreyer, odar, beckwith, & garrison, 2013) – do not engage in enough activity throughout their day (brown et al., 2009; brown, googe, mciver, & rathel, 2009; hinkley, et al., 2014; pate, mciver, dowda, brown, & addy, 2008; tucker, 2008) are additional, critical reasons to further explore the effectiveness of these interventions, especially when implemented by classroom teachers. within the breadth of physical activity literature, studies specifically examining the effects of antecedent movement on on-task behavior in children without asd have mixed results. many studies show that activity can prevent off-task behavior (grieco, jowers, & bartholomew, 2009; ma, mare, & gurd, 2014) and increase on-task behavior (howie, beets, & pate, 2014; jarrett, maxwell, dickerson, hoge, davies, & yetley, 1998; mahar, the effects of physical activity / miramontez &. schwartz 407 murphy, rowe, golden, shields, & raedeke, 2006). of these interventions, however, few take place within in the context of a classroom. in a review of literature examining the relationship between physical activity and academic performance, rasberry and colleagues (2011) found only 9 studies out of the 42 articles that met inclusion criteria took place in the classroom and only one of these measured on-task behavior (rasberry, et al., 2011). a second review and meta-analysis of physical activity and features of engagement (i.e., emotions, cognition, and behavior) showed that only 7 of the 38 studies looking at behavioral engagement were embedded into classroom activities, while only an additional 5 took short breaks from academics to engage in an in-class physical activity (owen, et al., 2016). for students with asd and/or developmental delays, much of the physical activity research focuses on using activity to decrease stereotypic and self-stimulatory behavior (celiberti, bobo, kelly, harris, & handleman, 1997; elliott, et al., 1994; kern et al., 1982; kern, et al., 1984; levinson & reid, 1993; powers et al., 1992; rosenthal-malek & mitchell, 1997; watters & watters, 1980). jogging has been the most common intervention (celiberti, et al., 1997; kern, et al., 1982; kern, et al., 1984; levinson & reid, 1993; rosenthal-malek & mitchell, 1997), while roller skating (powers, et al., 1992), stretches including sit-ups, arm circles, and toe touches (reid et al., 1988), and playing catch with a ball (kern, et al., 1984) have also been effective interventions for decreasing maladaptive behaviors. also noteworthy, the effort exerted during activity has resulted in varied decreases of behavior. for example, vigorous exercise, such as jogging or running on a treadmill was found to have a greater effect on reducing self-stimulatory behaviors than mild exercise such as ball playing or slowing riding a stationary bike (elliott et al., 1994; kern et al., 1984; levinson & reid, 1993). in addition to decreasing stereotypic and self-stimulatory behaviors, access to more frequent bouts of physical activity, mainly jogging, has shown an increase in expected classroom behaviors such as increased and correct responding (kern et al., 1982; rosenthal-malek & mitchell, 1997), on-task behavior (kern et al., 1982; nicholson, kehle, bray, & van heest, 2011; powers et al., 1992; reid et al., 1988) and work task completion (rosenthal-malek & mitchell, 1997). while effective, none of these interventions took place in the context of a classroom routine. embedding physical activity into the existing schedule may influence the ecological validity of the intervention, make the intervention accessible to more students, and increase the likelihood that an intervention will be sustained upon the conclusion of a research project. the purpose of this study then is threefold: 1) to examine the effect that embedded physical activity has on the on-task behavior during subsequent academic tasks for young students with asd, 2) to compare rates of on-task behavior based on different types of classroom based physical activities, and 3) to find an intervention that is easy to implement and sustainable in the context of a kindergarten classroom. method participants three male kindergarten students with asd participated in this study. all three students attended an inclusive full day kindergarten program at a comprehensive early intervention center. the kindergarten curriculum included focused instruction in literacy, mathematics, social skills, and school success skills. paul was a five year, ten month old caucasian, nate was a six year, one month old african-american, and liam was a six year, three month old caucasian. nate and liam were in the same classroom, while paul was in a second kindergarten classroom at the same school. on the peabody picture vocabulary test, 4th edition (ppvt 4), a receptive language assessment, paul received a standard international electronic journal of elementary education vol.9, issue 2, 405-418, december, 2016 408 score of 69, while nate’s standard score on the same assessment was 94. no additional assessment information was available for liam. all three participants were verbal, used full sentences to communicate, and had a diagnosis of asd from an agency not affiliated with the research. the head teachers and assistant teachers in each classroom also participated in the implementation of the study. the head teachers both had master’s degrees and certification in special education. the assistant teachers were graduate students working towards their special education certification. the classroom staff members were responsible for implementing the intervention at circle and supervising the students during journal time. experimental design an alternating treatment design (gast, 2010) was used to compare the effects of three different types of circle time activities on students’ on-task behavior during a journal writing activity that immediately followed the treatment. the three activities that were compared as part of the intervention were yoga, dance party and listening to a story (i.e., book reading). during the intervention portion of this study, one of these treatments occurred during the last five minutes of morning circle every day. immediately after being exposed to the treatment, children made the transition to their tables and the journal writing activity. data on child behavior were collected during journal writing. all children in the classroom participated in morning circle, the intervention, and journal writing. following baseline, treatments were randomly assigned each day. at the beginning of the morning circle, the teacher drew a piece of paper out of a cup to determine the treatment. the cup contained three pieces of paper, one with each of the three treatment options on it. decision rules for the interventions were established a priori and stated that no intervention option could occur more than two days in a row. if any intervention had been drawn for two consecutive days, on the third day, the piece of paper with that intervention option would be removed from the cup to ensure that a different choice would be drawn. following that draw, all treatment options would be put back into the cup and the randomization process would continue. procedure a baseline condition occurred prior to beginning the intervention. during the baseline phase, the teachers conducted circle and journal writing in the same way they had been doing all year. the circle followed the same schedule everyday consisting of the following activities: a song or book to start, calendar discussion, and introducing the daily journal topic. immediately following circle, students were directed to find their seat at small group and begin writing on the previously discussed journal topic. intervention the independent variable was the type of activity performed during the last five minutes of circle: book reading, yoga, or a dance party. for book reading, students sat on circle mats while listening to a book being read by the teacher. yoga involved all students in the classroom taking turns picking different yoga poses from a choice of pictures cards. once a pose was picked, all students performed the action based on the teacher’s model. if any of the target students were not performing the behavior, an adult prompted the child to perform the action using least to most prompting strategies (cooper, heron, & heward, 2007). between five and seven poses occurred during each intervention session. the dance party consisted of dancing to a collection of songs created for each classroom by the author. dance party was also teacher directed and consisted of students copying a “leader’s” dance moves and doing partner or group dancing within the circle area. if target the effects of physical activity / miramontez &. schwartz 409 students were not engaged in a dance movement or actively moving their arms and legs for more than 30 seconds, an adult prompted the child to start dancing using least to most prompting strategies (cooper, heron, & heward, 2007). prior research (kern et al., 1984; elliott et al., 1994; levinson & reid, 1993) has shown differential effects based on the intensity of physical exertion. though not objectively measured, yoga and dance party were chosen in attempt to replicate mild (yoga) and moderate (dance party) forms of physical activity that were common and easy to embed into a kindergarten circle routine. the book reading condition was used represent a current treatment control, since it is a sedentary active that is frequently part of a kindergarten circle time. measures the dependent variable was on-task behavior during a journal-writing task immediately following the independent variable (i.e., the book reading, yoga, or dance party). data for on-task behavior was collected for the first seven minutes of journal writing. on-task behavior was operationally defined as independent continuous appropriate engagement in the assigned activity. the operational definition of on task included looking at an adult giving a direction, looking at the assignment, manipulating task materials in an appropriate manner (including cutting, writing, drawing), and following group or individual directions. if adult support was needed to complete a task, the participant was scored off-task for that interval. adult support included prompts to engage in the appropriate behaviors, stay on task, and/or complete the task at hand. off-task behavior was defined as disengagement from the academic activity at hand, including looking away from the action, staring into space, out of seat behavior, engaging in a task that was not assigned (e.g., reading a book during journal writing time), not responding to group or individual directions and engaging in side conversation. momentary time sampling was used to collect the on-task behavior data. ten second intervals were used to record behavior, meaning that observers recorded the student’s behavior – if they were on-task or off-task – by looking up once every ten seconds and recording what behavior they saw at that moment. observers used a vibrating motivaider to signal the ten-second intervals and indicate when it was time to collect data for each interval. reliability inter-observer reliability was assessed for on-task behaviors during journal. two observers independently recorded incidences of on-task and off-task behaviors for 15% of data collection sessions. the observers agreed on the incidence of behavior type (on task or off task) 96% (range 91 – 100%) of the time for behaviors during journal. social validity and preference assessment. at the end of the study, student and adult participants took part in a social validity assessment. the purpose of the assessment was to attempt to understand participants’ attitudes about the intervention components and to determine if they were components of the intervention that they preferred (schwartz & baer, 1991). to assess the students’ preference of the activity choices, every student in the class was given a piece of paper and asked to write down which activity (i.e., book reading, yoga, or dance party) was their favorite. target participants’ responses were marked for data analysis. in an attempt to understand what the teachers thought about this intervention, teachers were asked to complete a brief questionnaire at the end of the study. it included questions about their perceptions about effect of the intervention on the behavior of the target participants, ease of intervention, how the intervention fit into their classroom routine, international electronic journal of elementary education vol.9, issue 2, 405-418, december, 2016 410 and likeliness to use the intervention again. all questions were answered on a 5-point likert-type scale, with 5 being the highest possible rating. the questionnaire also had a place for teacher comments. results in general, the results of this intervention suggest on task behavior during journal writing increased when students participated in an activity that required some movement immediately before that journal writing activity. there was, however, some variability within these findings. the results for all of the participants can be found in figures 1-3. paul demonstrated the most robust changes during the intervention. during baseline, paul (figure 1) was on-task during journal behavior an average of 53% (range 39 – 67%). when the independent variable was implemented he demonstrated immediate and noticeable changes in his behavior. following the yoga sessions, paul was on task for an average of 92% (range 90 – 93%) of the intervals during journal writing. during the journal writing observations following dance party sessions he averaged of 83% (range 77 – 89%) if on-task intervals. following the book reading condition, the condition that was most similar to baseline, paul was on-task during journal an average of 45% of the observed intervals (range 8 – 66%). figure 1. percentage of on-task behavior during journal writing for baseline and treatment conditions for paul liam (figure 2) demonstrated similar changes in his behavior. during baseline, liam was on-task during journal an average of 44% (range 34 – 58%). his average of on-task behavior increased following yoga sessions to 55% (range 30 – 66%) and 66% (range 61 – 71%) during dance party sessions. during the book reading condition, liam was on-task an average of 42% of the observed intervals (range 37 – 53%). 0 10 20 30 40 50 60 70 80 90 100 1 2 3 4 5 6 7 8 9 10 11 12 13 14 p e rc e n ta g e o f o n -t a sk b e h a v io r sessions baseline treatment yoga book dance party paul the effects of physical activity / miramontez &. schwartz 411 figure2. percentage of on-task behavior during journal writing for baseline and treatment conditions for liam nate (figure 3), who was absent five times during the month long intervention, was ontask during baseline an average of 48% (range 22 – 66%) of the observed intervals during journal writing. his percentage of on-task behavior increased to 73% during yoga (range 56 – 91%) and approximately 79% (range 64 – 97%) of the time after participating in the dance party. due to his high number of absences, he only experienced the book reading intervention one time, and his rate of behavior during the journal writing that followed book reading was similar to that during baseline (45%). figure 3. percentage of on-task behavior during journal writing for baseline and treatment conditions for nate 0 10 20 30 40 50 60 70 80 90 100 1 2 3 4 5 6 7 8 9 10 11 12 13 14 p e rc e n ta g e o f o n -t a sk b e h a v io r sessions baseline treatment dance party yoga book liam 0 10 20 30 40 50 60 70 80 90 100 1 2 3 4 5 6 7 8 9 10 11 p e rc e n ta g e o f o n -t a sk b e h a v io r sessions baseline treatment dance party yoga book nate international electronic journal of elementary education vol.9, issue 2, 405-418, december, 2016 412 social validity data in attempt to determine what the children, (i.e., one group of primary consumers) thought about this intervention, all 29 children in the kindergarten who had participated in the study were asked to write their favorite intervention on a piece of paper. these pieces of paper were collected before the last intervention session. of the 29 total kindergarten children, 19 choose the dance party, while 9 students chose yoga, and 1 child chose book reading. of the three target students, liam chose yoga while paul and nate chose dance party. the teachers also gave the intervention with very high marks. the 4 participating adults (2 head teachers and 2 assistant teachers) reported that they noticed an increase in appropriate behavior after both activity options (i.e., yoga and dance party). they rated the intervention as very easy to use, giving it a 4 out 5 on a 5-point scale. all the teachers said that they planned to use this intervention again. one teacher summed up the overall responses of the group by saying, “i thought this intervention as wonderful. a great study that i was happy to be involved with.” discussion the purpose of this study was to examine if brief bouts of physical activity that were embedded into a morning circle in an inclusive kindergarten classroom would have an effect on the on-task behavior of children with asd in a journal writing task that immediately followed. the answer is yes. this study demonstrated that kindergarten students with asd were more on-task following yoga or dance part (both effortful activities) as compared to book reading (a sedentary activity). the results of the social validity assessment suggest that this intervention was found to be acceptable by two groups of primary consumers – teachers and students. teachers reported that they would use this intervention again and 28 of the 29 students, including all three of the targeted participants, indicated that they preferred one of the effortful activities to the sedentary activity. the results of this study, though limited, are promising and support the prior research showing relationships between movement prior to academic tasks and increased on-task behavior. activity prior to seatwork seemed to benefit paul the most, with large increases in on-task behavior following both yoga and dance party sessions. liam, who had the lowest on-task percentage during baseline, had the most variability during intervention. however, it should be noted that only one of liam’s data points during the dance party and yoga interventions (session 10) overlaps with baseline percentages, while all three book reading phases (sessions 7, 8, and 13) are within the same on-task range as baseline. this does show that movement had some effect on liam as well. nate’s data were more difficult to analyze, given fewer data points due to a high rate of absences. his data, however, are still promising and he, like the other participants preferred movement to the baseline (book reading) condition when asked. in addition to overall increases following the two movement interventions, two of the three participants also showed greater on-task performance following the most fast-paced activity (dance party), which supports other findings (kern et al., 1982) that demonstrate less self-stimulatory behavior with more vigorous exercise. the implications of this study are that students who engage in adult led physical activity can increase their on-task behaviors for academic tasks that follow the physical activity. though moderate and vigorous physical activity both have an effect on the increase of ontask behaviors, vigorous activity, specifically, may also have a greater effect on on-task behaviors for students with autism. this may be because vigorous physical activity serves the effects of physical activity / miramontez &. schwartz 413 as a more fulfilling sensory experience, allowing students to be available for learning following active vigorous engagement. however, if a classroom teacher does not have the time or resources to engage in vigorous activity (i.e. it is not time for recess and vigorous activity is difficult and dangerous within the confines of a classroom), they can still create moderate activity sessions within the classroom routine that would improve on-task behavior in students with asd. additionally, and perhaps more important, was the high ranking (4 out of a 5-point scale) given by both teachers on ease of implementation and willingness to use this intervention again with a range of students. this intervention has implications for students who may not have a diagnosis, but who demonstrate behaviors that are concerning to teachers such as difficulty transitioning or getting starting on a new task, needing support attending to tasks, and fidgeting or engaging in distracting behaviors. it is also interesting to note that out of the 29 students who recorded their preferred activity on the final day of intervention, only one stated a preference for book reading, with all other students preferred movement. this information could imply that the act of book reading – which is common in many preschool and kindergarten circles – many not be the most preferred or helpful circle time activity. there are several limitations to this study. the small, homogenous subject size (three high functioning kindergarten boys with asd) and length of intervention may not produce the same effects when implemented for a longer time (both daily and intervention as a whole) with a greater variety of students. additionally, it may be difficult to make strong causal claims regarding the effort put forth while engaging in the two kinds of physical activity, as this was not monitored with equipment. areas for future research could include adjustments to the current study given the limitations noted above. monitoring the levels of energy exerted during each intervention phase with equipment can help answer questions related to the timing and dosage of physical activity. how much is necessary to see a change in on-task behavior and how long do the effects last? how long do the bouts of physical activity need to last? is there benefit of frequent short activities, or does sustained moderate to vigorous physical activity have a longer lasting effect on student behavior. it will also be interesting and important to determine how the amount and type of physical activity that is related to change in classroom behavior relates to the amount and type of physical activity that is recommend for health reason (i.e., obesity prevention). additionally, adding a social component to the intervention by using peer models instead of adult support could add to the feasibility of implementation. after all, “development of friendships is a high priority for any child” (staub, peck, gallucci, & schwartz, 1999; p. 386) and research shows that children gain skills they need to be successful by engaging in positive interactions and developing peer relationships that, in turn, make them feel like a member of the team (peck, staub, gallucci, & schwartz, 2004). it is critical that children of all ability levels participate in meaningful activities that promote friendship and, as a result, learning. physical activity is no exception. increasing the amount of sustained physical activity that children participate in every day is a topic of great interest across multiple disciplines. it will be important that as work in this area moves forward, researchers and advocates from different disciplines work together to highlight all of the benefits that increased activity may have for young people. it will also be important that as the national dialogue about obesity prevention moves forward, that children with disabilities are included in the discussion. like many national campaigns for child welfare, the anti-obesity and exercise promotion programs have largely excluded children with disabilities. the results of this study suggest that in international electronic journal of elementary education vol.9, issue 2, 405-418, december, 2016 414 addition to the health benefits, the quality of the classroom experience for children with disabilities may improve with a systematic program to increase their level of physical activity across the day. the participants with asd not only preferred the vigorous activities to book reading, their behavior improved enabling them to participate more fully and be more successful in the planned activities of the school day. in conclusion, physical activity should be an important part of the daily schedule for every child, including children with asd. in addition to the known health benefits, physical activity has behavioral and academic benefits as well. research has shown that students with asd show increased on-task behaviors following bouts of physical activity and this study supports the prior research findings. it is imperative that adults play a large role in facilitating and supporting physical activity participation for all children, especially those with asd. specifically, planned adult-directed activities have yielded strong increases in physical activity (wadsworth, 2011; brown et al., 2009). it is important for adults to guide activity and provide support and encouragement to help students maintain high levels of engagement. this notion is confirmed through studies that show higher levels of physical activity in young children during adult directed activities (brown, googe, mciver & rathel, 2009; pan, 2008). equally important is embedding these adult led activities – such as yoga and dance party – into classroom routines as minimal effort can produce impactful health and academic outcomes. • • • references american psychiatric association (apa). (2013) diagnostic and statistical manual of mental disorders. fifth edition. washington, dc: american psychiatric association. baumeister a. a., & maclean w. e. (1984). deceleration of self-injurious and stereotypic responding by exercise. applied research in mental retardation. 5(3), 385-93. brown, w. h., googe, h. s., mciver, k. l., & rathel, j. m. (2009). effects of teacher-encouraged physical activity on preschool playgrounds. journal of early intervention, 31(2), 126-145. brown, w. h., pfeiffer, k. a., mciver, k. l., dowda, m., addy, c. l., & pate, r. r. (2009). social and environmental factors associated with preschoolers’ nonsedentary physical activity. child development, 80(1), 45-58. bryan, l. c., & gast, d. l. (2000). teaching on-task and on-schedule behaviors to high-functioning children with autism via picture activity schedules. journal of autism and development disorders. 30(6), 553-567. carnahan, c. r., hume, k., clarke, l., & borders, c. (2009). using structured work systems to promote independence and engagement for students with autism spectrum disorders. teaching exceptional children, 41(4), 6-14. carnahan, c., musti-rao, s., & bailey, j. (2009). promoting active engagement in small group learning experiences for students with autism and significant learning needs. education and treatment of children, 32(1), 37-61. celiberti d.a., bobo h.e., kelly k.s., harris s.l., & handleman j.s. (1997). the differential and temporal effects of antecedent exercise on the self-stimulatory behavior of a child with autism. research in developmental disabilities. 18(2), 139-150. centers for disease control and prevention. (2016). prevalence of autism spectrum disorders among children aged 8 years: autism and developmental disabilities monitoring network, 11 sites, united states, 2012. mortality morbidity weekly report surveillance summaries, 65, 1–28. the effects of physical activity / miramontez &. schwartz 415 cooper, j. o., heron, t. e., & heward, w. l. (2007). applied behavior analysis, 2nd edition. upper saddle river, nj: pearson. dawson, g. & watling, r. (2000). interventions to facilitate auditory, visual, and motor integration: a review of the evidence. journal of autism and developmental disorders, 30, 415-421. delano, m. & snell, m. e. (2006). the effects of social stories on the social engagement of children with autism. journal of positive behavior interventions, 8(1), 29-42. egan, a. m., dreyer, m. l., odar, c. c., beckwith, m., & garrison, c. b. (2013). obesity in young children with autism spectrum disorders: prevalence and associated factors. childhood obesity (print), 9(2), 125-31. elliott, r. o., dobbin, a. r., rose, g. d., & soper, h. v. (1994). vigorous, aerobic exercise versus general motor training activities: effects on maladaptive and stereotypic behaviors of adults with both autism and mental retardation. journal of autism and developmental disorders. 24(5), 565. estes, a., rivera, v., bryan, m., cali, p., & dawson, g. (2011). discrepancies between academic achievement and intellectual ability in higher-functioning school-aged children with autism spectrum disorder. journal of autism and developmental disorders, 41(8), 1044-1052. gast, d. l. (2010). single subject research methodology in behavioral sciences. new york: routledge. goodman, g., & williams, c. m. (2007). interventions for increasing the academic engagement of students with autism spectrum disorders in inclusive classrooms. teaching exceptional children, 39(6), 53-61. grieco l.a., jowers e.m., & bartholomew j.b. (2009). physically active academic lessons and time on task: the moderating effect of body mass index. medicine and science in sports and exercise. 41(10), 1921-1926. hinkley, t., teychenne, m., downing, k. l., ball, k., salmon, j., & hesketh, k. d. (2014). early childhood physical activity, sedentary behaviors and psychosocial well-being: a systematic review. preventive medicine, 62, 182-192. holifield, c., goodman, j., hazelkorn, m., & heflin, j.l. (2010). using self-monitoring to increase attending to task and academic accuracy in children with autism. focus on autism and other developmental disabilities. 25(4) 230–238. doi: 10.1177/1088357610380137 hume, k., & odom, s. (2007). effects of an individual work system on the independent functioning of students with autism. journal of autism and developmental disorders. 37(6), 1166-1180. howie, e. k., beets, m. w., & pate, r. r. (2014). acute classroom exercise breaks improve on-task behavior in 4th and 5th grade students: a dose–response. mental health and physical activity, 7(2), 65-71. jarrett, o. s., maxwell, d. m., dickerson, c., hoge, p., davies, g., & yetley, a. (1998). impact of recess on classroom behavior: group effects and individual differences. the journal of educational research. 92(2), 121. kern l., koegel r.l., & dunlap g.. (1984). the influence of vigorous versus mild exercise on autistic stereotyped behaviors. journal of autism and developmental disorders. 14(1), 57-67. kern l., koegel r.l., dyer k., blew p.a., & fenton l.r. (1982). the effects of physical exercise on selfstimulation and appropriate responding in autistic children. journal of autism and developmental disorders. 12(4), 399-419. koegel, l.k., singh, a.k., & koegel, r.l. (2010). improving motivation for academics in children with autism. journal of autism and developmental disorders. 40, 1057-1066. levinson, l. j., & reid, g. (1993). the effects of exercise intensity on the stereotypic behaviors of individuals with autism. adapted physical activity quarterly. 10(3), 255. international electronic journal of elementary education vol.9, issue 2, 405-418, december, 2016 416 macduff g.s., krantz p.j., & mcclannahan l.e. (1993). teaching children with autism to use photographic activity schedules: maintenance and generalization of complex response chains. journal of applied behavior analysis. 26(1), 89-97. mahar, m. (2011). impact of short bouts of physical activity on attention-to-task in elementary school children. preventive medicine, 52, 60-64. mahar, m. t., murphy, s. k., rowe, d. a., golden, j., shields, a. t., & raedeke, t. d. (2006). effects of a classroom-based program on physical activity and on-task behavior. medicine and science in sports and exercise. 38(12), 20862094. marks, helen m. (2000). student engagement in instructional activity: patterns in the elementary, middle, and high school years. american educational research journal, 37(1), 153-84. marks, s. u., shaw-hegwer, j., & schrader, c. (2003). instructional management tips for teachers of students with autism spectrum disorder (asd). teaching exceptional children, 35(4), 50-4. morrison, r. s., sainato, d. m., benchaaban, d., & endo, s. (2002). increasing play skills of children with autism using activity schedules and correspondence training. journal of early intervention. 25(1), 58-72. national center for chronic disease prevention and health promotion. a report of the surgeon general on persons with disabilities. retrieved from: http://www.cdc.gov/nccdphp/sgr/pdf/disab.pdf national center for chronic disease prevention and health promotion. a report of the surgeon general on adolescents and young adults. retrieved from: http://www.cdc.gov/nccdphp/sgr/pdf/adoles.pdf national research council (u.s.). (2001). educating children with autism. washington, dc: national academy press. nicholson, h., kehle, t. j., bray, m. a., & heest, j. v. (2011). the effects of antecedent physical activity on the academic engagement of children with autism spectrum disorder. psychology in the schools, 48(2), 198-213. owen, k., parker, p., van zanden, b., macmillan, f., astell-burt, t., & lonsdale, c. (2016). physical activity and school engagement in youth: a systematic review and meta-analysis. educational psychologist, 51(2), 129-145. pan, c.-y., & frey, g. (2006). physical activity patterns in youth with autism spectrum disorders. journal of autism and developmental disorders. 36(5), 597-606. pan, c.-y. (2008). objectively measured physical activity between children with autism spectrum disorders and children without disabilities during inclusive recess settings in taiwan. journal of autism and developmental disorders. 38(7), 1292-1301. pate, r. r., mciver, k., dowda, m., brown, w. h., & addy, c. (2008). directly observed physical activity levels in preschool children. journal of school health, 78(8), 438-444. peck, c.a., staub, d., gallucci, c., & schwartz, i. (2004). parent perception of the impacts of inclusion on their nondisabled child. research & practice for persons with severe disabilities, 29(2), 1-9. rasberry, c., lee, s., robin, l., laris, b.a., russell, l., coyle, k., & nihiser, a. (2011). the association between school-based physical activity, including physical education, and academic performance: a systematic review of the literature. preventive medicine, 52, 10-20. rosenthal-malek, a., & mitchell, s. (1997). brief report: the effects of exercise on the selfstimulatory behaviors and positive responding of adolescents with autism. journal of autism and developmental disorders. 27(2), 193-202. schwartz, i., & baer, d. m. (1991). social-validity assessments: is current practice state-of-the-art. journal of applied behavior analysis, 24, 189-204. the effects of physical activity / miramontez &. schwartz 417 spriggs, a. d., gast, d. l., & ayres, k. m. (2007). using picture activity schedule books to increase on-schedule and on-task behaviors. education and training in developmental disabilities, 42(2), 209-23. staub, d., peck, c.a., gallucci, c. & schwartz, i. (1999). peer relationships. in m.e. snell & f. brown (eds), instruction of students with severe disabilities (381-407). upper saddle river, new jersey: prentice hall. tucker, p. (2008). the physical activity levels of preschool-aged children: a systematic review. early childhood research quarterly, 23(4), 547-558. wang, m., & holcombe, r. (2010). adolescents' perceptions of school environment, engagement, and academic achievement in middle school. american educational research journal,47(3), 633-662. watling, r. l., dietz, j. & white, o. (2001). comparison of sensory profile scores of young children with and without autism spectrum disorders. the american journal of occupational therapy. 55, 416-423. watters, r. g., & watters, w. e. (1980). decreasing self-stimulatory behavior with physical exercise in a group of autistic boys. journal of autism and developmental disorders. 10 (4), 379-87. wong, c., odom, s. l., hume, k. cox, a. w., fettig, a., kucharczyk, s., ... schultz, t. r. (2014). evidencebased practices for children, youth, and young adults with autism spectrum disorder. chapel hill: the university of north carolina, frank porter graham child development institute, autism evidence-based practice review group. international electronic journal of elementary education vol.9, issue 2, 405-418, december, 2016 418 www.iejee.com this page is intentionally left blank http://www.iejee.com/ microsoft word iejee_4_2_viseu_oliveria_287_300 international electronic journal of elementary education, 2012, 4(2), 287-300. issn:1307-9298 copyright © iejee www.iejee.com open-ended tasks in the promotion of classroom communication in mathematics floriano viseu ∗∗∗∗ cied-universidade do minho, portugal inês bernardo oliveira escola secundária da boa nova, portugal received: october 2011 / revised: february 2012 / accepted: february 2012 abstract mathematics programmes in basic education are currently undergoing reform in portugal. this paper sets out to see how teachers are putting the new guidelines for the teaching of mathematics into practice, with particular emphasis on maths communication in the classroom. to achieve this, an experiment in teaching the topic 'sequences and regularities' with open-ended tasks, using a qualitative and interpretative approach, is reported. data were collected during two class observations, from two interviews and by analysing the activities of the students. an exploratory task was chosen in the first lesson and a investigative one in the second. one month separated the two lessons, and during this time the teacher read and discussed texts on mathematics communication. observation of the first lesson showed that the communication in the classroom was mostly focused on the teacher, which provided little student-student and student-class interaction. in the second observed lesson, the teacher changed the attention she paid to what each student said and did, encouraging the students to ask each other and encouraged student-class and the student-student communication. keywords: reform of mathematics programmes; teaching mathematics; open-ended tasks; forms of communication; sequences and regularities. introduction the constant evolution of knowledge in the field of mathematical education determines the changes that are made periodically in maths programmes. in portugal, the reformulation of the basic education maths programs, which began in 2009/10, is now in the implementation stage and covers all the school years this academic year1. the basic education maths ∗ centro de investigação em educação, universidade do minho, braga, portugal. e-mail: fviseu@ie.uminho.pt 1 the portuguese education system encompasses 12 years prior to entry into higher education, as in most countries. the first nine of these years comprise basic education and the last three are secondary education. basic education consists of three cycles: the first lasts four years with just a single teacher; the second lasts two years, and the third lasts three years. during these nine years the maths curriculum is same for all students. in the three international electronic journal of elementary education, 2012, 4(2), 287-300. 288 programme that is being overhauled has existed since the early nineties (1990 for the 1st cycle and 1991 for the 2nd and 3rd cycles). the publication in 2001 of the national curriculum for basic education brought changes to the previous programme, especially to the learning goals and objectives and to the way that maths topics are addressed. these changes are justified by the need to update the curriculum to the new ways of developing knowledge about the teaching and learning of mathematics and to improve the coordination between the programmes of the three cycles. the programme begins by presenting the general aims and objectives of the teaching of mathematics, which are the main goals shared by three cycles of basic education. next, it presents the mathematical topics, numbers and operations, geometry, algebra, organisation and data processing. the present reformulation focuses on the organisation of maths topics that link together the different teaching cycles and the methodological guidelines for teaching these topics; the emphasis was on the cross-disciplinary aspects to be developed over one school cycle – resolution of problems, mathematical reasoning and mathematical communication. regardless of the study topic, the teaching of mathematics currently recommends the use of strategies which value student activity over a teaching process that is essentially centred on the activity of the teacher, where students mainly listen to and do what the teacher asks (nicol, 1999, nctm, 2007). but the conceptions of the teacher about the act of teaching go hand in hand with the curricular reforms (ponte, 1992), conceptions that are very often focused on teacher authority in validating what happens in the classroom. when appraising what the student says and does in classroom activities, the how the teacher stimulates and manages mathematics communication is paramount. current methodological guidelines for the 3rd cycle program are that the teacher should present different types of tasks that enable the “comparison of results, the discussion of strategies” (ministry of education, 2007, p. 8). teaching strategies therefore involve engaging the students in activities of analysing, doing, listening, reflecting, arguing, and discussing. such activities affect how teachers evaluate students’ reasoning and encourage them to analyse and respond to other students’ reasoning, which relates to how mathematical communication is promoted during classroom work: the creation of adequate opportunities for communication is assumed to be an essential part of the work being done in the classroom. (...) students compare their problemsolving strategies and identify the arguments made by their colleagues through oral discussion in class. they have the opportunity to clarify and explain in more detail their strategies and arguments through written work. (ministry of education, 2007, pp. 8-9) mathematics communication is essential to enabling students to understand about processes, discussions and decisions that are made. however, the achievement of curricular rules depends on how they are interpreted by teachers and on how they adjust them to their own conceptions on the act of teaching. in order to see to what extent the methodological guidelines are produced in practice, we seek to ascertain how maths teachers promote mathematical communication in the classroom through open-ended tasks. communication in mathematics classes taking the classroom as a special place for relationships between students and between them and the teacher, the way that this relationship is promoted becomes fundamental in the development of the teaching and learning process. by regulating the social interactions that are generated in the classroom, communication enables the sharing of ideas and clarification of mathematical understanding. here we have a perspective of teaching that years of secondary education, where students begin to be routed to a group of higher education courses, the maths curriculum varies according to whether courses in sciences, humanities, arts or technology are followed. open-ended tasks in the promotion of classroom communication in mathematics / f. viseu & i. b. oliveria 289 stimulates students to explore and make sense of the mathematical activities that are developed (brendefur & frykholm, 2000; nicol, 1999). the way in which the teacher promotes verbal or written communication determines how the students voice their doubts and justify their ideas. sharing and comparing the processes results in ways of thinking that promote the significance of mathematical concepts. when the students establish conjectures and discuss the activities with their colleagues, new collaborative knowledge is developed, which ensures that mathematics is seen as a normal human activity (nctm, 1994). brendefur and frykholm (2000) classify classroom communication as uni-directional, contributive, reflective and instructive. in uni-directional communication “teachers tend to dominate discussions by lecturing, using essentially closed questions. they create few opportunities for students to communicate their strategies, ideas and thinking” (p. 126). in contributive communication the teacher gives the students opportunities “to discuss mathematical tasks with one another, present solution strategies, or help each other to develop solutions and appropriate problem solving strategies” (p. 127). in reflective communication what the students and teacher do “becomes the subject for discussion. reflective discourse often occurs when students try to explain or refute conjectures offered by their peers” (p. 128). in instructive communication the interactions that occur in the classroom help the students to construct and modify their mathematical knowledge. by verbalising their ideas, the students allow the teacher to understand their thinking processes, their effectiveness and limitations, to alter the way the lesson develops and to draw conclusions for future situations. apart from uni-directional communication, the other types describe forms of communication to stimulate students to share their ideas, their thoughts, conjectures and mathematical solutions. this is precisely the direction indicated by the new basic education mathematics syllabus when it recommends that “students must be able to express their ideas and to interpret and understand the ideas that are presented and participate constructively in discussions about ideas, processes and maths results” (ministry of education, 2007, p. 8). as for asking questions, the nctm (1994) considers that the questions that the teacher formulates help students to make sense of their activities, to be able to decide whether something that is or is not mathematically correct, to speculate, argue about and resolve problems and to link mathematical ideas and applications. moyer and milewicz (2002) identified various strategies for questioning that the teacher can adopt: (1) follow the questions as planned, whereby the teacher passes from one question to another with little consideration for the students’ answers; (2) teach and transmit, whereby the teacher plants questions to direct the students’ answers and stops asking questions in order to teach the concept to be tackled without encouraging the students to think or frame a response; (3) ask questions and give follow up, whereby the teacher uses different types of questions to find out more about the ideas of the students and to meet their questions with other relevant questions, thus giving them the idea that their response is still open for discussion; (4) only question a wrong answer; (5) non-specific questioning, when the teacher follows up the students’ answers but with questions that indicate a lack of specificity; and (6) competent questioning, when the teacher listens to the students’ answers and uses them to gather information about their way of reasoning. the use of each of these strategies shows the importance that a teacher gives to questioning in the activities carried out in the classroom. besides the right questions at the right time, nicol (1999) says that teachers should know how to listen to their students – by paying attention to their words and trying to understand their contributions – and to respond to their actions constructively. a good question represents the difference between constraining international electronic journal of elementary education, 2012, 4(2), 287-300. 290 students’ thinking and encouraging new ideas, and between their retaining trivial facts or constructing meanings (moyer & milewicz, 2002). the nature of the tasks in promoting communication in mathematics one good way to encourage maths communication is to provide the students with a learning environment that arouses their active participation. one way of doing this is to use challenging tasks (ponte, 2005). stein and smith (1998) draw attention to the importance of choosing tasks that challenge the students to think, justify, explain and find meaning and which stimulate them to make connections. the nctm (1991) takes the same stand by recommending that the tasks permit students to actively “explore, formulate and test out conjectures, prove generalisations and discuss and apply the results of their investigations” (p. 148). the nature of the tasks can have implications for how students are involved in the construction of their mathematical knowledge. ponte (2005) distinguished tasks according to their degree of difficulty (low/high) and their structure (closed/open). though exercises and problems may be of a closed structure, they will differ in their degree of difficulty. exercises have a low degree of difficulty, which appeals to the mechanisation and repetition of the processes in pursuit of the intended response. problems have a higher level of difficulty since they translate non-routine situations for which students do not have an immediate solution process and which can be solved by various methods. these characteristics are also present in investigative task that according to ponte requires students to participate in the “specific formulation of their own questions to be solved” (p. 15), to search for regularities, establish and test conjectures, argue and communicate their processes and their conclusions. the tasks in which the students carry out a set procedure that is memorised in a routine way are, for stein and smith (1998), much less rewarding than the tasks that challenge the students to establish connections between mathematical concepts, to reason and to communicate mathematically. osana, lacroix, tucker and desrosiers (2006) stress the use of open-ended tasks which favour students’ involvement in class activities and encourage them to explore and investigate, increase their motivation for generalisation, look for models and links, communicate, discuss and identify alternatives. however, the selection of tasks does not in itself guarantee effective teaching. teachers are crucial to determining the “aspects to be underlined in a given task; like organising and guiding the work of the students; what questions to ask, so as to challenge the different levels of skills of the students” (nctm, 2007, p. 20). it is important for the students to “work on mathematical tasks that set up relevant subjects for discussion” (nctm, 2007, p. 66). discussion is thus the next step after the implementation of the set tasks, thus making it possible for the student to think, rationalise and communicate mathematically. stein, engle, smith and hughes (2008) set out five practices that promote discussion: (1) anticipating the students’ likely answers to cognitively demanding mathematical tasks; (2) monitoring the students’ answers to the tasks during the exploratory phase; (3) selecting some students to present their mathematical responses during the discussion phase; (4) intentionally sequencing the students’ responses; and (5) helping the class to make mathematical connections between the students’ different responses. (p. 321) these discussion practices contribute to teachers using the students’ answers, so as to develop the mathematical understanding of the class. it is within this framework that this study intends to analyse how maths teachers, promote mathematical communication in the classroom through open-ended tasks, during the implementation of the methodological guidelines for the current maths programmes of the third cycle of basic education. open-ended tasks in the promotion of classroom communication in mathematics / f. viseu & i. b. oliveria 291 method this study is about an experiment devised by two teachers, mariana and inês, on the methodological guidelines resulting from the reformulation of the basic education mathematics programme. to increase the implementation of these guidelines, the ministry of education created a national network for monitoring it, which consists of university lecturers and teachers representing a group of schools from the same geographical area. these representatives meet regularly with teachers from schools in their area to review and discuss the theoretical assumptions that underlie the changes made in the basic education mathematics programmes. this move had an impact on the professional practice that teachers develop in their schools. the individual work has led to work on an equal footing in the preparation, observation and discussion of lessons, sharing the experiences with their students and discussion of texts on the field of mathematics education. this is context of the work that inês and mariana undertook to implement the curriculum in a 7th year class. mariana was the class teacher and inês the representative of the schools in their geographical area; both have 13 years teaching experience. when implementing the methodological guidelines of the revised programme, they paid special attention to the relationship between communication in the mathematics classroom and the tasks the students are set. they therefore chose a topic, sequences and regularities, for which the new programme specifies a different approach from the previous one. the two teachers together prepared two classes, the first and last on this topic, with open-ended tasks. this topic deals with the general term of a numerical sequence, representation and algebraic expressions. in the first class, mariana implemented an exploratory task, while the second class involved a task of an investigative nature. the exploratory task – v flight – provides patterns with geometric figures that change at each position according to a rule. the work might be very intuitive at first, describing her natural reasoning, with the use of diagrams, the submission of calculations or the use of symbols. students can use different strategies to characterise a next term: analysis of the previous figures, analysis of regularity in the associated numerical sequence and decomposition of the figure into parts. when characterising a distant term, students can compare the figure number with the number of points in this figure. the investigative task explorations with numbers lets different paths be followed to obtain various regularities and numerical relationships. students are challenged to hypothesise, test and reformulate their conjecture and generalise. this task promotes written communication as students are asked to describe the regularities identified using natural language and mathematical language. the task provides an opportunity for students to express themselves orally in student-student dialogue on the regularities found, when they work in pairs or groups. students have to indicate clearly and use a mathematical language appropriate to their findings so that they all understand and can verify that these are always valid. they can also see if the same conclusions are reached, if other ones are reached, or if, based on the findings of their colleagues, they can identify new regularities. the two classes were observed by inês, whose attitude was that of non-participant; she focused on the interactions between mariana and her students during the collective discussion. the time between the two classes was about a month, to ascertain: (1) the development of student participation in classroom discussions and their involvement in learning the sequences and regularities topic, and (2) the teacher’s progress in monitoring these discussions, after meetings with inês when they read and discussed texts on mathematical communication in the classroom. international electronic journal of elementary education, 2012, 4(2), 287-300. 292 following a qualitative interpretative methodology, data were collected through two audiotaped interviews that inês held with mariana one before the experiment (i1) and one after (i2) -, the observation of two classes by inês (co1 and co2), recorded on video, of the discussion of these classes (dco1 and dco2) and activities produced by the students. from the analysis of data collected by these methods, the information was organised thus: (1) mariana’s class involving an exploratory task; (2) mariana’s class involving an investigative task; (3) mariana’s views about the influence of the tasks on classroom discussion. results mariana has been a teacher of mathematics for 13 years a profession that she thought of following in her ninth school year, as she very much liked this subject. in the current academic year (2009/10) her job was to co-ordinate the third cycle in the implementation of the mathematics syllabus of basic education (msbe) in her school and to teach a seventh year class and an education course class. her 7th year class consisted of 13 boys and 6 girls. it was an uninterested class; 47% of students had failed mathematics at the end of the 1st period. from her professional career mariana highlights the moment when the test became mandatory for 9th year students. she explains this because she sees exams as a way to regulate the practice of maths teaching and to encourage varying the type of tasks. for example, she says that prior to mandatory examinations the tasks that prevailed were mainly "exercises" (i1). realising that national exams have open-ended tasks, she saw that "there could not be more of the same, because the exams involve more than just exercises" (i1). with regard to the changes that have occurred in the pedagogical practices of teachers she stresses that “the collaborative work that has emerged over the past three years and the receptivity of teachers has opened the classroom door to other colleagues” (i1). this year, more than any other, she worked a lot with her colleagues. thanks to the implementation of msbe, she met periodically with her colleagues, who are also teaching the seventh year. at these meetings they prepared worksheets and tests, studied and defined strategies and debated the difficulties encountered in implementing them. one of mariana’s classes involved an exploratory task to start the topic of sequences and regularities, mariana selected the task ‘flight in the v (formation) of ducks’ because it allows: (i) checking if a number is a term in a sequence, (ii) determining the order of a known term, (iii) understanding the notion of a general term of a sequence, and (iv) formulating and testing conjectures. in the sequence that follows each figure represents a flock of ducks and each dot represents one of the ducks in the flock. here are the first four terms: answer the following questions and state your reasoning using words, diagrams, calculations or symbols. 1.1. how many dots does the next figure of this sequence have? 1.2. how many dots does the hundredth figure (term of the order 100) of this sequence open-ended tasks in the promotion of classroom communication in mathematics / f. viseu & i. b. oliveria 293 have? 1.3. is there a figure with 86 dots in this sequence? if there is, indicate the order to which it corresponds. 1.4. is there a figure with 135 dots in this sequence? if there is, determine the order to which it corresponds. 1.5. write a rule for determining the number of dots in each figure of this sequence. 1.6. write an algebraic expression that expresses the rule described in the previous question. figure 1: exploratory task given to the students, about sequences and regularities. the teacher started the class by organising the students into groups, and by instituting the following rules: “tell your ideas to each other (…) at the end we are going to discuss the conclusions and share the different strategies” (co1). next, she gave the task to all the students and delivered to each group an ohp transparency on which to record their responses. the students began to solve the tasks without any explanation from the teacher, who was busy with the management of the group work: “exchange ideas, explain your reasoning and only then write your answers on the transparency” (co1). when the students showed they were having difficulties, mariana asked questions to guide them in the activity that they were carrying out, as is shown in the following example: student: the rule is to add 2 to the previous figure. teacher: in fact, adding 2 to the number of dots in the previous figure does allow you to discover the number of dots in the next figure but will this be a practical strategy to find the number of dots in the hundredth figure? look at the various figures. what other characteristics do they have? what can we use to represent the given information? (co1) in the discussion phase, when the spokesman of one group presented its solution, the rest of the students in the class did not intervene spontaneously. the attention of the teacher centred on the answers that were given by each group spokesman and she confirmed them with statements like “very good” (co1). when the answers were wrong, mariana asked another group for its answer. after the presentation of the solutions by the group spokesmen, the teacher would interpret the solution by repeating what the student had explained: student: i made 101+100 = 201. the explanation is that in figure 1, 2+1 = 3; in figure 2, 2+3 = 5; in figure 3, 4+3 = 7; thus in figure 100, 101+100 = 201 teacher: do you see what he found out? do you see how? perhaps it was geometrical, wasn’t it? is there another group that also saw this characteristic? who did? was it you tiago? did anyone see it another way? diana: we drew a diagram. on one side we put 101 and on the other 100 teacher: this group used the same reasoning only they drew a figure. in figure 100 they imagined that on one side of the v they had 101 and on the other they had 100. so 2011100100 =++ . (co1) international electronic journal of elementary education, 2012, 4(2), 287-300. 294 the determination of the number of dots in figure 100 helped some students to formulate and test conjectures. only two groups indicated the general term ( 12+×n and 1++cc ). in this generalisation, one of these two groups turned to symbolic representation and the other group expressed their reasoning through symbolic representation and a diagram. mariana’s class involving an investigative task. for this class, mariana selected the task explorations with numbers2 so that she could encourage mathematical communication between the students in the discovery of numerical regularities and relationships. look at the following table: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 … … … … answer the following questions. give your reasoning using words, diagrams, calculations or symbols. 1.1. continue the representation of the table presented above until you obtain the number 40. 1.2. assume that this table continues indefinitely. identify the regularities that you manage to find. 1.3. in this table can you predict in which column you will find the number 64? and in which line? 1.4. can you predict in which column you will find the number 99? and in which line? explain how you proceeded. 1.5. taking any number, can you predict in which column and in which line it will be found in this table? explain your answer. figure 2: investigative task proposed to the students about sequences and regularities. with this task mariana wanted the students to guess in which line and in which column a specific number would be found, so that they would manage to generalise for any number. as she wanted to involve the students in the discussion about the task, mariana set the rule that whoever “does the presentation must involve the others and these others must ask questions, request explanations and, if they do not agree with what is being said or wish to add something, that they should intervene” (co2). the first regularity encountered by the students was the multiples of 4, which encouraged the students to “look for other multiples” (co2). when she found that they were only concerned with discovering multiples, the teacher encouraged them to look for “another type of number, one that we have already talked about in class (…) you have to communicate, describe the regularity encountered and write the algebraic expression” (co2). when presenting their activities, the students identified some regularities and showed the general term of the sequences that they had found: figure 3: students’ solution of some questions from the explorations with numbers task. 2 ponte, j. p., branco, n., and matos, a. (2009). álgebra no ensino básico. lisboa. ministério da educação, dgidc. open-ended tasks in the promotion of classroom communication in mathematics / f. viseu & i. b. oliveria 295 in relation to the question “can you predict in which column you will find the number 64?” one student gave his group’s answer on the interactive board, which exemplified the sort of communication that livened up some parts of the class: diana: the number 64 is in the first column and on row 17, because the multiples of 4 are in the first column and 64 is multiple of 4 teacher: read diana’s answer. student: why is it on line 17? diana: i always added 4 to the numbers in the first column and arrived at 64. it gave 17. student: it would be 4x16. joão: i did the table. i was writing the numbers and i got to 64. teacher: this is not the purpose of the question; it is to predict and not to confirm. did anybody find a strategy for prediction? somebody did? did anybody manage to predict why it is line 17? i will have to give a hint…. student: wait a minute… students: i know! i know! teacher: work on your idea. think better! keep thinking and check your strategy! [mariana gave them a little more time to think] teacher: diana, have you got it yet? diana: no. [the teacher asks diana to sit down and lets rui speak] teacher: those who do not understand ask rui. rui: 4x16 gives 64. but the 4 isn’t on the first line, 4 only comes on the second line. students:i don’t understand. teacher: you didn’t write what you’ve said … rui: it’s the way it is, 4x16 = 64. since 4 doesn’t come on the first line, this gives 17. teacher: who doesn’t understand? ask rui questions. those who already know can help rui to explain (the answer). renato: i don’t understand your explanation. rui: 64 is in the first column. but, as the 4 isn’t on the first line, you have to add 4 by 4, 16 times from 4 [the student exemplifies with gestures next to the table]. afterwards you have to add the first line. thus it’s on line 17. teacher: anybody want to add anything? is it clear now? students: yes. teacher: good… but we need to complete the answer. you get there without me giving a hint. (co2) the teacher tried to get the students to ask the colleague that presented their answer one of the specified questions. the question was a form of contributive communication sustained international electronic journal of elementary education, 2012, 4(2), 287-300. 296 by student-student, teacher-student and student-class interaction. there were only a few spontaneous interventions from the students, which raised the question: what happened to the requested explanation? the students begged their colleague for explanations instead of asking mariana. synthesis of mariana’s two classes the way the communication was promoted in the two maths classes differs, as noted in the interaction that develops between the teacher and students in each class. table 1. promoter of the communication in the classroom. lesson 1 lesson 2 initiations request response request for explanation teacher teacher teacher and students students answers answer explanations students students and teacher students students reconceptualisation reaffirm expand reformulate validate teacher teacher students students students students and teacher in the first class, when students presented their activities, their colleagues generally did not participate willingly. the teacher tried to get the students to justify their answers. but, was mariana who validated almost all the answers and who interpreted the students’ presentation to the class. after a student gave his explanation the teacher tended to reaffirm what this student said. when questions arose, students directed them at the teacher. in the second class, the students were more attentive to the presentations of their colleagues and student-student interaction was more frequent. the divergence of answers that the task provided meant that students gave presentations that had not been explained. the teacher took care not to validate the answers and so created space for the students to do it. when the students asked the teacher, she sent the question back to the class, which meant that they sometimes addressed and asked colleagues who were presenting their activity. mariana’s views about the influence of the tasks on classroom discussion. in terms of the methodological guidelines that have emerged from the reformulation of the mathematics syllabus for basic education, mariana pointed out “the type of tasks that are different from those usually implemented and the topics that are approached in an exploratory way by discovery” (i1). for mariana, the effect of the exploratory tasks on the learning of the students raised “many doubts about whether we would get better results, whether the students would be more competent mathematically (…) we have a lot of work and little supervision” (i1). she assumed that she would not always make “the students interact with each other, perhaps because they aren’t used to it” (i1). before the experiment, mariana recognised that the form of communication that predominated in her classes was uni-directional communication sometimes with interpolations when the students would be asked “to justify their reasoning and explain how they think” (i1). when analysing the first class observed, mariana identified critical aspects of her action, such as a tendency to repeat what the students said and did and the difficulty of encouraging the students to discuss their activities with one another: open-ended tasks in the promotion of classroom communication in mathematics / f. viseu & i. b. oliveria 297 the students explain and there’s something i always do, but i don’t know if it’s good or bad; the students explain and i repeat their explanation, i don’t know if i should do that. another thing that i think is that i don’t promote student-student communication, which i think is very difficult, but some people think it can be done. student-class and studentteacher communication exists, but student-student the type where one student puts their hand up and asks another – there’s none of that. it may have happened in groups but in groups it is very difficult to evaluate, we would have to monitor each group closely. they put their hand in the air to give their answers or when a student answers badly, but they don’t ask questions. i have to improve student-student communication. (dco1) the teacher recognised that the students did not question their colleagues’ answers and she questioned the way that she promoted the confirmation of the students’ answers: “i always confirm, never ask if they agree... i should ask for another strategy leaving out the previous automatic ratification” (dco1). she was aware, above all, that she asked questions from her point of view and that she gave little time for the students to respond to what she ended up doing. in the class discussion of a presentation by the spokesman of one group, mariana was able to solicit an explanation from this group and later on she was able to direct another group to present an explanation that would contradict or supplement the one given. from the analysis of the second class, mariana identified the initiative of the students in stimulating communication between them without her having to intervene, which in her view she did not manage in her previous classes: “i had the students communicating more student-student” (dco2). although she had used easy questions, she recognised that the question of generalisation was only understood by some students, which she would have widened if “in the previous questions, she had prepared them better for managing to generalise” (dco2). time limited her action and this prevented her from “exploring what the students did a little more” (dco2). when comparing the two classes, the teacher considered that “in the first the discussion was very much centred on me, it was me that confirmed” (i2), while “in the second class i gave more opportunity to the students but it made the class more time-consuming” (i2). including open-ended tasks in the classroom has implications for the care and time necessary for their preparation. carrying out of this type of task ensured that mariana paid heed to what the students said and did and she had to look for ways of involving them in class discussions: i liked to prepare the class, where i would have a place for discussion and not simply the preparation and solving of the task, more frequent in my day-to-day work. i was aware of the importance of frequent class discussions between the students and paid attention to my efforts to promote these discussions. during classes i asked myself: are they communicating among themselves? what questions should i put? and if this happens, what should i do? should i wait a bit longer? what example should i choose to stimulate discussion? (i2) besides the structure of the task, the rules that the teacher established for carrying out the student activities and the conceptions she had about the teaching of mathematics tended to influence how the students engaged in the class activities: all the same it was very difficult to involve many students in mathematical discussions and, while there was some progress between the first and second classes, the students did not communicate with each another but limited themselves to setting out their ideas. it was me that was always intervening, essentially by asking pointed questions and finishing off by confirming the answers. the student interventions coming from the class were short (presentation of their answers) and limited, since the interaction was predominantly from teacher to student. however, there were times when the students international electronic journal of elementary education, 2012, 4(2), 287-300. 298 questioned their colleagues and gave valid reasons, leaving me in a less prominent position. all the same, the students’ contributions came close to influencing the course of the lesson with some inspired discoveries and questioning about them by the others. inquiry questions predominated in discussions about the task of this class up to question 1.5, which is not my normal practice, while for question 1.5 focusing questions were asked, which is more normal for me. (i2) the completion of the two tasks in the classroom enabled mariana to perceive the influence that her conceptions had on the way that she promoted communication: uni-directional communication with little space for the students to intervene and, when they did, it was to present answers gave way to contributive communication during the presentation and discussion of the students’ solutions, which tended to influence the course of the class. her openness to innovate in her practice contributed to this change and it also helped her to read and discuss with colleagues texts about the didactical aspects of teaching. discussion of importance to the translation of the methodological guidelines of the current school syllabuses are the nature of the tasks that teachers should adapt for their classes, and particularly the attention to be given to student activities, as this gives an understanding of the way others think. the conceptions that teachers develops in their professional career about the teaching of mathematics tend to hamper the implementation of these guidelines (ponte, 1992). willingness to innovate in teaching practice and a critical analysis of it help to overcome some obstacles, as observed in the teaching practices of mariana in relation to how she fostered communication with her students. although she considered that discussion of classroom activities is one factor that stimulates student learning, the teacher did recognise the difficulty students have with the presentation of alternatives to the proposals presented by their colleagues. this difficulty tends to be due to the habits that students develop in learning environments where the authority of the teacher in the management of classroom activities prevails (moyer & milewicz, 2002; nicol, 1999). it is the belief that teaching is a uni-directional process of transmitting information to students in a way that enables them to reproduce what the teacher says and does (brendefur & frykholm, 2000). before completing the exploratory task, mariana questioned the importance ascribed by the methodological guidelines to this type of task in student learning, because of the time it requires, which would indicate a preference for repetitive tasks of a lower cognitive level (stein & smith, 1998). in this way she became aware of her fears about organising the students in groups in her classes. such fears were overcome in the class in which she proposed the exploratory task covering the topic ‘sequences and regularities.’ as the students were not used to working in groups, the teacher stated some rules about how the students should communicate their ideas to the others. she herself realised that it was not the rules that she defined or the nature of the tasks that were chosen that really altered the atmosphere in the classroom. in the first lesson observed, teacher activity prevailed to the detriment of student activity. when a student from one of the groups presented its solution, the others did not intervene. mariana tended to explain what the students were doing by repeating what they said, and to ask planned questions (moyer & milewicz, 2002). after the class with the exploratory task, the teacher recognised in this class that she repeated what the students said and did. she did this for the benefit of students who did not question their colleagues who had presented their solutions. although osana et al. (2006) consider that tasks of an open nature stimulate students to engage in class activities, nicol (1999) stresses the importance of teachers knowing how to listen to their students in order to encourage them to discuss the classroom activities. only then, as suggested by moyer and milewicz open-ended tasks in the promotion of classroom communication in mathematics / f. viseu & i. b. oliveria 299 (2002), can the teacher use student responses to collect information about their way of thinking. the reading and discussion of texts about mathematics education with other maths teachers made it clear that mariana needed to hear more about what her students said and that she needed to develop in them the habit of asking their colleagues when they had doubts or had other strategies for solving the task. this is what happened in the class with the investigative task. the students put questions to their colleagues, who answered, and when they had doubts they did not put them to the teacher but to their colleague who was presenting his group’s solution, with the aim of understanding the answers he gave. as the prevailing forms of communication tend to move away from uni-directional, students contribute to the course of the class and give meaning to learning (brendefur & frykholm, 2000). these are the guidelines that are emerging from the current programme of the 3º cycle (ministry of education, 2007). comparing the two classes, the teacher confessed that in the first class the students did not communicate with one another; there was no direct communication between them, and that she herself neither give them time nor stimulated student-student communication. she realised the need to try to get the students to present their ideas and to ask each other questions by taking on the role more of a moderator (stein et al., 2008). mariana admitted that it is “very difficult to involve many students in mathematical discussions but i noted some improvement from the first class to the second one, when there were times when the students questioned colleagues and confirmed reasoning, while i stayed more in the background” (i2). in the second class the teacher felt that “the contributions of the students came closer to influencing the course of the class with inspired discoveries and questioning from the others about these discoveries” (i2). the change in the way that she encouraged student communication gave the impression that it was due, as advocated by stein and smith (1998), to the higher cognitive level of the task that she proposed, which stimulated the discussion and formulation of conjectures. but also it was due to the attention that the teacher gave to the students’ answers. consideration of what the students say and do must become part of a classroom culture that is nurtured in the earliest of school years and should persist in the more advanced years. only then will the students understand that their involvement in class activities is not only enriching their own learning but it is also enriching the learning of their colleagues. the discussion of texts on mathematical communication with a colleague and the divergent nature of the open-ended tasks played a major part in bringing about this change. . . . references brendefur, j., & frykholm, j. (2000). promoting mathematical communication in the classroom: two preservice teachers’ conceptions and practices. journal of mathematics teacher education, 3, 125153. ministério da educação (2007). programa de matemática do ensino básico. lisboa: ministério da educação direcção geral de inovação e de desenvolvimento curricular. moyer, p. s., & milewicz, e. (2002). learning to question: categories of questioning used by preservice teachers during diagnostic mathematics interviews. journal of mathematics teacher education, 5, 293-315. international electronic journal of elementary education, 2012, 4(2), 287-300. 300 national council of teachers of mathematics (1991). normas para o currículo e a avaliação em matemática escolar. lisboa: apm e iie. national council of teachers of mathematics (1994). normas profissionais para o ensino da matemática. lisboa: iie e apm. national council of teachers of mathematics (2007). princípios e normas para a matemática escolar. lisboa: apm. nicol, c. (1999). learning to teach mathematics: questioning, listening, and responding. educational studies in mathematics, 37, 45-66. ponte, j. p. (2005). gestão curricular em matemática. in gti (ed.), o professor e o desenvolvimento curricular (pp. 11-34). lisboa: apm. ponte, j. p. (1992). concepções dos professores de matemática e processos de formação. in m. brown, d. fernandes, j. f. matos, & j. p. ponte (eds.), educação matemática: temas de investigação (pp. 185239). lisboa: instituto de inovação educacional. osana, h., lacroix, g., tucker, b. j., & desrosiers, c. (2006). the role of content knowledge and problem features on preservice teachers’ appraisal of elementary tasks. journal of mathematics teacher education, 9(4), 347-380. stein, m. k., & smith, m. s. (1998). mathematical tasks as a framework for reflection: from research to practice. mathematics teaching in the middle school, 3(4), 268-275. stein, m. k., engle, r. a., smith, m. s., & hughes, e, k. (2008). orchestrating productive mathematical discussions: five practices for helping teachers move beyond show and tell. mathematical thinking and learning, 10, 313-340. 333 © 2022 published by kura education & publishing. this is an open access article under the cc byncnd license. (https://creativecommons. org/licenses/by/4.0/) copyright © www.iejee.com issn: 1307-9298 international electronic journal of elementary education march 2023, volume 15, issue 4, 333-342 perceived parenting styles and emotional control as predictors of peer bullying involvement tina pirca,*, sonja pečjakb, anja podlesekc, mateja štirnd abstract introduction in our study we tested a model of the relationships among students’ perceived parenting styles, their emotional control and peer bullying involvement, since family characteristics are understudied and unclear in comparison with individual and school factors of peer bullying. our sample included 202 students from 7th and 8th grade from 14 lower secondary schools. the resulting model showed that the authoritative parenting style positively predicted emotional regulation and observation of bullying. the authoritarian parenting style positively predicted relational and physical bullying with teasing and observation of bullying, and negatively emotional control. emotional control as a mediating variable negatively predicted all forms of involvement in peer bullying. the results indicate the importance of promoting self-regulation skills in adolescents. some practical implications for parents and school staff are discussed. peer bullying at schools is described as aggressive, intentional acts carried out by a group or an individual repeatedly and over an extended period of time against a victim who cannot easily defend himor herself (olweus, 1993). newer definitions add an imbalance of power (physical or psychological) between the victim and the bully (e.g. volk et al., 2017). research suggests that it is a widespread phenomenon with serious shortand longterm consequences for students. a meta-analysis by gini and pozzoli (2009) revealed that students involved in peer bullying are at higher risk of psychosomatic problems, low emotional adjustment, poor peer relationships, health problems, and problems with academic adjustment. however, peer bullying is not only the result of the individual characteristics of students who are directly involved in it, but, following bronfenbrenner's (1979) socio-ecological systems theory, there is an interaction among individual, family, and school factors, as well as the influences of the media and wider society. in our study, we focused on parental style as one of the family characteristics that predict peer bullying involvement. this is based on nocentini et al.’s keywords: parenting styles, emotional control, peer bullying, adolescents received : 9 december 2022 revised : 6 april 2023 accepted : 17 april 2023 doi : 10.26822/iejee.2023.304 a,*corresponding author: tina pirc, department of psychology, faculty of arts, university of ljubljana, ljubljana, slovenia. e-mail: tina.pirc@ff.uni-lj.si orcid: https://orcid.org/0000-0002-9248-0608 b sonja pečjak, department of psychology, faculty of arts, university of ljubljana, ljubljana, slovenia. e-mail: sonja.pecjak@ff.uni-lj.si orcid: https://orcid.org/0000-0003-0402-3325 c anja podlesek, department of psychology, faculty of arts, university of ljubljana, ljubljana, slovenia. e-mail: anja.podlesek@ff.uni-lj.si orcid: https://orcid.org/0000-0003-2987-8338 d mateja štirn, institute for psychological counselling and educational development projects, ljubljana, slovenia. e-mail: mateja.stirn@isainstitut.si orcid: https://orcid.org/0009-0002-4106-0192 334 march 2023, volume 15, issue 4, 333-342 (2019) identification of three groups of family variables – contextual family variables (e.g. parents’ mental health, domestic violence), relational family variables (e.g., authoritative parenting, communication), and parents’ individual processes (e.g. parental beliefs about bullying, parental knowledge about strategies to manage bullying situations). although these factors have received more attention in recent years (nocentini et al., 2019), they remain understudied compared to research addressing individual (peer) and school factors. because previous research demonstrates that emotional regulation is one of the predictors of adjustment throughout development (cole et al., 2017), and is one of the links between parenting practices and child adjustment (eisenberg et al., 2004), this was another construct we considered important in examining the relationship between family characteristics and peer bullying involvement. furthermore, in a longitudinal study of adolescents, dickson et al. (2019) found that less favourable parenting was associated with impaired emotional regulation, which predicted a higher likelihood of perpetration of peer bullying and victimization by students in the following year. our goal, therefore, was to examine the relationship between students' perceived parental style (family characteristic), students' self-perceptions of emotional control (individual characteristic), and frequency of involvement in peer bullying as perpetrator, victim, or observer. parenting styles baumrind (1971) proposed three main parenting styles, which are a combination of parental control and parental warmth. she described authoritative parents as controlling and demanding on the one hand and warm, rational, and receptive to the child's communication on the other; authoritarian parents were described as distant, controlling, and less warm; and permissive-style parents were described as noncontrolling, nondemanding, and relatively warm. in her later work baumrind (2005) proposed that parental behaviour, as perceived by children, has two dimensions – demandingness (i.e., controlling behaviour, setting limits and expectations for the child) and responsiveness (i.e., responding to the child's needs, supporting and maintaining warm communication). thus, another parenting style was added to the existing three – the disengaged parent who is neither demanding nor responsive. nonetheless, and despite the fact that some previous researchers have expanded the study of parenting variables beyond parenting styles (e.g., gómezortiz et al. (2014) added parental use of humour and autonomy support; georgiou and stavrinides (2013) introduced parent-child conflict, parental monitoring, and child disclosure in examining the relationship between family characteristics and peer bullying), for the purposes of this study, we drew on the long tradition of the three parenting styles originally proposed as predictors of emotional control and peer bullying involvement. several studies have consistently shown that children of authoritative parents are better adjusted (steinberg et al., 1995) and have greater academic and psychosocial competence (mahapatra & batul, 2016). parenting styles and emotional control fosco and grych (2012) point out that children's emotional regulation is initially shaped by their first interpersonal context, the family, and that children exhibited more emotional regulation when their parents showed warmth and sensitivity to their emotions. similarly, in a longitudinal study with adolescents, herd et al. (2020) found that a positive family environment, as measured by parents' emotional regulation, parenting practices, and the quality of the parent-adolescent relationship, was associated with increases in emotional regulation on later measures. in addition, morris et al. (2007) argued that the family environment influences the development of emotional regulation in three ways: i) through observation, ii) through specific parental practices and behaviours, and iii) through the family emotional climate (quality of attachment, parenting styles, family expressiveness, etc.). specifically, the authors found that parental responsivity and negativity, the most salient features of parental styles which regard to the current study, influence children's emotions, emotional competence, and emotional regulation. in particular, authoritative parenting helps children to acquire more constructive emotion-coping strategies (chan, 2011) and to have more effective emotional regulation (mahapatra & batul, 2016). parental styles and peer bullying involvement several studies examined the relationship between parenting style and their children's involvement in peer bullying as perpetrators or victims. georgiou (2008) reported that parenting style was associated only with victimization but not with perpetration of peer bullying, i.e., children of permissive mothers were more likely to be victims than children who received other parenting styles. malm and henrich (2019) reached somewhat different conclusions in a longitudinal study – namely that poor relationships between mother and child were found to predict bullying perpetration, but not victimization. alizadeh maralani et al. (2019) found that an authoritarian parenting style predicted the role of perpetrators in peer bullying, a permissive style the role of victims, and an authoritative parenting style was characteristic of students who were not involved in peer bullying. charalampous et 335 perceived parenting styles and emotional control as predictors of peer bullying involvement / pirc, pečjak, podlesek & štirn al. (2018) reported that the authoritative parenting style predicted perpetration, but also victimization in bullying. conversely, the authoritative (or as the authors call it, flexible) parenting style was a negative predictor of all roles in peer bullying, whereas the permissive style predicted perpetration only. martínez et al. (2019) reported different findings, as in their study permissive parenting style was found to be a protective factor for experiencing peer bullying, while authoritarian parenting style was a risk factor for this. in a meta-analytic study, lereya et al. (2013) found that positive parental behaviors (authoritative parenting, parent-child communication, parental involvement and support, supervision, warmth, and affection) protected children from becoming victims of peer bullying, while negative parental behaviors (abuse/ neglect, maladaptive parenting, and overprotection) predicted a greater risk of children becoming victims or bully/ victims. interestingly, broll and reynolds (2021) found no association between parenting styles and bullying offending or victimization. therefore, the results of the aforementioned studies and the studies included in the systematic review by nocetini et al. (2019) suggest that the association between parenting styles and bullying involvement is inconclusive. in addition, we found no studies that considered the association between parenting styles and the role of witnesses to bullying among peers (observers). emotional control and peer bullying involvement among the important aspects of successful and adapted functioning in children is emotional regulation. thompson (1994) defines this regulation as the internal and external processes involved in initiating, maintaining, and modulating the occurrence, intensity, and expression of emotions. one of the features of emotional regulation, which was also used in our study, is emotional control, which rueda et al. (2022, p. 6) define as "an ability to regulate and modify emotions according to the circumstances in which the person finds themselves, for example, to overcome obstacles in everyday life." in the context of peer bullying, children with low emotional regulation have been shown to repeatedly violate social norms and rules, and were at risk of developing psychological and social maladjustments (eisenberg et al., 2004). mahady wilton et al. (2000) suggest that poor emotional regulation may be a risk factor for chronic victimization. this is also consistent with the findings of an emotional intelligence study of a sample of australian adolescents, in which lomas et al. (2012) found that low emotional control was associated with more victimization. blake et al. (2012) also indicated that adolescents with emotional dysregulation, which includes immaturity, lack of selfcontrol, and poor social skills, may be at risk for peer victimization. however, bettencourt et al. (2013) noted that such characteristics can also lead to bully/victim and bully roles. the aim of the study the aim of our study was to investigate the relationships among adolescent students' perceived parenting styles, their perceived emotional control, and their involvement in peer bullying. according to the findings of previous studies presented in the introduction section, we hypothesized a model in which parenting styles are associated with emotional control, which is in turn associated with involvement in peer bullying. figure 1 the tested model of the relationships among parental styles, emotional control and peer bullying involvement. method participants our sample included 202 7th (46.5%) and 8th grade (53.5%) students from 14 lower secondary schools from different statistical regions in slovenia. girls represented 51.5% of the sample. the average age of the students was 12.94 years (sd = 0.71). instruments students reported their perceptions of parenting styles, emotional control, and involvement in peer bullying. to examine parenting styles, we used the parenting styles and dimensions questionnaire-short version (psdq-short version; robinson et al., 1995; slovenian translation and adaptation of the version for parents hacin, 2019; the version adapted for children was created by the authors). the questionnaire contains 32 items in which students indicate on a 5-point scale how often their parents behave in a certain way (1 never, 2 rarely, 3 occasionally, 4 often, 5 always). the questionnaire measures three parenting styles: authoritative, 15 items (e.g., my mother/father comforts me and understands me when i am upset, α = 0.92); authoritarian, 12 items (e.g., my mother/father explodes with anger at me, α = 0.82); and permissive, 3 items (e.g., my mother/father announces punishments but does not carry them out, α = 0.56). confirmatory 336 march 2023, volume 15, issue 4, 333-342 factor analysis showed the marginally acceptable fit (according to the criteria listed by hu and bentler, 1999) of the factor structure proposed by the authors of the original questionnaire with regard to our data (cfi = 0.937, tli = 0.932, rmsea = 0.058, 90% ci [0.051– 0.066], srmr = 0.089). items 15 and 24 were excluded from calculating scale scores because of their high modification indices. we also decided to omit the permissive style scale from further analyses due to its low alpha reliability coefficient. emotional control was measured with items taken from the emotion regulation index for children and adolescents – self report (erica-s; macdermott et al., 2010; translation and adaptation by romih and košir, 2018). the questionnaire consists of 16 items that the participant rates on a 5-point likert scale (1 – strongly disagree, 3 – undecided, 5 – strongly agree). the subscales measure emotional control (7 items, e.g., when i get upset, i can get over it quickly), emotional self-awareness (5 items, e.g., i am a happy person), and situational responsiveness (4 items, when adults are friendly to me, i am friendly to them). for the purposes of the present study, only the emotional control subscale was used (for our sample, α = 0.71). confirmatory factor analysis supported the one-factor structure of the items (cfi = 0.988, tli = 0.982, rmsea = 0.041, 90% ci for rmsea = 0.000–0.085, srmr = 0.043). involvement in peer bullying was measured with an abbreviated version of the adolescent peer relationship index – bully target: bullying behaviour/ victimisation (apri-bt; parada, 2000; slovenian adaptation košir et al., 2018). we used three items for each form of peer bullying (verbal, physical, and relational) and constructed separate versions of the scale for observers, victims, and bullies/perpetrators. students indicated on a 6-point frequency scale (1 – never, 2 – once or twice, 3 – 2 to 3 times a month, 3 – once a week, 5 – several times a week, 6 – every day) how often they performed/experienced or observed a particular behaviour during the current school year (e.g., in the past year at this school, i have seen someone leave a student out of activities or games on purpose). exploratory factor analysis using varimax rotation was performed separately for each role in peer bullying, and the results showed a onefactor solution for observers and victims. the common factor explained 53% of item variance for observers (α = 0.89) and 58% of item variance for victims (α = 0.89, e.g., in the past year at this school, i was ridiculed by students saying things to me). for the role of bullies/ perpetrators, we found a two-factor structure, namely relational bullying (three items, e.g., in the past year at this school, i got other students to ignore a student, α = 0.75) and physical bullying with teasing (three items, e.g., in the past year at this school, i got into a physical fight with a student because i didn't like them, α = 0.72). we dropped three items from this scale because they had high loadings on both factors. data collection and statistical procedures prior to data collection, the study was approved by the ethics committee of the faculty of arts at the university of ljubljana. we also obtained parental consent for the students' participation in our study, as well as consent from students themselves to participate. data were collected via online survey in april 2022 (1ka, 2022). school counsellors gathered students in computer-equipped classrooms and guided them in completing the survey. we used spss 25.0 ibm for descriptive statistics and exploratory factor analyses and the r lavaan package (rosseel, 2012) for confirmatory factor analyses and path analyses. it should be noted that we were not able to test the whole structural equation model with all measurement models and a model of relations between latent constructs due to the small sample size, so we decided to first analyse all instruments, calculate scores and then include these scores in the path model. results table 1 shows the descriptive statistics of the variables and table 2 the correlations between the variables included in the model. table 1 descriptive statistics for the constructs included in the model n m sd skew. kurt. α authoritative style 202 3.70 0.79 –0.41 –0.70 0.92 authoritarian style 202 1.83 0.59 1.04 1.29 0.82 emotional control 202 3.59 0.57 –0.07 –0.36 0.71 relational bullying 202 1.22 0.46 3.02 10.58 0.75 physical bullying/ teasing 202 1.46 0.67 2.10 4.52 0.72 victimization 202 1.68 0.85 1.90 3.51 0.89 observing bullying 202 2.25 0.98 0.84 –0.05 0.89 the results show that on average students perceive their parents' parenting style as occasionally to frequently authoritative, and as authoritarian less often (table 1). they perceive their emotional control as moderate. regarding the different roles of involvement in peer bullying, involvement in either form of bullying is least common. somewhat more frequent is victimization and most common is involvement in peer bullying as an observer. it should be noted that skewness and kurtosis are high for all forms of active involvement in peer bullying (relational and physical 337 perceived parenting styles and emotional control as predictors of peer bullying involvement / pirc, pečjak, podlesek & štirn bullying with teasing and victimization), indicating that on average bullying has been observed, performed or experienced less than two or three times a month. the internal consistency coefficients for the variables included in the model are good or very good. table 2 pearson correlations between the variables included in the model 1. 2. 3. 4. 5. 6. 7. 1. authoritative style –.35** .29** –.20** –.18** –.01 .05 2. authoritarian style –.41** .40** .29** .23** .31** 3. emotional control –.38** –.36** –.30** –.28** 4. relational bullying .39** .19** .38** 5. physical bullying/teasing .25** .29** 6. victimization .40** 7. observing bullying note. ** p < 0.01 according to the results in table 2, there is a low, but significant negative correlation between authoritative and authoritarian styles as perceived by the students in our sample. there are also very low but significant negative correlations between student-perceived authoritative parental style and frequency of relational bullying and physical bullying with teasing. student-perceived authoritative parental style and their emotional control have a low positive correlation. student-perceived authoritarian style correlates negatively with student emotional control and positively with all roles of involvement in peer bullying. the correlations are low, but significant. student emotional control is significantly negatively associated with frequency of involvement in all roles of bullying, but the correlations are small. table 2 also shows that the frequencies of involvement in all roles of peer bullying have low, but significant correlations. due to the low reliability of some scales that we used for measuring the examined constructs, we had to exclude some constructs from our model. the model we tested and report here includes only authoritative and authoritarian parenting styles as predictors of emotional control. this acts as a mediator variable and predicts involvement in peer bullying, defined as relational bullying, physical bullying with teasing, victimization, and observation of bullying (figure 1). the model in figure 2 showed an excellent fit with the data, x2(20) = 195.36, p < .001, cfi = 1.00, rmsea = .000, 90% ci for rmsea = .000–.000, srmr = .000. figure 2 the resulting structural model of the relationships among parenting styles, emotional control and peer bullying involvement note. * p < .05, ** p < .01, *** p < .001 figure 2 shows the relationships among the variables included in the model. the results show that studentperceived authoritative parental style contributes to higher self-perceived emotional control and to a higher likelihood of students being involved in peer bullying as observers. students' perceived authoritarian parental style negatively predicts their sense of emotional control and positively their involvement in bullying as a relational or physical perpetrator, and as a bullying observer. all forms of involvement in peer bullying were negatively predicted by students' emotional control. discussion in our study, we examined the relationships among students' perceived parenting style, their emotional control, and their involvement in peer bullying. the results in figure 2 show that students who perceive their parents as more authoritative report having higher emotional control. although the path coefficient is small (.12), it is significant. it thus seems that children of parents who are more responsive to their feelings and needs, who talk to them and take their opinions into account, who involve them in planning, who set rules and explain to them the consequences of breaking the rules, and who try to comfort and understand them when they are upset, have higher emotional control. it means that these children feel that they are less likely to get upset when something does not go the way they want, and that they are able to calm down more quickly, have fewer outbursts of anger, are less likely to be disruptive at inappropriate times, and do not have such a hard time waiting to get what they want. the findings are consistent with those of fosco and grych (2012), whose results showed that children with warm and empathetic parents were better at regulating their emotions. moreover, in a study of adolescents by jabeen et al. (2013) an authoritative parental style was shown to have a significant positive effect on the children's emotional regulation. on the other hand, students' perceived 338 march 2023, volume 15, issue 4, 333-342 authoritarian parenting style was negatively (-.35) associated with their emotional regulation. students whose parents used physical punishment, withheld privileges, or threatened or punished them without explanation, yelled at them, and criticized them when teaching order and discipline had weaker emotional control. the same was found in some other studies: negative parenting was associated with less adaptive emotional regulation (calkins et al., 1998) and strict and conflictual parenting was associated with difficulties in behavioural regulation (brody and ge, 2001). shaw and starr (2019, deriving from grolnick et al., 1999 and sroufe et al., 1996) suggest that punitive or hostile parenting practices (features of authoritative style) could lead to children’s suppressing negative emotions instead of discussing them in a supportive environment, and so reduce the possibilities to learn and practice effective emotional control strategies. in terms of involvement in peer bullying, we found a positive relationship between the authoritative style and observers of bullying (.26). students who perceived their parents' style as more authoritative were more likely to notice all forms of peer bullying (teasing, deliberate bumping into each other, physical altercations, spreading of rumours, encouragement to ignore, exclusion, and so on). this could perhaps be explained by the fact that parents with an authoritarian parenting style talk more with their children about rules and rule-breaking, and encourage them to talk about their problems and feelings and to express their opinions. we can assume that students with more authoritative parents are more sensitive to rule violations and perceive certain behaviours as peer bullying earlier than most others. we found no associations between the authoritarian parenting style and other forms of peer bullying involvement. these results are consistent with the findings of broll and reynolds' (2021) study, which showed no associations between parenting styles and peer bullying involvement. however, the same and even stronger association (.48) was found between authoritarian style and observers of bullying. it is possible that different parenting styles are associated with different roles students engage in as observers. for example, pečjak and pirc (2014) found differences between passive and active observers, ignorers, and defenders of the victim. children with more authoritarian parents could be involved in peer bullying as active or passive promoters of peer bullying. apart from this relationship, the model in figure 2 also shows that the authoritarian style predicts relational bullying (.23) and physical bullying with teasing/ridiculing (.17). this means that students with more authoritarian parents are more likely to spread rumours, encourage others to ignore and exclude from activities (relational bullying), ridicule others, crash into or hit them (physical bullying with teasing). a positive relationship between the authoritarian style and perpetration of peer bullying has already been established in previous studies (alizadeh maralani et al., 2019; charalompous et al., 2018; espelage et al., 2000). some previously established relationships were not significant in our model. for example, we found no association between the authoritative parental style and active bullying involvement (perpetration or victimization), which is not consistent with the findings of charalampous et al.'s (2018) study, which found that the authoritative parental style negatively predicted peer bullying involvement. in light of previous research (lereya et al., 2013; martínez et al., 2019), one might also expect the authoritarian style to be associated with victimization, but we did not find this association in our student sample either. with the inconsistency of associations between parental styles and roles in peer bullying, it is necessary to note that the family is only one of many microsystems (bronfenbrenner, 1979) that determines what role the student takes in peer bullying. this is also influenced by the student's personality traits, and most importantly, by the classroom culture, which is co-created by the teachers and students in each classroom with class rules and behaviours when bullying occurs. emotional control negatively predicted all forms of involvement in peer bullying. that is, students who had less control over their emotions were more likely to be involved in perpetrating or experiencing peer bullying and observing bullying. the results of several studies showed that poor emotional control was a risk factor for victimization (blake et al., 2012; lomas et al., 2012; mahady wilton et al., 2000). one possible explanation could be that children who are quick to get upset when things do not go their way, have outbursts of anger and are disruptive at inappropriate moments become targets of peer bullying because their behaviour does not conform to social norms and rules. it is also possible that they are unable to stand up for themselves and have difficulties in calming down and controlling fear, shame or feelings of hopelessness. as blake et al. (2012) point out, there is a chance that such individuals are perceived as provocative victims by their peers. lomas et al. (2012) explain these associations with one of the features of peer bullying dynamics, namely a power imbalance. they hypothesize that children with lower emotional management and control skills have less power in a bullying situation, because they are unable to respond appropriately to the perpetrators. however, we also found a negative correlation between emotional control and bullying perpetration (relational and physical). apparently, the same lack of emotional control skills that could be a risk factor for victimization could also lead to bullying perpetration. bettencourt et al. (2013) argue that it is possible that students who are non-victimized perpetrators have other social skills that protect them from victimization. 339 perceived parenting styles and emotional control as predictors of peer bullying involvement / pirc, pečjak, podlesek & štirn finally, more bullying has been observed among students with poor emotional control. it is possible that students who themselves have difficulty responding in accordance with social group norms (e.g., their classroom peer group) are more aware of other situations in which peer bullying occurs. in summary, our findings suggest that emotional control is an important factor in exploring students' involvement in peer bullying. it appears that better social and self-regulation skills are significant protective factors against any role in peer bullying. our study also showed that parental style can help or hinder emotional control. implications two extensive meta-analytic studies (huang et al., 2019; ttofi & farrington, 2011) have shown that it is important to include the parental component in peer bullying prevention programmes. thus, it is important to incorporate knowledge about the relationships among parental styles and emotional control and peer bullying involvement into such programmes, as previously suggested by lereya et al. (2013). one way is to directly attempt to change the less appropriate practices that parents use in their interactions with children. it is possible that demonstrating effective strategies for reducing detrimental consequences for students involved in peer bullying to the parents would motivate them to reflect on their practices and change their behaviours. another way is to encourage parents to put more effort into implementing activities that would promote better emotional control in their children. it is also important to educate parents that one of the most important mechanisms for learning certain behaviours is observation and modelling (bandura, 1977). parents should be aware that how they communicate with their children, how they solve problems with them, how they themselves demonstrate emotional control and so on, models for their children how to apply these skills in their interactions with peers. the important influence of weak emotional control on involvement in peer bullying is also important information for teachers and other school personnel. our findings suggest that programmes to develop emotional regulation should be implemented in schools, which is emphasized also in “the european framework for personal, social and learning to learn key competence” (sala et al., 2020). since it is realistically difficult to expect that parents will significantly change their parenting styles, the compensatory role of the school in the development of emotional regulation (especially emotional control) is all the more important. although schools have little or no ability to promote changes in parenting styles, teachers and other school staff could make efforts to implement activities to promote emotional regulation in students. in addition, it may be possible for schools to identify students with weak emotional regulation skills early on, which would help reach the most vulnerable students and protect them from the potential consequences of involvement in peer bullying (lomas et al., 2012). limitations, strengths and future directions although we believe that the results of our study make an important contribution to the research of the relationships among family characteristics, emotional control, and involvement in peer bullying, there are some limitations that should be mentioned. first, our results are based solely on self-reported data, which means that responses could be subject to bias, such as social desirability. in addition, we measured students' perceptions rather than parents' actual behaviours. second, because of the small sample size, we were unable to define roles in peer bullying as categories; instead, we defined bullying involvement as an interval variable for more or less involvement. therefore, we were unable to include in our model a group of students who are in both roles – as bully/ victims – and who are assumed to be particularly vulnerable. for example, juvonen et al. (2003) reported that bully/victims have the most problems related to behaviour, school, and peer relationships. according to bettencourt et al. (2013), aggressive victims are most likely to perceive ambiguous situations as threatening, experience emotional dysregulation, and respond aggressively to peers. furthermore, we included the group of observers of peer bullying in our model, but without distinguishing between the roles they take in observing bullying, which would need to be considered in future studies. even though path analysis was used, the direction of the relationships reported here is preliminary. as such, longitudinal studies are needed to determine the directionality of the relationships among the constructs studied. in our study, we included only one specific family relational variable (parental style), so it would be beneficial to include other family and individual characteristics in the model, such as children's relationships with their siblings as a potential mediator or moderator variable in the relationship between parental style and emotional control. another limitation relates to our instrument for measuring parental style, namely the low internal consistency of permissive style, which prevented us from including one of the three traditionally researched parental styles in the model. in future research, special attention should be paid to the use of valid and reliable instruments when studying the characteristics of parental styles. despite these limitations, we believe that one of the strengths of our study is that we focused on adolescents, who are rarely considered when looking at the relationships between parenting 340 march 2023, volume 15, issue 4, 333-342 styles and emotional control because most studies have been conducted on young children (shaw and starr, 2019). another strength is that we included the perspective of bullying observers, which to the best of our knowledge has not been considered in previous studies of the relationships between parenting styles and bullying involvement. acknowledgement: this work was supported by the slovenia ministry of health in the context of the program neon safe without violence. references 1ka (version 21.02.16) [software]. (2022). ljubljana: faculty of social sciences. https://www.1ka.si. alizadeh maralani, f., mirnasab, m., & hashemi, t. (2019). the predictive role of maternal parenting and stress on pupils’ bullying involvement. journal of interpersonal violence, 34(17), 3691–3710. https:// doi.org/10.1177/0886260516672053 bandura, a. (1977). social learning theory. englewood cliffs, nj. baumrind, d. (1971). current patterns of parental authority. developmental psychology, 4(1p2), 1–103. baumrind, d. (2005). patterns of parental authority and adolescent autonomy. new directions for child and adolescent development, 108, 61–69. https://doi.org/10.1002/cd.128 bettencourt, a., farrell, a., liu, w., & sullivan, t. (2013). stability and change in patterns of peer victimization and aggression during adolescence. journal of clinical child & adolescent psychology, 42(4), 429–441. https:// doi.org/10.1080/15374416.2012.738455 blake, j. j., lund, e. m., zhou, q., kwok, o. m., & benz, m. r. (2012). national prevalence rates of bully victimization among students with disabilities in the united states. school psychology quarterly, 27(4), 210–222. https://doi.org/10.1037/ spq0000008 brody, g. h., & ge, x. (2001). linking parenting processes and self-regulation to psychological functioning and alcohol use during early adolescence. journal of family psychology, 15(1), 82–94. https://doi.org/10.1037/0893-3200.15.1.82 broll, r., & reynolds, d. (2021). parental responsibility, blameworthiness, and bullying: parenting style and adolescents’ experiences with traditional bullying and cyberbullying. criminal justice policy review, 32(5), 447-468. https://doi. org/10.1177/0887403420921443 bronfenbrenner, u. (1979). the ecology of human development. harvard university press. charalampous, k., demetriou, c., tricha, l., ioannou, m., georgiou, s., nikiforou, m., & stavrinides, p. (2018). the effect of parental style on bullying and cyber bullying behaviors and the mediating role of peer attachment relationships: a longitudinal study. journal of adolescence, 64, 109–123. https://doi. org/10.1016/j.adolescence.2018.02.003 calkins, s. d., smith, c. l., & gill, k. l. (1998). maternal interactive style across contexts: relations to emotional, behavioral, and physiological regulation during toddlerhood. social development, 7(3), 350–369. https://doi. org/10.1111/1467-9507.00072 chan, s. m. (2011). social competence of elementary‐ school children: relationships to maternal authoritativeness, supportive maternal responses and children's coping strategies. child: care, health and development, 37(4), 524– 532. https://doi.org/10.1111/j.1365-2214.2010.01196.x cole, p. m., hall, s. e., & hajal, n. j. (2017). emotion dysregulation as a vulnerability to psychopathology. in t. p. beauchaine & s. p. hinshaw (eds.), child and adolescent psychopathology (3rd ed., pp. 346–386). hoboken. dickson, d. j., laursen, b., valdes, o., & stattin, h. (2019). derisive parenting fosters dysregulated anger in adolescent children and subsequent difficulties with peers. journal of youth and adolescence, 48, 1567–1579. https://doi.org/10.1007/s10964-01901040-z eisenberg, n., liew, j., & pidada, s. u. (2004). the longitudinal relations of regulation and emotionality to quality of indonesian children’s socioemotional functioning. developmental psychology, 40(5), 790–804. https://doi-org. nukweb.nuk.uni-lj.si/10.1037/0012-1649.40.5.790 espelage, d. l., bosworth, k., & simon, t. r. (2000). examining the social context of bullying behaviors in early adolescence. journal of counseling & development, 78(3), 326–333. ht t p s : //d o i . o rg / 1 0.1 0 0 2 / j .1 5 5 6 6 676 . 2 0 0 0. tb01914.x 341 perceived parenting styles and emotional control as predictors of peer bullying involvement / pirc, pečjak, podlesek & štirn fosco, g. m., & grych, j. h. (2012). capturing the family context of emotion regulation: a family systems model comparison approach. journal of family issues, 34, 557–578. https ://doi.org/10.1177/01925 13x12 44588 9. georgiou, s. n. (2008). bullying and victimization at school: the role of mothers. british journal of educational psychology, 78(1), 109–125. https:// doi.org/10.1007/s11218-007-9048-5 georgiou, s. n., & stavrinides, p. (2013). parenting at home and bullying at school. social psychology of education, 16(2), 165–179. https://doi. org/10.1007/s11218-012-9209-z gini, g., & pozzoli, t. (2009). association between bullying and psychosomatic problems: a metaanalysis. pediatrics, 123(3), 1059–1065. https://doi. org/10.1542/peds.2008-1215 gómez-ortiz, o., del rey, r., casas, j. a., & ortegaruiz, r. (2014). parenting styles and bullying involvement/estilos parentales e implicación en bullying. cultura y educación, 26(1), 132–158. https://doi.org/10.1080/11356405.2014.908665 hacin, k. (2019). parenting styles and dimensions questionnaire [unpublished translation]. department of psychology: university of ljubljana. herd, t., king-casas, b., & kim-spoon, j. (2020). developmental changes in emotion regulation during adolescence: associations with socioeconomic risk and family emotional context. journal of youth and adolescence, 49(7), 1545–1557. https://doi.org/10.1007/s10964020-01193-2 hu, l.-t., & bentler, p. m. (1999). cutoff criteria for fit indexes in covariance structure analysis: conventional criteria versus new alternatives. structural equation modeling, 6,(1), 1–55. https:// doi.org/10.1080/10705519909540118 huang, y., espelage, d. l., polanin, j. r., & hong, j. s. (2019). a meta-analytic review of schoolbased anti-bullying programs with a parent component. international journal of bullying prevention, 1(1), 32–44. https://doi.org/10.1007/ s42380-018-0002-1 košir, k., pivec, t., klasinc, l., špes, t., & horvat, m. (2018). psihosocialne značilnosti učencev kot napovedniki različnih oblik medvrstniške viktimizacije in nasilnega vedenja. [students’ psychosocial characteristics as predictors of different forms of victimization and bullying]. psihološka obzorja [horizons of psychology], 27, 171–186. jabeen, f., anis-ul-haque, m., & riaz, m. n. (2013). parenting styles as predictors of emotion regulation among adolescents. pakistan journal of psychological research, 28(1), 85–105. juvonen, j., graham, s., & schuster, m. a. (2003). bullying among young adolescents: the strong, the weak, and the troubled. pediatrics, 112(6), 1231–1237. https://doi.org/10.1542/peds.112.6.1231 lereya, s. t., samara, m., & wolke, d. (2013). parenting behavior and the risk of becoming a victim and a bully/victim: a metaanalysis study. child abuse and neglect, 37, 1091–1108. https://doi. org/10.1016/j.chiab u.2013.03.001. lomas, j., stough, c., hansen, k., & downey, l. a. (2012). brief report: emotional intelligence, victimisation and bullying in adolescents. journal of adolescence, 35(1), 207–211. https:// doi.org/10.1016/j.adolescence.2011.03.002 olweus, d. (1993). victimization by peers: antecendents and long-term outcomes. in k. h. rubin & j. b. asendorpf (eds.). social withdrawal, inhibition, and shyness in childhood (pp. 315–341). hillsdale, nj. macdermott, s. t., gullone, e., allen, j. s., king, n. j., in tonge, b. (2010). the emotion regulation index for children and adolescents (erica): a psychometric investigation. journal of psychopathology and behavioral assessment, 32(3), 301–314. https://doi.org/10.1007/s10862009-9154-0 mahady wilton, m. m., craig, w. m., & pepler, d. j. (2000). emotional regulation and display in classroom victims of bullying: characteristic expressions of affect, coping styles and relevant contextual factors. social development, 9(2), 226–245. https://doi.org/10.1111/1467-9507.00121 mahapatra, s., & batul, r. (2016). psychosocial consequences of parenting. journal of humanities and social science, 21(2), 10–7. malm, e. k., & henrich, c. c. (2019). longitudinal relationships between parent factors, children’s bullying, and victimization behaviors. child psychiatry & human development, 50(5), 789– 802. https://doi.org/10.1007/s10578-019-00882-9 martínez, i., murgui, s., garcía, o. f., & garcía, f. (2019). parenting in the digital era: protective and risk parenting styles for traditional bullying and cyberbullying victimization. computers in human behavior, 90, 84–92. https://doi. org/10.1016/j.chb.2018.08.036 342 march 2023, volume 15, issue 4, 333-342 morris, a. s., silk, j. s., steinberg, l., myers, s. s., & robinson, l. r. (2007). the role of the family context in the development of emotion regulation. social development, 16, 361–388. https ://doi.org/10.1111/ j.1467-9507.2007.00389.x. nocentini, a., fiorentini, g., di paola, l., & menesini, e. (2019). parents, family characteristics and bullying behavior: a systematic review. aggression and violent behavior, 45, 41–50. https://doi.org/10.1016/j.avb.2018.07.010 parada, r. h. (2000). adolescent peer relations instrument: a theoretical and empirical basis for the measurement of participant roles in bullying and victimization of adolescence: an interim test manual and a research monograph: a test manual. pečjak, s., & pirc, t. (2014). sem opazovalec medvrstniškega nasilja: kaj lahko naredim? [i am a bystander in peer bullying: what can i do?]. pedagoška obzorja: časopis za didaktiko in metodiko [didactica slovenica], 29(1), 155–168. robinson, c. c., mandleco, b., olsen, s. f., & hart, c. h. (1995). authoritative, authoritarian, and permissive parenting practices: development of a new measure. psychological reports, 77(3), 819–830. https://doi.org/10.2466/pr0.1995.77.3.819 romih, a., & košir, k. (2018). učinki treninga regulacije čustev pri učencih osnovne šole. [the effects of emotion regulation training on primary school students]. pedagoška obzorja: časopis za didaktiko in metodiko [didactica slovenica], 33(3/4), 36–54. rosseel, y. (2012). lavaan: an r package for structural equation modeling. journal of statistical software, 48(2), 1–36. https://www.jstatsoft.org/ v48/i02/. rueda, p., pérez-romero, n., cerezo, m. v., & fernándezberrocal, p. (2022). the role of emotional intelligence in adolescent bullying: a systematic review. psicología educativa, 28(1), 53–59. https:// doi.org/10.5093/psed2021a29 sala, a., punie, y., garkov, v., & cabrera, m. (2020). lifecomp: the european framework for personal, social and learning to learn key competence (no. jrc120911). joint research centre (seville site). shaw, z. a., & starr, l. r. (2019). intergenerational transmission of emotion dysregulation: the role of authoritarian parenting style and family chronic stress. journal of child and family studies, 28(12), 3508–3518. https://doi. org/10.1007/s10826-019-01534-1 steinberg, l., darling, n. e., & fletcher, a. c. (1995). authoritative parenting and adolescent adjustment: an ecological journey. in p. moen, g. h. elder, jr., & k. lüscher (eds.), examining lives in context: perspectives on the ecology of human development (pp. 423–466). american psychological association. https://doi. org/10.1037/10176-012 thompson, r. a. (1994). emotion regulation: a theme in search of definition. monographs of the society for research in child development, 59(2/3), 25– 52. https://www.jstor.org/stable/1166137 ttofi, m. m., & farrington, d. p. (2011). effectiveness of school-based programs to reduce bullying: a systematic and meta-analytic review. journal of experimental criminology, 7(1), 27–56. https:// doi.org/10.1007/s11292-010-9109-1 volk, a. a., veenstra, r., & espelage, d. l. (2017). so you want to study bullying? recommendations to enhance the validity, transparency, and compatibility of bullying research. aggression and violent behavior, 36, 34–43. https://doi. org/10.1016/j.avb.2017.07.003 international electronic journal of elementary education, 2015, 8(2), 243-266 issn:1307-9298 copyright © iejee www.iejee.com the challenges of distributing leadership in irish post-primary schools margaret o’donovan  university college cork. ireland abstract this study explores the challenges and opportunities in relation to developing distributed leadership practice in irish post-primary schools. it considers school leadership within the context of contemporary distributed leadership theory. associated concepts such as distributed cognition and activity theory are used to frame the study. the study is situated in a space which acknowledges the current complex reality in our schools, where school leadership is characterised by increased workload and an ever-expanding role-definition. drawing on the empirical findings from semi-structured interviews conducted with principals, deputy principals, post of responsibility holders and teachers in three case-study schools, the study probes: (1) how school leaders (re)construct a form of leadership suited to the needs of the current reality, by exploring their leadership and management styles; (2) how the internal conditions are created in which distributed leadership can function; (3) the challenges posed by distributed leadership and how they might be overcome. the findings clarify that school leadership is a construct beyond the scope of the principal alone. while there is widespread support for a distributed model of leadership, the concept does not explicitly form part of the discourse in the case-study schools. this poses challenges for school leaders and policy-makers to put mechanisms in place to re-culture schools, to develop teacher-leadership capacity and to reflect on the future direction of leadership in irish post-primary schools. keywords: leadership, distributed leadership, distributed cognition, activity theory, case study, policy, irish post-primary schools, discourse. introduction in the educational climate of recent decades, there are increased demands for greater accountability, along with standards and outcomes-based reform measures. into the mix must be added the new challenges for school leaders, resulting from a changing social environment across many countries, due to rapid technological innovations, mobility, globalization and attendant changes in legislation (moller, 2009). the irish post-primary system is no exception to this trend. in this frame, school leadership is high on policy  margaret o’donovan, regina mundi college, cork and school of education, university college cork. ireland. phone: + 353 21 4291681, email: modonovan@reginamundicork.ie http://www.iejee.com/ international electronic journal of elementary education vol.8, issue2, 243-266, 2015 244 agendas and there is a focus on the relationship between leadership, school improvement and sustaining change (harris, 2005). there is worrying evidence, however, that school leaders (principals) feel more pressurized than did their peers a number of generations ago, with the phenomenon of increased workload, coupled with the manner in which the role has expanded and intensified, as well as its inherent stresses, leaving principals feeling disempowered (evans, 1996, bottery, 2004). this point is clearly evidenced in the irish post-primary system where the current administrative workload, accountability measures, legislative requirements, budgetary cutbacks, with the erosion of the middle management structure, limit the capacity and detract from the primary role of the principal as the strong visionary leader of learning in his/her school. while this current situation in school leadership presents challenges, it also provides opportunities to reflect on new modes of organizational and work re-design and lateral capacity building. because the process of change reflects not only adaptation to external forces, but also an investment of energy in what people do in their mutual relations (wenger, 1998), it allows us to consider innovative ways of looking at the practice of leadership in our schools and on how lateral capacity might be developed, as schools seek to re-configure and re-structure school leadership roles. in engaging with this current educational leadership conversation, the pertinent questions requiring reflection and closer scrutiny in the irish context include: (i) what forms of school leadership are required to mediate the influences of the broader discourses of external forces and policies?; (ii) in the current constantly evolving and fluid educational landscape, how are the leader and follower roles and identities constructed?; (iii) if the core work of a school is teaching and learning and leadership is framed as the how of education, what are the issues pertaining to and capacity building of teachers?. in responding to these questions, the construct of distributed leadership is utilized as the lens through which to analyze leadership practice in irish post-primary schools. in essence, leadership from the distributed perspective is premised on capacity-building in the school organization, with the engagement of many people being at the core of distributed leadership in action (harris and muijs, 2005). schools in the irish education system, along with the teachers in them have not had a tradition of engaging in collaborative planning and evaluation. historically, the model for school leadership was based on a hierarchical system of governance, focused on authority, power and knowledge being vested in the principal at the apex of the organization, especially in religious run voluntary secondary schools, which had a very weak middle management system. the culture was counter-collaborative with the ‘lay teacher’ (non-religious) being denied agency. in the current complex era, this poses challenges for school leaders to enact posthierarchical options and to develop their school organizations as participatory communities, whereby leadership is collaborative in nature. in this article, the stance is taken that the challenge of interweaving and mediating the historical narratives with new policy paradigms, shapes the construction of leadership in irish post-primary schools. it must be borne in mind, however, that change always involves both continuity and discontinuity. the challenge for school leadership is to mediate the tensions between continuities and discontinuities. the argument will be made that these two contrasting scripts can be integrated and “these new relationships can awaken new interests and translate into a re-negotiation of the enterprise” (wenger, 1998, p.97), as schools restructure and re-define themselves. the challenges of distributing leadership in irish post-primary schools / o’donovan 245 the purpose of this paper is to explore the nature and process of leadership practice in irish post-primary schools and to probe the challenges in distributing leadership. i first briefly review the literature on distributed leadership and extract a set of tools which inform the study. i then discuss findings from my qualitative case-study research on three post-primary schools. the empirical work that currently exists in ireland, concerning distributed leadership and on how leadership mediates adaptive challenges, is limited. this study, though small-scale offers a powerful empirical lens to illuminate leadership practice in irish post-primary schools in ireland. theoretical background while current discourses on school improvement downplay the traditional notion of the single, strong leader, schools are being drawn in a direction where there is a shifting emphasis in education from individual to collective responsibility (fullan, 2001, de lima, 2008). distributed leadership is one prominent conceptualization of leadership prevalent in the current discourse about leadership practice in schools. there is general agreement among all researchers and analysts that achieving results with others is the essence of leadership and the role of transformational leadership is to help others to find and achieve new goals, individually and collectively (hallinger and heck, 2003). most authors also seem to agree with the conceptualization that distributed leadership is not the same as dividing responsibilities and tasks among individuals who perform certain defined organizational roles, but instead, it comprises dynamic interactions between multiple leaders and followers (timperley, 2009). a distributed perspective recognizes that there are multiple leaders, with the focus being upon the interactions, rather than the actions of those in formal and informal roles, as it is primarily concerned with how leadership influences organizational and instructional improvements (spillane et al, 2004, spillane, 2006, harris and spillane, 2008). the research for this article has been informed by the seminal work of two contemporary theorists, james spillane (2004, 2005, 2006) and peter gronn (2000, 2002, 2008, 2009), whose comprehensive body of work adopts a socio-cultural perspective. i have chosen these two seminal theorists for two primary reasons: (i) in theoretical terms the most contemporary and robust analysis of distributed leadership is located within their work (harris and muijs, 2005), (ii) because distributed leadership does not have a broad empirical base, they both utilize the concept as a diagnostic and analytical tool for thinking about leadership and not as a prescription for how it should be practiced. spillane (2005, 2006) draws heavily from distributed cognition, which proposes that human knowledge and cognition are not confined to an individual, but are distributed in the interactive web of actors, artefacts and situation, relying on certain cultural tools and ways of being that are valued in particular contexts. spillane et al (2004), drawing on rich empirical data from a four-year longitudinal mixed-methods study in chicago, which remains the largest contemporary study of distributed leadership practice, suggest that a distributed perspective involves two aspects: the leader-plus and the practice aspects. the leader-plus aspect implies a social distribution of leadership, involving multiple leaders. spillane et al (2004) contend that understanding how various leaders in a school work together to enact leadership tasks is a key aspect of the social distribution of leadership practices. they further suggest that this social distribution is much more than an additive model, thus acknowledging a division of labour in the enactment of tasks. rather, it involves understanding how practice is stretched over the work of various school leaders. in this conceptualization, leadership practice is primarily about interactions between the many rather than the few, it is not simply about roles and positions, but international electronic journal of elementary education vol.8, issue2, 243-266, 2015 246 rather a reciprocal interdependency is created between individuals’ actions (spillane, 2005). in this frame, thus, the practice is the central concern: leadership practice is generated in the interactions of leaders, followers and their situation; the situation defines leadership practice and is itself defined through leadership practice. the situational distribution of leadership is pertinent and merits further elaboration. in keeping with distributed cognition, spillane et al (2004) argue that situation is not external to leadership activity, but serves to influence leadership activity from within the activity, being stretched over the many facets of the school situation, including tools, language and organizational structure. in their definition, structure refers, not only to organizational structures, but also to broader social structures and the way they influence human agency and the interactions between leaders and followers, in enacting school leadership. spillane et al (2004) draw on the work of etienne wenger (1998) to illuminate how situation is constructed as the socio-cultural context or “reifications of practices” that exemplify school leadership activity. wenger’s theory suggests that people’s identities are constructed in relation to their participation in “communities of practice”, and ways of knowing can be derived from participation in these communities. from this perspective, “communities of practice” are constitutive of shared histories of learning and evolve through the processes of participation and reification, interacting and intertwining over time. this perspective is of value in enabling understanding of leadership activity and practice, as a school community is comprised of many forms of participation and school leadership is mediated by “communities of practice”, in which “meanings are negotiated in practice” (wenger, 1998, p.85). considering these conceptual issues, a key challenge in adopting a distributed leadership framework and, influencing the current research is to identify and analyze aspects of the situation that constitute leadership activity in schools. similarly, peter gronn (2000, 2002, 2008, 2009) argues for a distributed perspective on leadership based on a numerical view, whereby leadership is dispersed among some, many or possibly all members of the organization. this multiple sense, numerical view of leadership allows for all organizational members to be leaders at some stage. this additive understanding does not privilege the work of certain individuals, nor is there a presumption about which individuals’ behaviours carry more weight with colleagues (gronn, 2000). rather, this type of leadership is characterized by synergistic relationships and assumes that the aggregated sum of leaders’ work adds up to more than the individual parts. the theoretical basis of gronn’s conceptualization of distributed leadership is underpinned by cultural-historical activity theory. in activity theory, there is an emphasis on the division of labour, creating different positions for the participants. the activity system has multiple layers, with work being mediated by tools and performed in conditions of collective activity and “conjoint agency”. gronn’s (2000, 2002) perspective suggests that leadership has an emergent property. this dynamic, additive holistic concept posits that leadership is more appropriately understood as fluid, rather than a fixed phenomenon. this conceptualization has enormous potential to move forward our understanding of leadership by foregrounding organizational work, more and more as the focus of analysis and discussion is an ongoing trajectory of organizational evolution (gronn, 2000). an additional advantage of this participative emergent approach is that it opens up a wide range of development options and possibilities, as individuals can learn from each other in a meaningful and mutually supportive way (harris and muijs, 2005). in this way, practice is ultimately produced by organizational members, through the negotiation of the challenges of distributing leadership in irish post-primary schools / o’donovan 247 meaning, as the continuity of such emergent structures derives, not from stability, but from adaptability (wenger, 1998). together with this technical side of the division of labour, there is also a social side, which is based on the values, interests and ethos of the groups. this perspective is closely allied to the activity theoretical principle of historicity, which posits that activity systems take shape and may only be understood against their own history. engestrom (1998) proposes that history needs to be studied at local level and also at the level of theoretical ideas and tools that have shaped the activity. thus, work in a school needs to be analyzed against the history of its local organization and also against the more global history of the concepts, procedures and tools employed and accumulated in the local activity. this perspective is of value in studying and analyzing the how and what of leadership practice in our schools and in mediating contradictions and innovating change. the socio-cultural context of the school is fundamental to any discussion on school leadership and the distribution of leadership in schools. in light of the theoretical research base and to move forward our understanding of the phenomenon of leadership in irish post-primary schools, the distributed leadership conceptualization is considered a promising approach to analysis. the application of the distributed leadership framework will enable an elucidation on the how and what questions of school leadership and will serve to open up both the “blank spots” (heck and hallinger, 1999) in our understanding of the challenges of distributing leadership in irish post-primary schools. methodology this study has adopted a phenomenological approach and seeks an understanding of the life-world (lebenswelt) of the participants, and how they make sense of that life-world (kvale and brinkmann, 2009). the key phenomenologically-inspired elements of this research are: (i) to generate, through interviews, a clarity of the life-world situations of the participants in the case-study schools and to gain a deeper understanding of how leadership is experienced from the inside, and (ii) to analyze, interpret and understand the meanings pertaining to key themes in the subjects’ life-worlds, as they relate to leadership, as well as the factors and processes that give rise to and shape them. in this way, it is intended to arrive at the essence of the meaning of the leadership experience. purposeful sampling (merriam, 2009) was used to select 3 case-study schools, to enable a deeper understanding of the how and why of leadership practice. in selecting cases, account was taken of the different socio-cultural backgrounds and contexts: one from the voluntary secondary sector, one from the community and comprehensive sector and one from the vocational educational sector; i interviewed teachers at all levels in the school. the three case-study schools have a broad geographical spread, with variations based on school size, history and socio-cultural context; one is an urban all girls’ voluntary secondary school, one is an urban all boys’ voluntary secondary school and the third is a rural co-educational community college. the research in the schools took place during autumn 2010 and spring 2011. semistructured interviews, which form the kernel of the empirical component, were in-depth and in each school, the principal, deputy principal, a number of assistant principals, special duties teachers, non-post of responsibility holders and newly appointed teachers were interviewed. the data base also includes seven secondary interviews, one with the chairman of the board of management of one of the case-study schools, and 6 focus group interviews, one with each of the parents’ councils and one with each of the student councils of each of the case-study schools. international electronic journal of elementary education vol.8, issue2, 243-266, 2015 248 all interviews were audio-recorded and subsequently transcribed manually. all data analysis was done manually, employing a combination of inductive and thematic analysis (boyatzis, 1998), as a means of encoding the information and extracting themes and patterns from the data. findings by focusing on the lebenswelt and everyday lives of the principals and teachers in the casestudy schools, by using three of the emergent sub-headings from the data, the purpose of this article is to probe and analyse the implications and challenges of these characteristics for the distribution of leadership in the schools. the themes which will be discussed in this article are (i) constructions of leadership, (ii) managerial leadership and (iii) instructional leadership. constructions of leadership the initial question probed respondents’ understanding of leadership and where they locate themselves conceptually. at the outset there was a general consensus that the role is becoming more complex and challenging, as they endeavour to navigate and mediate the increasingly fluid and blurred roles between the concepts of leadership, management and administration. viewed from the perspective of school leaders themselves, the role of principal is positioned between and seeks to balance externally policy initiatives, on the one hand, with creating and maintaining school goals, identity and ethos on the other hand. moral purpose. notwithstanding the evolving educational landscape, with its external imperatives resulting in principals spending more time on administration and management tasks, the respondents extract the visionary and moral leadership component as being essential to their leadership practice: the single phrase i use most often is moral purpose (p1) for me the leadership role is the bit that inspires people, moves them forward (p2) emotional construct. i then explored how principals and deputy principals establish their identities as emotional beings and argue that these protagonists need to have an emotional connectedness with their school communities (teachers, students, parents, boards of management), enacted through relationship building, in order to achieve the school’s goals. participation in a “community of practice”, refers not alone to the process of taking part, but to the relations and social experience of living in the world, involving one’s whole person (wenger, 1998). thus, because leading and managing are not static activities, the emotional construct of educational leadership needs to be fore-grounded, in any analysis on the implementation of distributed leadership in schools. the frame outlined above prompted a discussion with principals and deputy principals on relational methods of mutual engagement. one respondent indicated that: i’ve moved my time to the affirmative part of the school, so i would spend the vast majority of time visiting classes, to praise teachers or say how superb things are going (p3). the challenges of distributing leadership in irish post-primary schools / o’donovan 249 the incumbent points out that, in order to realize the role model aspect of his position, he has reduced his role in organizational work and in the disciplinary process. this role has now been distributed to year heads, as part of the middle management structure of the school. the principal explains that he now spends a lot of his time on the corridor, meeting staff, conversing with them, giving them reassurance and direction. the interviewee acknowledges that this strategic enactment of positive emotional leadership is possible, because the school is relatively new and posts of responsibility have not been lost due to staff retirements. another principal offers a divergent view on how he enacts relationship building: if he doesn’t contribute to the system, he will drain the system. leadership is really about preventing energy drainage out of the system and the release of as much into the system as one can. i guess it ultimately boils down to conveying a sense of support, encouragement and validation (p2). the principal in the above vignette refers to the issue of the “redirection of some energy” and sees leadership in that context, as dissuading the staff member, without injuring him, and thereby allowing him a perceived agency by coming up with an alternative or remodeling of what the staff member had in mind. the channeling of this energy flow within the school organization is characterized by the emotional intersubjectivities, in the professional relationships of the members of that school organization. the emotional intensity, on the part of the principal and deputy principal, of constructing the school’s “emotional map” (moller, 2005), is not only focused on fostering good relations internally by validation and affirmation, but is also part of a process of knowing, and indeed regulating, the emotional subjectivities of all the agents in the school. trust. as an extension of this discussion, a traditional ethos of care and nurture, underpinned by values of respect, inclusion, openness and approachability has framed educational leadership in ireland. the empirical evidence from the current case-study research indicates that leadership, as practiced in irish post-primary schools, is embedded in an ethos of care. within this ethos and a key factor in securing teacher leadership is trust and it serves to give teachers a sense of autonomy (harris and muijs, 2005). as one principal suggests: the human being always responds to validation, and trust is probably the ultimate validation, because it’s a signal that you’re capable (p1). this view is very much corroborated in the interviews with the other principals. there are divergences, however, in how trust is enacted. thus, the question needs to be posed regarding how authentic that trust is or if it is merely rhetoric as the assertion that any enactment of a team spirit based on trust is: a claim anybody can make, and people are going to make it, whether there is or there isn’t (p2) multiple meaning can be ascribed to enactments of trust, based on the subjectivities of the protagonists, from models that are carefully orchestrated by principals, to those that are enabled through discussion and debate, where people are allowed to make mistakes. building an identity as a school principal consists of negotiating the meaning of one’s experience of membership of school communities, and the negotiations are processes where emotions and questions of power play important roles (moller, 2005, p. 145). my philosophy is this, if i give a role or a job to somebody, we [principal and deputy principal] start at the point that people need a lot of help, but our object is within two years, to empower that person to a point where they can do the job, without recourse to us except to check that the way things are moving is within the general ethos of the school (p1). international electronic journal of elementary education vol.8, issue2, 243-266, 2015 250 risk-taking. there are other ways in which trust can be enacted. another principal puts forward the view that in a school culture which promotes empowerment and the distribution of leadership, and which supports people during times when that leadership works, and at times when it does not work, risk-taking is a key component. this principal sees her role as one of trusting, supporting and confidence-building and in encouraging teachers to take a risk. i would see most members of staff now being willing to risk something, whether it’s making a presentation in front of staff or taking on another role. this is one of the things i love, people are willing to take on leadership roles, without having an official title (p2). risk-taking is very much part of this principal’s trajectory of leadership practice, as she creates the conditions necessary to take risks in the interests of new learning and identity formation (sugrue, 2005). the sub-text of the above narratives may be read as instances of the influential spaces occupied by teachers in the decision-making and leadership processes of schools. key questions arise around who are the power-holders in the schools and who is empowered to act? in the cultures of management in these schools, power and influence are presented as being distributed, negotiable and are exercised both formally and informally. shared practice. another overarching conceptualization across the three case-study schools constructs leadership as: giving empowerment right down through the organization for all members to be leaders themselves (p1) i think it applies throughout the school at all levels (p2) the only way you can give leadership is by allowing people to practice leadership (p3). consistent with the literature (ritchie and woods, 2007), a recurrent theme emanating from the interviews with principals and deputy principals is a recognition that leadership in the twenty first century is a shared phenomenon, with the aim of empowering all members of the school community. in respect of this debate, it is important to note, however, that conversations with members of the parents’ councils and student councils of the three schools constructed leadership as residing primarily in school principals. this calls on school leaders to facilitate debate among all stakeholders to consider “the changing landscape of agency in work organizations” (engestrom, 2008, p. 207). teachers’ perceptions of principals’ leadership practices. the research also sought to analyse the lenses through which teachers position the roles of their principals, along with the factors and processes influencing decision-making procedures in the schools. the following three quotes are illustrative of typical themes emerging from the interviews with teachers. the vignettes include themes and modes of belonging which, (i) construct hierarchy as a constraint, (ii) propose that principals lead from the front and the back, thereby, implying that leadership and agency are becoming increasingly distributed, (iii) fore-ground issues of identity and agency, (iv) recognize that leadership practices are informed by school ethos and characteristic spirit. i recently came back to the classroom after fourteen years. it is now much more structured, the organization, the red-tape, the paper work, the legal implications and the whole sense of accountability. i think sometimes a principal has to lead from the front and sometimes they have to lead from the back. it must be very difficult trying to marry that with everybody. i suppose leadership is in the middle of the circle (t1). the challenges of distributing leadership in irish post-primary schools / o’donovan 251 i think a good leader should share skills. i think it’s actually good to share out the skills. you can’t be dependent on one person [principal], i think that makes for a happier atmosphere (t2). i see leadership throughout the school. i mean the principal and deputy principal are the top when it comes to authority, but there are a lot of other circles and networks going on (t3). in the above quotes, there is an explicit recognition of the increasingly complex and challenging role of the school principal and of extrinsic variables in the guise of prescriptive, externally-mandated demands for accountability and legal compliance. implicitly, what emerges from the narratives is that a principal’s identity is constructed by mediating external variables with the internal life-world and history of the school community. the comment that the principal has to marry different perspectives and experiences of all members of the school community supports the finding from the interviews with the principals, that in constructing the school’s “emotional map”, the principal focuses on fostering good relations and regulating the emotional inter-subjectivities of all agents in the school. from this perspective, negotiating school identity and meaning involves both interpretation and action on the part of the school principal, as school leader. in negotiating this meaning, there is a recognition that it is incumbent on the principal to be sensitive and respectful to the diverse needs of the school community and to bring together the multiple perspectives, interests and interpretations that participation entails (wenger, 1998). a key point emanating from the above data is that, in the increasingly complex and pressurized school environments of today, the notion of the heroic, solo leader at the apex of the organizational structure no longer prevails. while there is an implicit expectation that school principals create the environments to enable the diffusion and distribution of leadership practice throughout the schools, the tone of the discourses indicates that this is a reciprocal process, with teachers expressing satisfaction and support for their principals in that endeavour. transformational leadership. congruent with the findings from the interviews with the principals, from the perspectives of the teacher interviewees, principals’ leadership practice has a strong transformational component. teachers identified the importance of transformational models of leadership practice as being significant in mediating complex situations. the conceptualisations of transformational leadership are as follows: i see leadership as both guiding and nurturing. the guidelines for leadership, certainly within this school, i see them coming from the top (t1). i suppose it [leadership of the principal] is a combination of a number of things. i think, the principal, as a true leader has to act in such a way that people see them doing such things and follow in their footsteps (t2). i’m always very careful that the principal is not standing alone in the cold and everybody else down on the other end of the pitch. it’s important that the principal knows the staff are behind her. now that can only happen if the principal behaves in a certain manner too. it’s a two-way thing (t3). i think the whole tone of the place is set by the principal (t4.) the above quotations indicate that there are diverse views on what constitutes transformational leadership. the various perspectives range from constructing this element of leadership practice of the principal as setting the vision, enabling a mutually international electronic journal of elementary education vol.8, issue2, 243-266, 2015 252 supportive environment, using the tools of communication and collaboration and being visible. this empirical evidence constructs practice in a participative, shared model informed by the core values of the school. while it is acknowledged that the quotations are informed by the respondents’ subjectivities, a commonality in all discourses is the positioning of the principal as the visionary leader in the school, who leads by example. this symbolic force of the principal’s leadership, whereby s/he communicates what s/he stands for provides meaning to the school and gives both teachers and students a sense of order and direction, to which they respond with motivation and commitment (sergiovanni, 2001). the construction by teacher 2 above, of the principal as the “true leader” is an interesting rhetoric as it points towards a practice that privileges the perspective of those who define the organizational procedures and implicitly “hides the knowledgeability of those who apply them” (wenger, 1998, p. 261). while the argument may be made that the institutional authority is an essential aspect of negotiability, it must also be borne in mind that an organization which functions in an adequately coordinated fashion, without excessive recourse to privileging thrives on “intensive negotiation of meaning” (wenger, 1998, p. 261). the challenge for transformational leadership, thus, is to enable the school, as a learning community, to develop new understandings and identities of participation. managerial leadership this section of the analysis provides insights into how managerial leadership unfolds in the schools. it also provides an understanding on how the instruments and tools utilised to enable the enactment of leadership and management across the various agents in the school communities – both positional and non-positional (spillane et al, 2009). it explores the various positional (middle management teams in the guise of post of responsibility holders and subject department coordinators) and informal leaders (who exercise leadership to complete a specific task), whereby teachers exercise leadership and decision-making (i) to enable the school organisation to achieve its goals and (ii) to enable the lateral distribution of leadership and the development of collaborative school cultures. the pervasive practice of volunteerism is also identified as a central dimension of teacher, non-professional, leadership practice. middle management structure. it was clear from the conversations with the principals and deputy principals that their identities and their lived reality, at this time, are shaped by adaptation to the exigencies of externally mandated accountability regimes. the following quotation is typical of sentiments expressed by principals in all the case-study schools. a lot has come back to the principal and deputy principal. it means that by taking on the day-to-day things, an awful lot more, that you have less time to do the planning, and the planning is done outside of school time, because you’re caught up with so much of the day-to-day things (p1). all interviewees concurred that there is an increasing pressure on the principal and deputy principal to fulfil an increasing number of administrative duties, because schools are required to adhere to accountability regimes. the embargo on promotional posts, as an enactment of state intervention, has adversely impacted on the formal distribution of leadership through the middle management structure. by extension, it also impacts on the primary role of the principal as educational leader, to enable capacity building, in order to create a professional learning community in his/her school. if you don’t build people’s leadership skills at this stage, when we eventually get our posts back, we won’t have anybody ready to fill them (p2). the challenges of distributing leadership in irish post-primary schools / o’donovan 253 one way in which principals manage this complex situation is to distribute leadership by negotiation, persuasion and strategic planning. this leadership distribution is both opportunistically and strategically managed by principals, as they focus on school improvement (macbeath, 2009). what is clearly evident from the interviews is that as principals navigate these new realities and “modes of belonging”, issues of power come to the fore, in terms of how actions, transactions and “patterns of activity” shape distribution of leadership. the conversations with post of responsibility and non-post of responsibility holders alike, indicate that the post of responsibility structure constitutes the formal, remunerated middle management stratum, supporting the principal in leading and managing the school. there is an explicit recognition of the need to share duties across members of the teaching staff and “to support the principal” in the complex life-world of today’s school organizations. the year head system is a key component of the middle management structure, allowing the protagonists to have an effective role in managing student progress. year heads’ meetings. one of the tools and routines utilized in all the schools, to enable the distribution and enactment of leadership across these positional agents, is the convening of year heads’ meetings with the principals and deputy principals. following spillane et al’s (2004) task-centred approach, the following typical quotes centre on the tasks around which school leaders organize practice. i’m a year head, we’ve meetings twice a week and we meet in the principal’s office. we discuss pastoral and discipline issues relating to the students. as a follow-on from these meetings, if parents need to be consulted, i would get in touch with them. i meet the teachers and find out how things are going [re student discipline]. i visit classes the odd time (t4). it [year heads’ meeting with the principal] is a kind of think-tank, where you share ideas, and i think that’s really important. i would follow up on any issues for the next meeting (t6). the above data suggest that teacher leadership is being recognized across all school communities and that teacher leadership is being distributed and actively supported in schools. the timetabling of structured meetings between the principal, deputy principal and year heads is indicative of a strong commitment to the central work of the school, student achievement and welfare. there are a number of key points implicit in the data: (i) year heads have decision-making power and autonomy, enacted through the routine of meeting with teachers and parents; (ii) the metaphor of a “think-tank” suggests that expertise is shared and all points of view are considered as part of the process; (iii) the sharing of best practice implies open lines of communication and relationships based on trust and collegiality. conversely, however, a situation that is typical across all three case-study schools, the evidence also indicates that there is a hierarchical character to the structuring of team meetings. due to time constraints, meetings are not convened with special duties teachers, who also hold promotional posts of responsibility. if the leadership potential of these posts of responsibility holders is to be pro-actively enhanced and supported, it is important to convene occasional structured meetings with the role incumbents to inspire and motivate confidence in the role. if teacher leadership is to become truly transformative, there needs to be time and opportunity for teachers to focus on aspects specific to their role (harris, 2005). international electronic journal of elementary education vol.8, issue2, 243-266, 2015 254 while there was general consensus across all three case-study schools that the work and sphere of influence of the post of responsibility holder extends beyond the individual classroom, there was a lack of congruence as to whether their responsibilities constitute a key leadership role within the school organization. i wouldn’t see myself as being any more powerful than anybody else. i’m getting paid to do it. as for leadership, i suppose we work as a team really (t1). i suppose my view of leadership is probably quite narrow, because i wouldn’t be that experienced in it. but from my point of view, i wouldn’t like to sit at the back of the bus, i’d like to drive the bus (t2). my post of responsibility would be part of the leadership in the school. if leadership is not distributed around the school, it’s impossible for people to develop their own levels of leadership. people in this school are given total leadership and freedom, by the principal, to do the job (t3). i would see leadership as being a part of my middle management role, as part of my post (t4). in applying an activity-theoretical perspective as a diagnostic tool to grasp the essence of leadership practice, it is evident from the foregoing data that while teacher leadership is a key factor in the lived reality of today’s school organizations, different perspectives are rooted in different communities and historical formations. the above quotes are representative of two competing paradigms, with different conceptualizations on the power differential and authority conferred on the role. on the one hand, the views of the teachers in the first two quotes, equate power with rank, authority and control. in their conceptualizations, leadership is shared by post of responsibility holders, as part of a collaborative team, with the distribution of power and agency being located in the group. the principal is the “real leader” of the school, with the role of the middle management team being constructed in functional and egalitarian terms. implicit in this constructivist view of distributed leadership is the notion that the team derives its identity and meaning from its shared understandings and culture. also worthy of note is the fact that one interviewee constructs his role as that of mere functionary, whereby he performs tasks for which he receives remuneration. an explanation for this world-view may lie in the fact that because his school is a voluntary secondary school, legacies of traditional, hierarchical modes of governance, as discussed earlier, pertain. deep-rooted subjectivities, thus, serve to reproduce “the social structure in which it takes place” (wenger, 1998, p. 13). the alternative view is premised on the notion that in the complex activity systems that are today’s school organizations, in the distribution of leadership and agency, the power and authority to lead are vested, not alone in the principal at the apex of the organization, but are distributed, to empower teachers working in collaborative working environments. power is facilitative and enables others to accomplish something that they think is important (sergiovanni, 2001, p. 152). the metaphor of the post of responsibility driving the bus is a powerful description of a school organization which supports a flatter and more lateral type of leadership distribution. it is engaged in creating “the structural conditions or spaces where leadership can best operate and flourish” (harris, 2008, p. 40). this debate makes it clear that conceptualizations of team and power are largely dependent on the character of the school organization and on the particular historical contexts in which they are embedded. thus, the construction of the objective (leadership practice) does not happen in the challenges of distributing leadership in irish post-primary schools / o’donovan 255 “harmonious unity”, with different perspectives continuing to exist in different activity systems (engestrom, 2008, p. 129). while there is clear evidence from all three schools of distributed leadership practice, with the agents carrying out their leadership and management responsibilities with diligence, pride and commitment, due to their historical legacies, however, the concept of distributed leadership is not an explicit component of the discourse of voluntary secondary schools, which adhere to more traditional conceptualizations of middle management team construction. in light of the foregoing evidence, the point may be made that current changes in school organizations may create historically new features of intentionality and distributed agency (engestrom, 2008). in the conversations with non-post of responsibility holders, there was general consensus that posts of responsibility constitute a key leadership role within the school organizations. all teachers cited the year head system as being a key component of the middle management structure, in that it has an effective role managing student progress and behaviour. the following quotes, which represent typical responses, provide interesting data on 3 aspects of practice worthy of closer analysis. these aspects are (i) perceptions of the role, (ii) power differentials and (iii) the impact of the moratorium on promotional posts. issues concerning communication underpin all 3 aspects. i do not have a post of responsibility. i didn’t ever intend to go that route. i have no interest really in taking a post of responsibility. i just want to teach, i love teaching (t7). there are some posts of responsibility where they would get extra pay for the position. i’m a class tutor and if it [a discipline issue] gets to the stage that it’s out of control, the year head then comes in and i suppose they make a decision between the year head and the principal (t8). i’m a permanent whole-time teacher with no post of responsibility. if it wasn’t for the moratorium, i think i probably would have a post for the work that i do (t9). in the first iteration above, the interviewee provides a lens through which she positions the roles of responsibility holders. she expresses the view that she has no interest whatsoever in middle management, but derives her identity as a teacher in the classroom, encouraging and motivating students. implicit in her narrative is the notion that the holding of a post of responsibility is extraneous to her construct of classroom teacher. the point must be made, however, that the two constructs are not irreconcilable and that “communities of practice become resources for organizing our learning as well as contexts in which to manifest our learning through an identity of participation” (wenger, 1998, p. 273). in the second iteration the interviewee’s discourse indicates how she, as a class teacher (a voluntary role catering to the pastoral needs of a particular class group), positions the relationship between the principal and the year head, as post of responsibility holder, and the power differential between leader and follower. implicit in this narrative is the view that school leadership has a linear property, with a demarcation between those who have formal leadership authority and those who do not. this has implications for school communities to address and re-define how collegiality is understood and how leadership activity is coordinated and transacted. a key message in the third interviewee’s narrative, as he elucidates on his career trajectory, and congruent with that of post of responsibility holders, is that the moratorium on promotional posts adversely affects teachers’ beliefs about their working conditions and their capacity to develop leadership skills. the discourse suggests that international electronic journal of elementary education vol.8, issue2, 243-266, 2015 256 there are levels of cynicism and disempowerment presented by this barrier to teacher leadership. a key factor in the debate is the discussion on extrinsic motivation and financial recompense for post of responsibility holders. notwithstanding the fact that teachers uphold the vocational dimension of their role and derive great intrinsic reward from contributing to the development of the school organization, the current study suggests, however, that a frustration with externally-mandated demands and policy agendas has a negative impact on teachers’ motivation. it results in extrinsic concerns, pertaining to transactional models of leadership, being fore-grounded. subject department coordinators. any discussion on distributed leadership and the how of leadership practice needs to focus on the roles and responsibilities of subject department coordinators. the empirical evidence from the case-study schools indicates that the mechanisms by which subject department coordinators are appointed and the levels of agency and influence attributed to them are largely dependent on the stage of development of the individual school organization. the trajectories of leadership practice vary between schools, in some schools the posts are voluntary and rotate, while in others they are held either by a senior teacher or post holder. in addition, the scope and range of duties, responsibilities and role definitions of the subject department coordinators also vary across the case-study schools, on a continuum from a practice which is merely functional and fixed at one end, to one where the coordinators are entrusted with leadership and engage with deep student learning and classroom practice, at the other end of the continuum. the model expounded by one principal, whereby the role of subject department coordinator is a voluntary one and remains with the incumbent for as long as s/he wants, indicates that this frame of instructional leadership practice is a fixed phenomenon. key questions need to be asked regarding who has agency and where the nexus of decision-making is located. from an activity-theoretical perspective, it prompts one to explore the principle of historicity with intersecting developmental layers (engestrom, 2008). in other instances, there is an organizational intentionality to rotate the role and thereby develop the leadership skills of all members of the subject department teams. in the proposed intentionality to rotate the subject coordinators’ positions and, thereby, to create a space for the leadership capacity of all teachers, there is an alignment with the activity-theoretical perspective that current changes in work organizations have the possibility to create historically new features of collective intentionality and distributed agency. by adopting a rotational department heads’ system, there is a platform for all teachers to enact leadership and decision-making powers and, thereby, a distribution of agency ensuring that school leaders have a significant impact on student outcomes. in all instances, however, and one of the most salient findings from the research data is that the construction of subject department teams is a shared phenomenon negotiated as part of a joint enterprise based on team-work. students are at the centre of the discourse and subject department coordinators thus influence classroom practice and are “important gatekeepers to change and development within their subject areas” (harris and muijs, 2005, p. 19). the evidence also indicates that the subject department coordinator, in some instances, has a key role to play in engaging in professional dialogue with the school principal and also responds to demands for external accountability in meeting with a representative of the department of education and skills’ inspectorate. the significant divergence in practices across the case-study schools, however, prompts the need for a systems-wide debate on the enactment of this highly influential role. the challenge for school principals is to create the space for increased leadership opportunities for teachers. if this is to be achieved, both structural and cultural changes the challenges of distributing leadership in irish post-primary schools / o’donovan 257 are required within schools as “communities of practice”. if subject department coordinators are to have agency and voice in influencing high quality learning experience for students and in developing collegial relations within the subject team, it requires principals and senior management teams to create an open space for professional dialogue and the authentic development of the construct. prior research shows that giving teachers opportunities to lead has had a positive effect on the quality of relationships and teaching within schools (harris and muijs, 2005). volunteerism. principals’ and teachers’ narratives across the three case-study schools indicate that, in constructing a tableau of contemporary informal school leadership, volunteerism must be analyzed as a key lateral strategy in the enactment of leadership practice. the empirical data indicate that this voluntary model of teacher leadership is enacted (i) by non-post of responsibility holding staff taking on duties and responsibilities and (ii) by leading and influencing the extra-curricular life of the school. in exploring the volunteerist dimension of leadership from a “leader-plus” perspective, it must be remembered that the concept of volunteering is deeply-rooted in the irish post-primary system, particularly in faith-based schools, where school activities are mediated by the christian values of volunteering for the community. it is also mediated by commitment to the ethos of the school community which, in all instances, promotes the nurture, care and holistic development of the student. in spite of this agency, however, there were discernible tensions as some of the principals who were interviewed feared that the tenets of the croke park agreement, (which requires teachers to work a further 33 hours outside of school time), would adversely affect volunteerism and the spirit of collegiality in schools. from the perspectives of the teacher interviewees, the response on the part of the government to the rupture of the social and economic sphere (recession and downturn in the economy) and resultant austerity measures have resulted in a different reality for them, in how teacher leadership is framed to meet the exigencies of the newly emerging reality. a number of teacher interviewees believe that there is not as much volunteerism as formerly, with government policy and the tenets of the new national employment arrangement (croke park agreement) being the primary causal factors. there are further tensions regarding the casualization of teachers’ contracts and a view that there is more pressure to volunteer on young teachers who do not have permanent whole-time positions, as they aspire to acquire contracts of indefinite duration (cids) of fixed-term contracts. the interview data also suggest that there are tensions and emotional conflicts experienced by teachers as they seek to find a balance between the multiple demands of their roles as teachers in this new reality and the lack of acknowledgement of the enormity of teacher voluntary input in enabling school communities to function as collaborative, effective organisations. essentially in the new managerial framework, what is at issue here is a conflict of discourses regarding the governance and purpose of education and “the role of relational human beings within the process” (lynch et al, 2012). despite these factors, however, implicit in the teachers’ dialogues is the notion that identity is a fluid construct, influenced by school history and context. many teacher interviewees espouse that continuity with the school’s traditions and values is central to their identity as teachers. they hold the aspiration that voluntary activities would not be withdrawn, as volunteerism is constructed as a fundamental practice in enabling sustainable informal teacher leadership practice. in respect of the above discussion, while in activity-theoretical terms the tensions and contradictions cause disturbance and conflicts, there is an onus on the schools to produce international electronic journal of elementary education vol.8, issue2, 243-266, 2015 258 “innovative attempts to change the activity” (engestrom, 2008, p. 206). while the tensions may arise due to external mandates, there is an onus of responsibility on school leaders to assiduously develop tools and creative ways of managing the emotional care of teachers. instructional leadership an analysis on instructional leadership practice in the case-study schools is essential if we are to gain an in-depth insight into the core work of school organizations. i seek to illuminate how school leadership practice in the schools connects with its object, i.e. the core of schooling teaching and learning (spillane, 2006), and how this construct of instructional leadership converges with conceptualizations of distributed leadership practice (gronn, 2002; spillane, 2006), by examining how leadership for learning is distributed in the schools. the argument will be made that effective school leadership today must meaningfully balance and combine administrative and managerial duties and responsibilities with educational vision and the building of professional learning communities, with the quality of teaching and learning being the main priority in realizing that vision. the research findings indicate that a number of characteristics are shared by all the case-study schools in the enactment of distributed leadership practice, with many instruments being utilized in the construction of this frame. staff meetings – staff meetings constitute a key institutional routine which encourages a culture of cooperation and facilitates the distribution of instructional leadership practice. the feedback from principals indicates that there is keen awareness of the need for dialogue concerning practice. we have to start thinking about methodology much more as teachers. it has now come to the point where two out of every three [staff] meetings are about learning in the classroom (p1). very often at staff meetings we have teacher input. i would very often chair the business part of the meeting. then, if there’s input, i’d just hand over to members of staff at that stage, depending on what the topic is. we then go off in to various subject groupings and we work on some of those ideas .......... to try and percolate the ideas (p2). there is an intentionality in the imperative for organizational members to collectively reflect on teaching methodologies and innovations. this strategically developed “leaderplus perspective” enables new forms of “collaborated leadership” (spillane, 2006) and is premised on individual teachers and members of subject departments engaging in meaningful professional dialogue. implicit in the proposal for the collaborative model, is the notion that teachers assume the responsibility of devising methodologies to transform organizational learning and planning for the future direction of the school. the evidence also indicates that the use of different artefacts (e.g. outside facilitators, use of internal expertise as a leadership routine), enables a flatter leadership structure. curriculum. the teachers’ reflections mirror much of the debate and contemporary thinking on curriculum, as part of the processes of change in the twenty-first century school (collins and dolan, 2012). the key insightful considerations emerging from the narratives include: (1) curricular change is generally perceived as a change in subject content, with the move towards curricular change and reform being recommended by teachers as being an essential component in the construction of engaging learning environments for teachers and students. the lack of innovative change in some curricula perceived as having a constraining property, leading to teacher frustration and lack of agency. (2) implicit in all narratives is the overarching presence of the terminal examination as a key determinant in shaping classroom practice. as suggested by one interviewee, due to the current architecture of the examination system and because of the the challenges of distributing leadership in irish post-primary schools / o’donovan 259 huge pressure on students to get points at leaving certificate, teaching and learning practices continue to be dominated by traditional norms of practice, whereby teacher input continues to be significant and prescriptive. the challenge for teachers in leading classroom practice is to mediate inherited, traditional norms by empowering students to become self-directed learners. (3) teachers take the stance that in the construction and development of schools as professional learning communities, there is a need to engage with innovative teaching methodologies. on-going in-service training is identified as a requisite tool in enabling the acquisition of professional expertise. reflective practice. in respect of the above discussion, one principal asserts that, due to the examination system, learning is still very teacher-directed, with the public discourse on curricular content being outputs-driven. “my whole interest in teaching and learning is that we actually all become self-motivated ourselves, whether we’re teachers or children. children should be taught to be inquisitive and be encouraged to learn themselves.” (p2). the rhetoric in the foregoing extract is a powerful argument for a distributed model of reflective practice to enable depth rather than breadth of learning. it also calls for teachers and students to engage in the co-construction of knowledge in a culture of holistic organizational learning. the challenge for principals is to provide opportunities for teachers to become reflective practitioners and to refract deeply embedded subjectivities about what constitutes knowledge and learning and, in this way to engage and empower teachers to build professional learning communities. the creation of such a culture of learning also calls for continuous professional development for principals and a space for them to be self-reflective and self-evaluating. accountability. in order to extend the debate further, issues pertaining to accountability regimes, assessment procedures and teacher and student identity in the face of new technology need to be addressed. in the interviews with the teachers, elements of the discussions centre on how the demands for participation in a global knowledge economy shape the re-framing of curriculum and teacher practice. in respect of the new project maths one teacher interviewee opined as follows: “i’m not sure of the thinking behind it, but this new course is supposed to drive the results and increase levels in maths. i’d kind of be a purist in maths. i wouldn’t agree with making courses easier, i think it’s down to external forces.” another view, “if you allowed yourself become too immersed in it [accountability], you could become quite robotic. i’d never lose sight of my ethos, i’m able to reconcile the two.” the foregoing rhetorics suggest that change is carefully interrogated and negotiated, with teachers, as practitioners, deriving identity, empowerment and agency through engaging in reflective enquiry. mediating external accountability demands by clarifying and supporting the internal values and ethos of the school is a fundamental component of this reflective enquiry. the challenge for school leadership is to mediate the two paradigms, by providing a space to enable teachers to engage in reflective enquiry, to refract deeply embedded subjectivities about what constitutes deep learning and, thereby, provide leadership for school improvement. this would enable educators to become more sophisticated in their ability to develop capacity for teaching and learning in schools (sergiovanni, 2001). it is also essential that there is central investment to support principals in that endeavour. the preponderance of evidence indicates, however, that the enactment of empowering methodologies is constrained by the assessment and examination systems and adherence to prescriptive, externally mandated accountability regimens. while teachers support and recommend a move toward curricular change and reform, they are of the view that they are denied agency by being marginalized in the top-down, decision-making process. engaging teachers and positioning them centrally in the processes of curriculum design and assessment procedures is imperative if teachers are to play a significant role as international electronic journal of elementary education vol.8, issue2, 243-266, 2015 260 empowered leaders of classroom practice. this entails teachers pro-actively asserting their voice and school principals play a pivotal role in this regard. a further curricular challenge includes the integration of new technology, as an artefact, in the classroom. the unprecedented pace of technological change and innovation impacts on teaching and learning. one teacher proffered the view “students have become spectators, with the technology that they’re spending their time on. it’s now getting students to focus and engage and to take on board the leadership that if there’s a problem that they can’t work out, that they try again and again, that seems to be gone.” the rhetoric suggests (i) that the integration of new technology represents a fundamental departure from traditional norms of teaching and learning and (ii) that students are passive recipients of information and that the discourse of knowledge is determined by wider global market ideologies. what this view implies is that learning is linked to the instrumental purposes of human capital development and economic selfmaximization (rizvi and lingard, 2010). this creates new pressures for teachers, particularly for those who pre-date the technological era, to change with a curriculum that is relevant to young people’s lives, while simultaneously leading and enabling problemsolving and, thereby, fulfilling the intrinsic aims of education. the challenge for school leaders is to mediate the contrasting paradigms by providing time for dialogue and reflection to enable flexible and holistic teaching and learning environments and to enable teachers to adapt to the new technologies. how principals enact their instructional leadership roles: the principal’s instructional leadership role focuses on coordinating instruction and curriculum and requires the principal to be “hip deep” and deeply engaged in stimulating, supervising and monitoring teaching and learning in the school (hallinger, 2005). following interviews with the protagonists, four key dimensions were extracted as being pertinent to the principal’s leadership role. shared practice. the findings indicate that, in all contexts, principals are deeply committed to the pursuit of excellence and to providing a positive and supportive environment for teaching and learning. there is a recognition that, while the principal, as príomhoide– principal teacher and leader of learning in the school sets the standard and influences the learning environment, both directly and indirectly, the practice is a shared one, involving many agents. there is a broad acknowledgement of the professionalism and expertise of teachers and an assertion that the solo leader cannot lead on his/her own. this proposes a view of the school community, as a community of practice, where relationships are dynamic and where power, influence and leadership are distributed to teachers who are directly involved with teaching and student learning in the classroom. teacher learning and professional development. opportunities for continuous professional development and on-going teacher learning are in operation at many levels and seen as a priority in all case-study schools. teachers are facilitated to attend courses organized and convened by their subject associations and also those provided by the department of education and skills. the principals perceive the role of the instructional leader as being one where teachers are affirmed and encouraged to engage in life-long learning. this aligns closely with the viewpoint from the literature that, in maximizing teacher leadership and teacher learning and by placing teachers at the centre of change and development, there is greater opportunity for organizational growth (harris and muijs, 2005). sharing expertise. teacher leadership and teacher learning are further enhanced and enabled by organisational routines which focus on interactions among teachers, the object of which is to share leadership and expertise. the challenges of distributing leadership in irish post-primary schools / o’donovan 261 we’re working at a culture of sharing with the subject departments (p1). teachers talk to each other in relation to giving information and resources. it’s all communication, it can be structured and it can be with anyone. (p2). the clear intentionality in these leadership practices, both formal and informal, is to lead to improvement in classroom practice and, thereby, lead to improvement in student learning. the empirical evidence implies that leadership is conceptualized as a collective phenomenon, and in line with activity theory (engestrom, 2010), there is an intentionality to distribute agency and to unleash the collective capacity of teachers, by encouraging the sharing of expertise and bringing the construction of knowledge outside the individual classroom. it is clear from the second iteration that, the unleashing of this type of distributed agency has an emergent quality. it is intended that by sharing expertise, teachers will learn from each other and support each other. both statements imply interdependencies and co-performance of leadership routines and indicate how teachers share responsibility and how leadership practice is distributed among formal and informal leaders (harris and muijs, 2005; spillane, 2006). principals’ involvement in classroom teaching and learning. as already discussed, the focus in any learning community is the learning – are the students learning? one of the criteria used to evaluate how students learn is for the principal, teachers and students to have a collaborative dialogue and engagement on the issue and, thus, to eliminate teacher isolation and open the heretofore impenetrable classroom door. traditionally, in the irish education system, the teacher had virtual autonomy in his/her classroom. the following three elucidations indicate three diverse modes of belonging and practice and belonging. i would do it with teachers new to the school, but not with established teachers. i would not formally assess. (p1). never, i’m not in favour of the practice. leave surveillance to the inspectorate. i’m always in and out of classes with messages, you’d soon know what’s going on (p2). i would go in and find out how students are getting on. for example, if i’m taking a free class, i could be looking at copies (p3). while the evidence indicates that there is no consistency in practice across the schools, the indication is that, in line with the talis (2009) findings on direct supervision of instruction in the schools, collaborative dialogue with teachers on the learning taking place in their respective classes does not form a significant part of the culture and practice of irish post-primary schools. as evidenced in the above statements, an integral component of the principals’ instructional leadership practice in the case-study schools is visiting classrooms on a regular basis. these quick and short “classroom walkthrough” (kachur et al, 2010) represent an important tool utilized by principals to informally observe classroom practice and student learning and to form an impression on what is taking place in the classrooms. while the principals’ statements indicate that these routines are designed to observe teaching and learning, there is no explicit acknowledgement that this the function of the routines as feedback is not provided. an important dimension not included is a shared and collaborative dialogue with the teacher(s) on the learning that is taking place in the classroom. while the argument is cogently made (gladwell, 2005) that instantaneous impressions can have efficacy, the key question needs to be posed, can principals assess the quality of learning and teaching in a short observation? the argument may be made that such short international electronic journal of elementary education vol.8, issue2, 243-266, 2015 262 visits are flawed for two reasons. first, if principals are not equipped with an observation protocol and a clear understanding of the purposes of classroom visits, their observations can be ineffective (pitler and goodwin, 2009). second, if teachers and students are not engaged in the conversation, they are denied agency and are not permitted to be active members of the learning process the above analysis suggests that this aspect of leadership merits urgent discussion in the national educational landscape. because it presents a significant challenge to our schools, it may well be the new kid on the block as we move forward. the comment by one principal, leave surveillance to the inspectorate, also merits attention. it prompts the need for discussion on the role of the inspectorate in respect of where school leadership will be located in the future. conclusion across all the case-study schools, there is an explicit recognition of the complex and challenging role of the school principal, whose leadership role and identity are constructed in terms of mediating external variables, in the guise of mandated policies and accountability regimens, with the internal life of the school community. there is an implicit recognition that, in mediating these contrasting realities, the hierarchical construct is a constraint and the notion of the heroic solo leader at the apex of the organizational structure no longer pertains. among all participants, there is widespread support for a distributed model of leadership, though it is important to point out, that the concept does not explicitly form part of the discourse in the case-study schools. what the findings clearly reveal is that leadership is multidimensional with transformational, moral and shared dimensions. what is also clear from the evidence is that any attempt at analyzing leadership practice must recognize that school principals figure very prominently in the narrative, enjoying multiple leadership styles and strategies to affect leadership distribution. while implicitly, across all the case-study schools, there is an expectation that school principals create the environments to enable the diffusion and distribution of leadership practice, the tone of the discourses, however, indicates that this is a reciprocal process. while there are inconsistencies in the discourses and different constructions of leadership across the case-study schools, procedurally, however, what is clear is that leading and managing are not static activities, the whole process is relational and grounded in emotional subjectivities and organizational connectedness. a distributed leadership perspective suggests that the development of leadership needs to give careful attention to the school situation, including tools, routines and other aspects (spillane, 2006). the current research findings show how any enactment of leadership practice, in the case-study schools, is underpinned by and embedded in an ethos of care, nurture and higher-order values. what is clearly evident in participants’ narratives is that leadership practice constantly interweaves and mediates external, prescriptive legislative requirements and accountability mandates with a commitment to the school’s ethos and value system. the evidence clearly indicates that both the process and influence relations have a top-down, bottom-up and lateral character. consistent with the literature (wenger, 1998), this type of production also emphasizes the influential space occupied by teachers in the leadership process, as across the case-study schools, they express a commitment to safeguarding ethos. two salient points emerge from the evidence in respect of managerial leadership; (i) there is no single global view of what constitutes managerial leadership and (ii) the trajectories of leadership practice vary between schools, as schools are at different stages on the road to constructing agentic, distributed and enabling models of practice. notwithstanding these aspects, however, there are a number of components common to the challenges of distributing leadership in irish post-primary schools / o’donovan 263 all schools, which merit further discussion. there is considerable focus on capacitybuilding, with authority and agency being distributed and embedded, in both formal and informal structures. the formal positional structures are constructed in middle management teams, with the informal structures being more fluid and emergent and are often specific to a single task or pursuit, including extra-curricular activities. a key dimension of informal school leadership, which has a uniquely irish character and which does not receive significant attention in the distributed leadership literature, is volunteerism. the findings suggest, however, that there are fears and tensions being expressed by the participants that the recent embargo on promotional posts may adversely impact on teachers’ willingness to volunteer. this is a challenge for the distribution of school leadership, has implications for practice in schools and, in engestromian terms, opens up a new layer of work for analysis. in activity-theoretical terms, the embargo on promotional posts and attendant depletion of the middle management structure of schools, may be constructed as one of the primary disturbances or contradictions currently pervading our activity systems. a key assumption underpinning the empirical evidence is that the embargo has led to a considerable increase in the administrative and managerial work of the principal and deputy principal and detracts from their instructional leadership role. in the current austerity context, the leadership behaviours of the principal are opportunistically and strategically managed, they distribute leadership by persuasion, negotiation, asking favours and expending considerable emotional energy by employing motivational tools. regarding this point, however, as schools engage with these new and emerging realities, the challenge for school leadership is to construct the disturbance as both a challenge and opportunity to negotiate and produce new meanings. in exploring the how of leadership practice, a key finding in the evidence from the casestudy schools is that distributed models of instructional leadership practice are variously enacted. while all principals espouse that they are instruction-oriented and are strongly committed to student learning and achievement, practices are configured differently in schools, largely determined by school histories and contexts. in some instances, strong collaborative cultures exist, where new forms of “co-performance” are enacted, for example, as subject departments engage in meaningful professional dialogue, reflective and innovative practices. in other instances, more traditional forms of engagement exist, with collaborative cultures being more evolutionary in character. across all the case-study schools, however, a significant finding, and meriting further research and discussion, is that the isolationist culture continues to pertain. traditionally in ireland, the teacher has had virtual autonomy in the classroom, operating behind a “closed door” culture. across the case-study schools, the principals express a reticence to counter that culture, in deference to staff sensibilities and micro-politics and to remnants of a culture where the powerful teacher unions vehemently supported the “closed door” system. from a distributed instructional leadership perspective, this presents challenges to principals and school communities to negotiate meaning anew. the empirical evidence raises critical questions concerning the future direction of teacher-leadership and the distribution of that leadership in irish post-primary schools, and how it might be operationalized. the findings also reveal that there is a challenge for school leaders to make visible and explicit the concept of distributed leadership. meaning must be negotiated internally in school organizations by looking at constructive, creative changes and leadership configurations that would have the greatest potential for the development of school leadership. as there is a dearth of research in the irish educational international electronic journal of elementary education vol.8, issue2, 243-266, 2015 264 context, there needs to be a contextual programme of leadership research drawing on diverse methodologies. • • • margaret o’donovan is principal of regina mundi college, cork. the work for this paper was undertaken as part of the author's phd research at the school of education, university college cork. margaret has particular expertise and interest in the area of educational leadership and contributes to staff development for aspiring school principals and she also supports teacher education students doing pedagogic research as part of their professional masters programme. references bottery, mike (2004). the challenges of educational leadership. london: paul chapman. boyatzis, richard e. (1998). transforming qualitative information thematic analysis and code development. thousand oaks, ca: sage. collins, tom, dolan, rose (2011). leadership and the curriculum – a question of trust. in leading and managing schools, ed. helen o’sullivan, john west-burnham, 75-88. london: sage publications. creswell, j.w. (2007). qualitative inquiry & research design (2nd ed.). thousand oaks, ca: sage publications. de lima, jorge avila (2008). department networks and distributed leadership in school. school leadership & management, vol. 28, no. 2, 159-187. engestrom, y. (1999a). communication, discourse and activity. the comminication review, 3(1-2), 165-185. engestrom, y. (1999b). expansive visibilization of work: an activity-theoretical perspective. computer supported cooperative work, 8, 63-93. engestrom, yrjo (2008). from teams to knots. activity-theoretical studies of collaboration and learning at work. cambridge university press. evans, m.g. (1996). r.j. house’s a path-goal theory of leader effectiveness. the leadership quarterly 7, 305-309. fullan, michael (2001). leading in a culture of change. san francisco: jossey-bass. gladwell, m. (2005). blink: the power of thinking without thinking. new york: little brown and co. gronn, peter (2000). distributed properties, a new architecture for leadership. educational management and administration, vol. 28(3), 317-338. gronn, peter (2002). distributed leadership as a unit of analysis. the leadership quarterly 13, 423-451. gronn, peter (2003). the new work of educational leaders: changing leadership practice in an era of school reform. london: paul chapman. gronn, peter (2009). from distributed to hybrid leadership practice. in distributed leadership, different perspectives, ed. alma harris, 197-217. milton keynes: springer the challenges of distributing leadership in irish post-primary schools / o’donovan 265 gronn, peter (2009). hybrid leadership. in distributed leadership according to the evidence, ed. kenneth leithwood, blair mascall & tiiu strauss, 17-40. routledge: new york. grubb, norton w., flessa, joseph j. (2006). “a job too big for one”: multiple principals and other non-traditional approaches to school leadership. educational administration quarterly, vol. 42, no. 4, 518-550. hallinger, p. (2005). instructional leadership and the school principal: a passing fancy that refuses to fade away. leadership and policy in schools, 4(3): 221-239. hallinger, p. & heck, r. (2003). understanding the contribution of leadership to school improvement. in m. wallace & l. poulson, ed. learning to read critically in educational leadership and management. london: sage publications. harris, alma (2005). teacher leadership: more than just a feel-good factor? leadership and policy in schools, 4: 201-219. harris, alma and daniel muijs (2005). improving schools through teacher leadership. london: open university press. harris, alma and spillane, james (2008). distributed leadership through the looking glass. management in education, vol. 22(1): 31-34. heck, r. & hallinger, p. (1999). next generation methods for the study of leadership and school improvement. in j. murphy & k. louis, ed., handbook of research on educational administration, 141-162. san francisco: jossey-bass. hutchins, e. (1995). cognition in the wild. cambridge, ma: mit press. hutchins, e. (1995). how a cockpit remembers its speed. cognitive science 19: 265-288. kachur, d.s., stout, j.a., edwards, c.l. (2010). classroom walkthroughs to improve teaching and learning. larchmont, ny: eye on education, inc. kvale, steinar, brinkmann, svend (2009). interviews learning the craft of qualitative research interviewing. thousand oaks: sage publications. leont’ev, a.n. (1981). the problem of activity in psychology. in j. wertsch, ed., the concept of activity in soviet psychology. armonk, ny: m.e. sharpe. macbeath, john (2009). distributed leadership: paradigms, policy and paradox. in k. leithwood, b. mescall & t. strauss (eds,), distributed leadership according to the evidence. new york: routledge. lynch kathleen, bernie grummell and dympna devine (2012). new managerialism in education. commercialism, carelessness and gender. london: palgrave macmillan. merriam, sharan b. (2009). qualitative research. a guide to design and implementation. san francisco: jossey-bass. moller, jorunn (2005). old metaphors, new meanings: being a woman principal. in c.sugrue, ed., passionate principals, learning from the life histories of school leaders. abigdon, oxon: routledge. moller, jorunn (2009). approaches to school leadership in scandanavia. journal of educational administration and history, vol. 41, no. 2, 165-177. oecd (2009). creating effective teaching and learning environments: first results from talis, teaching and learning international survey. paris: oecd. pitler. h. & goodwin, b. (2009). classroom walk-throughs: learning to see the trees and the forest. the learning principal, vol. 4, no. 4. ritchie, ron and woods, philip a. (2007). degrees of distribution: towards an understanding of variations in the nature of distributed leadership in schools. school leadership and management, vol. 27, no. 4, 363-381. international electronic journal of elementary education vol.8, issue2, 243-266, 2015 266 rizvi, fazal, lingard, bob (2010). globalising education policy. london: routledge. sergiovannni, thomas j. (2001). leadership, what’s in it for schools? new york: routledge. spillane, james p. (2005). distributed leadership. the education forum, vol. 69: 143-150. spillane, james p. (2006). distributed leadership. san francisco: jossey-bass. spillane, j., halverson, r. and diamond, j.b. (2004). distributed leadership: toward a theory of school leadership practice. journal of curriculum studies, vol. 36, no. 1, 3-34. spillane, j.p., camburn, e.m. & stitziel pareja, a. (2009). school principals at work: a distributed perspective. in k. leithwood, b. mascall & t. strauss, eds., distributed leadership according to the evidence. new york: routledge. sugrue, ciarán (ed.) (2005). passionate principalship, learning from the life histories of school leaders. london: routledge. thomas, gary (2011). how to do your case study, a guide for students and researchers. london: sage publications. timperley, h. (2009). distributing leadership to improve outcomes for students. in k. leithwood, b. mescall & t. strauss eds., distributed leadership according to the evidence. new york: routledge. vygotsky. l.s. (1978). mind in society: the development of higher psychological processes. cambridge, ma: harvard university press. wenger, etienne (1998). communities of practice learning, meaning and identity. cambridge: cambridge university press. wenger, etienne (2008). communities of practice learning, meaning and identity (18th printing). cambridge: cambridge university press. yin, robert k. (2009). case study research, design and methods. thousand oaks: sage publications. © 2022 published by kura education & publishing. this is an open access article under the cc byncnd license. (https://creativecommons. org/licenses/by/4.0/) 357 copyright © www.iejee.com issn: 1307-9298 international electronic journal of elementary education march 2023, volume 15, issue 4, 357-369 the agenda of the reading teacher journal on reading and reading skills: a corpus analysis in the last decade turan temura,*, taner sezerb abstract reading is one of the main skills to be learned and used in schools and through the entire life of an individual. its crucial importance made reading one of the central topics in different academic disciplines such as education, psychology, linguistic and neuroscience. advances in multidisciplinary approaches to and studies of reading gave us the reason to talk about the science of reading. improvement in research into reading during the last century reveals the fact that different approaches emphasized the importance of partially different aspects of reading skills. different approaches presented sound arguments for different priorities in teaching and learning of reading. for some decades ago strong dichotomic arguments were so sharp that in the united states the reading literature talked about ‘the reading wars’. during the recent decades, the debate of reading and approaches to reading, reading skills and reading processes have changed. studies of reading presented different arguments for partially different priorities. the reading teacher (rt) has for many decades been the one of the most important arenas for the presentation of new findings, ideas, arguments, and trends. our present corpus-based study captured the frequent collocations and their usage in the rt publications – 952 articles comprising 3 548 008 words during the last decade, 2012-2021. treetagger was used for part-of-speech tagging and cqpweb/cwb framework was employed as the corpus interface in the analysis process. the findings clearly revealed that "reading comprehension" was the most frequently used collocation, and it has never weakened its position in the rt publications in the last decade. while instructional activities and processes like ‘shared reading’, ‘during reading’, ‘while reading’, and ‘after reading’ were given considerable attention, ‘before reading’ was almost forgotten. findings also show that linguistic diversity among students, particularly in the us, also made an impact on the emerging of new topics and thus frequency of concepts like bilingual, multilingual, translanguaging, and coaching. findings related to the ascending and descending trends reading collocations in rt publications regarding other aspects of reading, teaching of reading, and reading processes are discussed and some ideas for future research are presented. keywords: reading, collocations, corpus, reading comprehension received : 13 december 2022 revised : 22 february 2023 accepted : 10 march 2023 doi : 10.26822/iejee.2023.288 a,* corresponding author: turan temur, department of elementary education, faculty of education, anadolu university, turkey. e-mail: turantemur@gmail.com orcid: https://orcid.org/0000-0002-9410-8295 b taner sezer, department of english language and literature, faculty of humanities and social sciences, mersin university, turkey. e-mail: tanersezerr@gmail.com orcid: https://orcid.org/0000-0002-7328-7650 358 march 2023, volume 15, issue 4, 357-369 introduction reading is critical for children's success in school and social life. therefore, the science of reading aims to explain the process of teaching and learning reading to educators, families, and students. this helps them to gain a deeper understanding of this vital skill. this research study can be considered as a new step towards understanding the reading skill. it is a result of a corpus-based analysis of the studies on reading and teaching reading published in the reading teachers (rt) in the last ten years, 2011-2021. reading skill has been the subject of research for those working in the fields of education, linguistic, cognitive science and cognitive neuroscience for years. while researchers working in the field of education and language focus on the applications of reading in the classroom and teaching methods, cognitive science and cognitive neuroscience focus on the cognitive processes of this skill and the functions of the brain during reading. according to the results of these studies, seidenberg (2013) emphasizes that the reading-based functions of the brain and the mechanisms underlying students' basic reading skills can be explained with neuroimaging and computational models. when it comes to the approaches to reading, one can find many similarities but also some differences. according to anderson, hiebert, scott and wilkinson (1985) “reading is the process of constructing meaning from written texts. it is a complex skill requiring the coordination of a number of interrelated sources of information.” (p. 7). the researchers emphasize the complexity of reading and the need to coordinate various sources of information. in another approach to reading wixson, peters, weber and roeber (1987) consider it as “the process of constructing meaning through the dynamic interaction among: the reader's existing knowledge; the information suggested by the text being read; and the context of the reading situation.” ( p. 750 ). here one can see that the researchers emphasize the dynamic interaction between the reader's knowledge, the text being read, and the reading context. another approach to reading by torgesen, houston, rissman and kosanovich, (2007) is more result-oriented compared to the previous approaches. the researchers take into account the competencies that students should acquire at the end of primary school and state the following: “…students be able to identify the words on the page accurately and fluently; that they have enough knowledge and thinking ability to understand the words, sentences, and paragraphs; and that they be motivated and engaged enough to use their knowledge and thinking ability to understand and learn from the text.” (p. 1). they also stress the importance of accurate word recognition, thinking ability, motivation, and engagement in reading proficiency, as well as “knowledge”. this last point as ‘content knowledge’ was given even more attention and seen as an important factor for reading in general and reading comprehension in particular by the american national governors association (2013) when they discuss the prerequisites for reading: “reading proficiency requires three sets of interrelated skills that develop over time: language and communication, mechanics of reading, and content knowledge.” (ngacbp, 2013 p. 3). here they also identify ‘language and communication’ and ‘mechanics of reading’ along with content knowledge as important factors for reading proficiency. although these approaches to some extend emphasize different aspects of reading proficiency and the skills it encompasses, it is possible to say that they have similarities in some basic points; a) all of the statements acknowledge that reading involves the construction of meaning from written texts. b) they all recognize that reading proficiency requires a combination of skills and abilities. c) they all emphasize the importance of the reader's existing knowledge and engagement in the reading process. considering the components that the above mentioned approaches include regarding the reading skill, it can be said that it is a multidimensional and complex developmental process. several researchers (kendeou, van den broek, white, & lynch, 2009; grabe, 2009) describe this process in two stages: one is lowerlevel processes, including word recognition, syntactic parsing, and meaning encoding, and the other is higher-level, including text model formation, situationmodel building, inferencing, executive-control and strategic processing. it seems that the acquisition and development of reading skills is a gradual process that starts with the letter-sound relationship and leads to capturing and interpreting the meaning in the text. the focus of rt: evidence-based literacy instruction rt is a respected journal in its field with its theoretical and applied studies on reading and teaching reading. the journal is published biannually by wiley-backwell on behalf of the international literacy association (ila, 2023). rt has a publishing policy that covers a wide range of topics related to literacy. within the framework of this policy, publications are made in the form of theoretical and practical research, book reviews and sample applications. with these publications, the journal can be described as the 359 the agenda of the reading teacher / temur & sezer meeting point of teachers, researchers, policy makers and educators interested in literacy skills. rt is a peer-reviewed journal. it publishes researchbased best practices for literacy studies. the journal has a wide readership such as classroom teachers, literacy coaches, reading specialists, researchers, and teacher educators. the journal is also indexed in various academic databases and indexes such as web of science, social sciences citation index (clarivate analytics), scopus (elsevier), eric, proquest professional education and academic search complete (ebsco). it is true to claim that unlike several other academic journals, rt presents theoretical and methodological approaches to reading and teaching reading. in addition, rt publications have teaching and learning of reading its main area of focus. these qualities made rt as a journal of cuttingedge studies of reading and a respected voice of science of reading. previous studies on rt the first one of the earlier studies of the focus of rt (staiger, et al., 1992), with regard to reading, concluded that the articles in the journal from 1950 thru 1954 did the following: 1) they informed classroom teachers about current research and promising projects on teaching reading, 2) they provided practical information on classroom activities and reading materials for teachers and students, and 3) they provided information about the international council for the improvement of reading instruction and its local councils. in the other five studies, researchers (jerrolds, 1992; stahl & fisher, 1992; pearson, 1992; dillon, et al., 1992 and mohr, et al., 2017) analysed the articles in rt which were the milestones for trend topics and the journal through content analysis. for instance, jerrolds (1992) determined the topics as milestones in teaching reading from 1947 to 1991. among these topics, elementary reading, secondary reading, suggestions for teachers, classroom reading teacher, reading comprehension, modern instructional techniques, parent-teacher cooperation, literacy-related drawings created by children, and reading educators were mentioned. one of the most striking points in this study was that rt published articles from countries such as finland, then russia, bulgaria, ukraine, mexico and china after 1982. stahl and fisher (1992) examined the first 20 years of rt and emphasized that the following four different reading related pedagogical issues were the major themes: a) reading readiness, b) word recognition, c) reading comprehension, and d) reading disabilities. in the study conducted by pearson (1992), the issues of rt between 1968-1990 were included. it was found that 22 different topics were addressed. among these topics, the most recurring were the following: a) integrated language arts, b) children's literature, c) comprehension, d) language and cultural influences, and e)testing/assesment and adminstrating reading programs. dillon, et al. (1992), on the other hand, identified the following most frequent article topics by quantitative analysis of 2,168 articles in rt between the years 1948-1991: a) instructional strategies, b) assessment, c) children's literature, d) emergent literacy/reading readiness, e) attitudes/habits/interests, and f) crosscultural and reading problems in a recent study, mohr et al. (2017) examined the articles in rt published between 1992-2016 and found the following most frequently used concepts: a) instructional, b) children's literature, c) writing, d) attitudes/habits/interests, e) comprehension, f) teacher education, g) assessment, h) content reading, and i) technology, and cross-cultural. these findings, generally obtained by using keywords, are very important in terms of analysing the content of rt. however, there is another option through which rt as a journal of an outstanding role in the field of reading can be analysed. in our view a corpus-based study. the aim of the present study the aim of this corpus-based study is to add some valuable ideas and perspectives to the existing literature and expand our knowledge base. through conducting a corpus-based study on the millions of words used in 952 articles published in rt during the period of from 2012 thru 2021, we identified the words which the concept of reading often form collocation with. by doing so, we aimed to answer the following research questions: 1. what are the most frequently used concepts in the rt corpus? 2. what concepts frequently form collocations with "reading" in the rt corpus? what is the distribution of these collocations by year? 3. what are the adjectives that form collocations with "reading" in the rt corpus? what is the distribution of these collocations by year? 4. what are the verbs that form collocations with "reading" in the rt corpus? what is the distribution of these collocations by year? 5. what are the major concepts in the first five and last five years of the rt corpus? 360 march 2023, volume 15, issue 4, 357-369 we hope that the findings of our study will expand our knowledge about the advances in reading research and the current areas of focus in the science of reading. methodology corpus linguistics is primarily a quantitative approach, relying on numerical data representing words and phrases within corpora (mcenery & hardie, 2011). therefore, statistics plays a vital role in corpus linguistics by facilitating the effective processing and analysis of quantitative information (brezina, 2018). corpora are the tools used for linguistic analysis of datasets, providing qualitative and quantitative data about a particular language. corpus has been defined in the literature by considering its different characteristics. sinclair (1991) defines a corpus as "a collection of texts selected and assembled according to certain linguistic criteria that can be used as examples of language", while leech (1992) describes it as "a large amount of data collected in a digital environment (computer)". mcenery and wilson (2001) define corpus as "a collection of texts carefully selected to represent a language", while togninibonelli (2001) defines it as "a collection of texts put together for linguistic analysis, supposedly capable of representing a language.” these conceptions of corpus emphasize three key elements of corpus features: a) they are "structured", b) they "consist of large amounts of data" and c) they are "computer-readable". thus, it is reasonable to consider a corpus as "a collection of large volumes of digitized language data brought together for a specific purpose." studies using corpus provide information about the structure of the language, its use, finding and observing various patterns, the change and features of the language. corpus linguistics is a discipline that encompasses the preparation processes of corpora and studies using them. “brown corpus” (kucera & francis, 1967) is accepted as the first corpus prepared on digital platforms. with the increase in data processing and storage capacities, especially after the 1990s, large corpora such as british national corpus (bnc) (2023), corpus of contemporary american english (2023) and bank of english (2005) were prepared and corpus linguistics has become an accepted discipline. corpora are also defined according to the "data" they contain. the ones that attempt to present a general profile of a language are referred to as "reference corpora", while the corpora prepared based on a specific topic or a particular type of text or source are called "specialized corpora". the corpus prepared within the scope of this study and serving as the main source of it is also a specialized corpus. rt corpus the corpus for this study was created by including all the studies published online versions of the rt in the last decade (2012-2021). the main reason for taking the last ten years as a reference is that the content analysis of the journal was made with keywords until 2016 and the last five years were not resolved. considering that the number of words used in the first five years would not be sufficient for a corpus analysis, the second fifth year backwards was also included in the study. online publications of rt in the last decade were recorded based on the year, volume and issue. the data were then transformed into raw text. figures, tables, graphic information, and footnotes were excluded from the data. each text was annotated with “year, issue, volume, keyword 1 and keyword 2” tags. treetagger was used for part-of-speech tagging, and cqpweb/cwb framework was employed as the corpus interface. the whole data formed a corpus of 3,548.008 words as one can see in the table 1 below: table 1. the reading teacher corpus metadata metadata total number of corpus texts 952 total words in all corpus texts (token) 3.548.008 word types in the corpus 54.295 type/token ratio 0.0153 collocations phillips (1983, 1985) offers the theoretical foundation for collocation networks, which examines the connection between collocates and macrostructure within a text. according to phillips (1989), these networks, also referred to as "lexical networks," can be utilized to implement the psychological concept of a text's "aboutness.". in essence, collocation networks serve to understand a text's central themes and ideas (vaclav, mcenery, & wattam, 2015). in reading teacher, the concepts that formed collocations with the concept of reading were examined in the following three different ways: a) the first collocations (such as reading comprehension, close reading, reading recovery) consisted of the most frequently used concepts with reading regardless of the word types. b) in the second stage, concepts that described the concept of reading and collocated with an adjective or similar words (such as independent reading, oral reading and repeated reading) were examined. c) in the third stage, the verbs that formed collocation with reading were identified and discussed. concepts that formed collocation with reading are classified within themselves. 361 the agenda of the reading teacher / temur & sezer keywords list for sub-corpora rt corpus was divided into 2012-2016 and 2017-2021, and two different corpora were formed. these two corpora were compared in terms of concepts used in the first five years and the ones in the last five years. in this way, the concepts with ascending trend and the concepts with descending trend in the rt corpus. analysis in the data analysis of rt, word forms and maximum window span were taken as +/-5 in determining the concepts that formed collocation with reading. in filtering the obtained collocations as word types, +/-3 distances and at least five (5) frequency values were taken into account. analyses were conducted considering the total number in the whole corpus, expected collocate frequency, observed collocate frequency, the number of texts they appeared in and log-likelihood values of the concepts forming collocation with reading. findings in this section, the most frequently used words in rt, the nouns, adjectives, and verbs that form collocations with the concept of reading, and the usage of the collocations by years were presented. in addition, the rt corpus was divided into two five-year periods as mentioned above. by doing this, we identified the ascending and descending trends in the use of those collocations. table 2. the most frequent ten words in rt corpus number word frequency 1 students 40347 2 reading 19919 3 teachers 13117 4 text 10975 5 writing 9709 6 literacy 8264 7 words 8156 8 use 8076 9 children 7878 10 texts 7417 table 2 shows the most frequently used content (lexical) words in the articles published in reading teacher in the last decade. these words show which concepts were focused intensively in the academic studies published in rt in the last ten years. based on this, it can be said that the studies on reading in rt focused on the following concepts of a) reader [students and children] (marked with blue colour), b) reading and writing ability (marked with gold colour), c) teacher (s) (marked with green colour) and d) reading material [ text, texts, and words] (marked with orange colour) a vast majority of the articles stress the important role these elements in the teaching-learning activities play an important role in the improvement of reading/ reading comprehension. the reason for the high frequency of these words might be due to the fact that the study area of the journal is directly related to a) twelve-year-old children, b) teaching contentreading skills for these children, and c) educators providing this education. thus, it is reasonable to say that research efforts in the field in the mentioned decade, considered the focus on students, their reading and writing, their teachers and the reading materials (texts and the vocabulary the students encounter in the reading materials) as important areas for reading research. one can raise the question: why so much emphasize on these areas? we hope to discuss this question by the help of our findings about reading collocations presented in the following table: in table 3, the first twenty words that form a collocation with the concept of reading are listed according to their log-likelihood values. as one can see in the table (table 3) the concept of comprehension is the one that collocates with reading most. considering the loglikelihood value of comprehension and the frequency of its occurrence and collocation in the corpus, it can be said that the last decade’s reading studies mostly focus on understanding the content of the texts. the findings also reveal that the content focus is on sub-concepts such as reading practices/instructional routines [guided, independent, close, oral, aloud and share], intervention/support [recovery, specialist, specialists], time of reading [during, while, after], understanding and accuracy [achievement, fluency, comprehension], standards [programs], relationship [writing] and reader-based components [motivation, difficulties] are the concepts frequently collocate with the concept of reading. we wanted to conduct a more detailed analysis of these collocations during the last decade (2012-2021) and see whether there is any difference between the first five-year (2012-2016) and the last five-year period (2017-2021). the following figure (figure 1) presents the results: in figure 1, the usage frequency of the concepts collocating with reading within one million words in the last ten years is shown according to years. the results show the following: a) “reading comprehension, reading, and writing relationship and reading instruction” were frequently used within a million words (f ≥= 100) almost every year in the last decade. b) “close 362 march 2023, volume 15, issue 4, 357-369 table 3. reading collocations word total no. in the whole corpus expected collocate frequency observed collocate frequency in no. of texts log-likelihood value 1 comprehension 3433 115.64 923 270 2440.065 2 close 915 30.822 532 86 2392.052 3 guided 931 31.361 484 126 2025.445 4 independent 927 31.226 442 141 1748.987 5 recovery 162 5.457 158 29 1034.431 6 oral 1100 37.053 344 92 1018.649 7 during 3866 130.226 611 286 994.173 8 fluency 1053 35.47 319 98 922.58 9 writing 9709 327.046 980 378 895.342 10 specialist 207 6.973 145 49 734.989 11 motivation 1206 40.624 294 55 717.169 12 specialists 277 9.331 159 33 708.613 13 achievement 723 24.354 232 103 699.896 14 aloud 1028 34.628 270 137 699.551 15 shared 1739 58.578 337 112 672.056 16 instruction 6771 228.08 669 256 590.058 17 while 2286 77.004 356 200 569.633 18 difficulties 295 9.937 125 41 457.377 19 programs 763 25.702 182 47 435.889 20 after 2524 85.021 327 210 422.374 figure 1. distribution of collocations by years 363 the agenda of the reading teacher / temur & sezer reading” and “independent reading” were used more frequently in the first years, but their usage frequency has decreased recently. c) the use of independent reading in 2012 (f = 513.28) was the highest of the most frequent collocations in the last ten years, but the frequency of use (f = 111.31) in 2013 dropped by about one-fifth, and this descending tendency continued gradually after 2018. d) independent reading, reading comprehension, guided reading and reading instruction were the most frequently used collocations in 2012. e) the results show a clear tendency: f) reading comprehension has never fallen off the agenda of rt, even with the least usage rate (f = 161.39) in 2019 and in the following years. g) the year of 2021 is the year in which reading comprehension was used the most frequently (f = 415.78). h) in addition to reading comprehension, mostly used collocations in recent years were reading-writing (f = 226.79), reading motivation (f = 141.11), shared reading (f = 201.59) and reading instruction (f = 251.99). these findings clearly show that understanding, instruction, writing and instructional routines are among the main concepts that collocate with reading. although they have different usage rates according to years, they constitute the main areas of reading research and practicing research-based reading instruction. we wanted to go even further and identify the kind of reading activities the researchers emphasized during the period of 2012-2021. in order to do this, we identified the most frequent reading collocations predominantly with adjectives and similar type of descriptive words. the following table present the results: in table 4, the first twenty adjectives that collocate with reading are listed according to their log-likelihood values. these concepts, frequently used in rt corpus, can be categorized and discussed with focus on the way they describe the nature of reading. the results show the following: a) adjectives that describe the concept of reading mostly consist of instructional routines [close reading, fluent reading, independent reading, oral reading, repeated reading, shared reading, silent reading, choral reading, dialogic reading, take-home reading, wide reading]. b) the log-likelihood value of take-home reading (programs) is high, and it is likely to be used in future studies considering the observed value and the number of texts it is used. c) some of the collocations in the table are towards improving the basic reading skills [basal reading, early reading and foundational reading]. d) reading format/medium [online reading and traditional reading] can be considered as another category. e) the frequency of reading online is higher than traditional (paper-based) reading. among the collocations, in spite of its observed value and being mentioned in just one text during the last ten years, the log-likelihood value of canine-assisted reading is observed to be high. canine-assisted reading is usually found in programs for children with special educational needs. this can be a topic for another project. table 4. the most frequent reading collocations with adjectives no. word total no. in the whole corpus expected collocate frequency observed collocate frequency in no. of texts log-likelihood value 1 close 915 30.822 474 81 1979.048 2 independent 927 31.226 434 141 1696.99 3 oral 1100 37.053 331 89 952.506 4 repeated 407 13.71 106 44 272.687 5 fluent 181 6.097 67 52 223.625 6 shared 1739 58.578 198 74 215.518 7 silent 160 5.39 60 25 202.063 8 choral 47 1.583 34 23 176.035 9 online 1363 45.912 157 33 173.643 10 take-home 33 1.112 23 1 116.177 11 early 1810 60.97 158 84 112.408 12 informational 1798 60.565 157 73 111.758 13 dialogic 238 8.017 47 12 95.31 14 wide 366 12.329 57 33 91.119 15 voluntary 32 1.078 19 8 86.511 16 basal 129 4.345 33 9 83.666 17 canine-assisted 15 0.505 13 1 76.515 18 informal 220 7.411 38 23 67.693 19 traditional 701 23.613 71 36 64.965 20 foundational 225 7.579 35 24 55.875 364 march 2023, volume 15, issue 4, 357-369 in figure 2, the frequency of the use of adjectives that collocate with reading in one million words in the last decade is shown by years. the displayed results clearly show the following: a) independent, close and oral are the adjectives that frequently describe the concept of reading. b) independent reading is the most frequently used collocation with its usage rate in 2012 (f=500.99). the usage rate of this collocation shows a descending tendency after 2018 (f=160.78). close reading, on the other hand, shows an ascending tendency in 2012, 2013 and in 2014 its collocation reached the highest usage rate (f=332.54). in the following years its usage rate decreased to its lowest level in 2019 (f=65.12). c) although the usage rate of shared reading, early reading, traditional reading, and informal reading in research studies varies from year to year, there is an ascending tendency in their usage in recent years. d) the usage rates of fluent reading, silent reading and wide reading varies from year to year, their usage shows a descending trend in recent years. e) in the following section, we went further and tried to find the most frequent reading collocations with verbs. by doing so, we expect to identify the expected activities related to reading and/purposes of reading activities. the following table show the results. in table 5, the first twenty verbs collocating with reading are listed according to their log-likelihood values. in our view, the verbs that form collocations with reading can be classified under five different groups considering their usage: a) action verbs to achieve a goal or solve a problem about reading [implement, deliver, adjust, monitor, facilitate] b) improvement verbs to make reading better or more effective [improve, enhance, stimulate, raise] c) goal-oriented verbs to achieve a specific outcome or objective for reading [persist, increase, achieve, promote] d) problem-solving verbs to address a challenge [differentiate, prevent, solve] e) impact verbs to influence or affect reading [affect, promote, deliver and monitor]. some of the collections may fit into more than one group, depending on their context. the above mentioned reading related activities and purposes reminds us of the importance of teaching-learning related activities that usually are incorporated or expected to be incorporated in reading. they remind us about the necessary activities and purposes when educationists are targeting improvement of the students’ reading comprehension. the following figure shows the distribution of reading collocations with the above mentioned verbs by year. as one can see in figure 3, the frequency of the use of verbs that collocate with reading in the last decade of rt varies from year to year. the results can be summarised as follow: a) the verb improve is the most frequently used one. although its usage was low in 2013 (f = 3.18) and 2018 (f = 2.59), its usage rate is quite high in figure 2. distribution of reading collocations with adjectives 365 the agenda of the reading teacher / temur & sezer table 5. the most frequent reading collocations with verbs no. word total no. in the whole corpus expected collocate frequency observed collocate frequency in no. of texts log-likelihood value 1 improve 554 18.661 83 60 127.157 2 enjoy 261 8.792 28 24 27.958 3 persist 49 1.651 10 9 20.898 4 increase 623 20.986 43 36 18.484 5 enhance 474 15.967 35 30 17.678 6 affect 295 9.937 25 16 16.817 7 promote 537 18.089 32 27 9.064 8 differentiate 117 3.941 11 10 8.914 9 prevent 63 2.122 7 3 7.355 10 solve 365 12.295 22 14 6.462 11 stimulate 59 1.987 6 4 5.523 12 achieve 252 8.489 15 14 4.233 13 implement 395 13.306 21 18 3.934 14 reduce 97 3.267 7 3 3.352 15 deliver 61 2.055 5 3 3.152 16 keep 573 19.301 27 21 2.836 17 monitor 208 7.006 11 10 2.016 18 facilitate 380 12.8 18 16 1.947 19 adjust 123 4.143 7 6 1.698 20 raise 147 4.952 8 7 1.645 figure 3. distribution of reading collocations with verbs by year 366 march 2023, volume 15, issue 4, 357-369 other years. the usage rate of improve in 2012 (f = 46.1), 2015 (f = 29.57), 2016 (f = 28.69), 2017 (f = 50.71), and 2021 (f = 27.72) is higher than all other verbs. b) the usage rate of the verb improve in 2017 is the highest one among the first twenty verbs that form a collocation with reading. c) improve reading is generally used to improve reading comprehension, reading abilities, achievement, reading fluency, reading outcomes, reading rate. similarly, it is seen that increase and enhance are used to improve and increase students' reading level, reading comprehension, reading motivation, reading fluency, automatic, reading development skills frequently in recent years. these findings show that the studies published in rt focused more on the development of reading-related skills in the last decade. the verb prevent used only in 2016 (f=18.26) in the last ten years is among the first twenty actions with the observed collocate frequency, in number of texts and log-likelihood values, and it is observed that it is used with the concepts of reading difficulties, reading problems and reading failure. the frequency of the words, adjectives and verbs that are collocated with reading in research articles tells us not only the main focus in the studies of reading, they also gives readers an important information about what to be aware of when one work with reading and what and how the educationists must target to improve the skills that contribute to the improvement of good readers i.e. readers who possess the necessary skills and strategies necessary to comprehend a written text at an age appropriate level. we mean that the keywords in the scientific research articles have a similar function. in the following section, we present the results of our corpus study in which we looked at the frequency of the keywords in the first five years (2012-2016) compared to the last five years (2017-2021): the data in table 6 is based on the two sub-corpora of rt corpus. the first sub-corpora shows the period of 2012-2016 (freq 1) and the second one for 20172021 (freq 2). based on this, the first five years and the last five years were compared, and the frequently used concepts and their usage were presented. the results indicate the following: a) when the frequency values are compared, children, common core state standards (ccss) reading, e-books, picture, child, e-book and read were used more in the first five years compared to the last five. b) in the last five years, the concepts of literacy, multilingual, translanguaging, linguistic, coaching, bilingual, preservice, languages, spanish, practices and running were used more than the first five years. table 6. keywords first five years (2012-2016, freq 1) compared to last five years (2017-2021, freq 2) no word freq 1 freq 1 (per mill) freq 2 freq 2 (per mill) +/log-likelihood 1 children 5287 3152.42 2591 1384.91 + 1262.58 2 ccss (*) 655 390.55 28 14.97 + 783.84 3 literacy 3018 1799.51 5246 2804.03 390.17 4 multilingual 56 33.39 520 277.94 382.14 5 reading 10723 6393.68 9196 4915.33 + 345.56 6 translanguaging 23 13.71 350 187.08 309.77 7 e-books 247 147.28 15 8.02 + 274.44 8 standards 1222 728.63 628 335.67 + 264.75 9 linguistic 159 94.81 603 322.31 229.58 10 coaching 149 88.84 576 307.88 224.05 11 bilingual 143 85.26 556 297.19 217.65 12 preservice 54 32.2 356 190.28 217.12 13 languages 270 160.99 757 404.62 190.35 14 picture 1449 863.98 916 489.61 + 186.62 15 child 902 537.82 476 254.43 + 184.64 16 e-book 164 97.79 10 5.35 + 182.04 17 spanish 247 147.28 698 373.09 177.82 18 practices 962 573.6 1796 959.98 173.38 19 read 4029 2402.32 3317 1772.96 + 169.25 20 running 92 54.86 388 207.39 165.43 (*) ccss: common core state standards in the usa. 367 the agenda of the reading teacher / temur & sezer one of the two important findings in this table is that a) the frequency rates or usage situations of the concepts e-book and e-books decreased by 1/6 in the last five years. it is noteworthy that although technological possibilities develop day by day and offer different applications and systems related to reading, they occupy less space in studies on reading. b) linguistic diversity related concepts such as multilingual, translanguaging, bilingual and languages became more prominent in the last five years. discussion and conclusion in this study, we investigated concepts that collocate frequently with the concept of "reading" in rt journal in the last ten years, the distribution of these collocations by years, which concepts have come to the fore in rt recently, and which concepts have decreased in their usage rate. the corpus-based findings in the research show that "comprehension" is one of the most important components of reading. besides being one of the most frequently used concepts on its own in rt corpus, comprehension is the most likely concept to be seen collocating with reading. in the corpus, it was also observed with different components such as techniques, skills, strategies, activities, scores, attitudes and abilities. concepts such as close, independent, fluency, which often collocate with reading, are comprehensionoriented concepts. it is reasonable to conclude that research on reading during the last decade stress the importance of reading comprehension by identifying the following: a. the factors, the skills and the reading strategies that are necessary for reading comprehension, and b. the instructional methods and strategies that help the students to improve their skills for reading comprehension. although there is a limitation in this study due to the fact that it is based only on corpus data from 2012 thru 2021, our research is based on peer-reviewed research 952 articles published in a journal, the reading teacher, which is one of the world’s well-known journals with a special focus on reading. the findings reveal that the main goal of reading studies is to highlight the elements of reading comprehension and the methods, techniques, skills, strategies, activities and attitudes necessary to achieve the ability of reading comprehension. our findings are consistent with the earlier studies which were done through content analysis (jerrolds, 1992; stahl & fisher, 1992; pearson, 1992; dillon, et al., 1992 and mohr, et al., 2017). from the past to present, reading comprehension has been one of the main subjects of academic studies and it has maintained its importance almost in most of the reading-focused publications. our study also shows that reading and reading comprehension are multidimensional and complex processes and there are different components that need to be investigated further. however, in the aforementioned studies, rt was examined by content analysis and generally through keywords. the main difference of the current study is to make direct word frequencies and collocation analyses on the words used in the article content. while content analysis makes it easier to look at studies in the field of reading from a wider perspective and to understand reading based on it, this corpus-based study has given the chance to present more detailed results. considering the usage rates in rt corpus in the last ten years, two more concepts are also worth mentioning. one of them is writing and the other is instruction. these concepts frequently collocated with reading in rt in the last decade, and their usage rates were quite high compared to other concepts almost every year. for example, independent reading is a quite frequent concept, but it has either a very high (2012) or a very low (2021) usage rate according to the years of use. however, writing and instruction are intensely used concepts every year in the last decade. this finding is very important to argue that reading as an important and lifelong-needed skill cannot be considered independently of writing skill. in addition, when the verbs and adjectives are forming collocations with reading, our study shows that instructional routines and actions that support and improve reading are always necessary. furthermore, we can say that a tendency towards frequently used concepts related to reading sends the educators an important message: the students’ and the teachers’ learning and development are interrelated. another remarkable finding is that the concept of reading difficulties was frequently examined only in 2016 (f= 174.76) and used quite few in other years, compared the use of elements to improve reading process and reading comprehension, such as "comprehension, fluency, recovery, while and after reading". reading problems was the eighth most frequent article topic in the study conducted by dillon, et al., (1992), and the twenty-first in mohr, et al., (2017). it seems that researchers, educators and editors of the journal preferred to focus on comprehension rather than focusing directly on “difficulties or problems” during the last decade. finally, we want to underline the influences of the national contexts and the debates on the researchers’ agendas. there is a likelihood that several national 368 march 2023, volume 15, issue 4, 357-369 events and expert-panel publications in the usa have had some impact on the research topics that the researchers and journal editors have given priority in the last decade. with these reflections in mind, we hope the following conclusions from our study presents an important contribution to the field of reading: 1. the ten most frequently used concepts in rt journal consisted of reader, teacher, reading material and reading-writing related concepts. reading, the main subject of this study, was the second most frequently used concept. 2. regardless of the part of speech, it was seen that the most frequently used concepts collocating with reading are the collocations for reading practices or instructional routines. among these collocations, the most frequently used ones in the last ten years are independent reading and reading comprehension. reading comprehension, reading-writing and reading instruction are the two most common collocations in the publications from 2012 to 2021. 3. when the collocations with reading formed by adjectives are examined, it is seen that the concepts that often describe reading are instructional routines, basic reading skills, and reading medium. it is also observed that the most frequently used concepts in the last ten years among the adjectives collocating with reading are close, independent, oral reading. independent reading was the most frequently used collocation last year, but its usage rate had decreased after 2018. close reading, shared reading, online reading, early reading and informational reading collocations, had different usage rates till 2021, but their usage have increased since then. 4. the verbs forming collocation with reading consist of action, improvement, effective, goaloriented, problem solving, and impact verbs for reading skills. in the last ten years, the strongest verb collocating with reading is “improve”. although the usage situations have changed over the years, the verbs whose usage rate increased in 2021 compared to previous years are improve, enhance, solve, implement. 5. in the rt corpus, children, common core state standards (ccss) reading, e-books, picture, child, e-book and read were used frequently between 2012 and 2016 but less in 2017-2021. concepts with higher usage rates between 2017-2021 compared to 2012-2016 were literacy, multilingual, translanguaging, linguistic, coaching, bilingual, preservice, languages, spanish, practices, and running. this tendency can be related to national reading-related debates, technological developments, and the increased awareness of linguistic diversity in the american schools. acknowledgement thanks to dr. abdullah kaldirim and hasan er for their great help in organizing the article data and reviewers for outstanding suggestions. references anderson, r., hiebert, e., scott, j., & wilkinson, i. (1985). becoming a nation of readers: the report of the commission on reading. national institute of education. bnc. (2023, february 18). british national corpus. http:// www.natcorp.ox.ac.uk/corpus/index.xml brezina, v. (2018). statistics in corpus linguistics a practical guide. cambridge university press. coca. (2023, february 18). corpus of contemporary american english. https://www.englishcorpora.org/coca/ dillon, d. r., o'brien, d. g., hopkins, c. j., baumann, j. f., humphrey, j. w., pickle, j. m., . . . pauler, s. m. (1992). article content and authorship trends in the reading teacher. the reading teacher, 45(5), 362-368. https://www.jstor.org/ stable/20200867 grabe, w. (2009). reading in a second language moving from theory to practice. cambridge university press. ila. (2023, february 18). journal overview. the reading teacher. https://ila.onlinelibrary.wiley.com/ hub/journal/19362714/journal-overview.html jerrolds, b. w. (1992). milestones: a chronological look at the reading teacher. the reading teacher, 45(5), 342-343. https://www.jstor.org/ stable/20200865 kendeou, p., van den broek, p., jane white, m., & lynch, j. s. (2009). predicting reading comprehension in early elementary school: the independent contributions of oral language and decoding skills. journal of educational psychology, 101(4), 765–778. https://doi.org/10.1037/a0015956 kucera, h., & francis, w. (1967). computational analysis of present-day american english. brown university press. langenberg, d. n. (2023, january 15). findings of the national reading panel. reading rockets. findings of the national reading panel: https:// www.readingrockets.org /article/findingsnational-reading-panel leech, g. (1992). corpora and theories of linguistic performance. in j. startvik (ed.), directions in corpus linguistics (pp. 105-122). mouton de gruyter. 369 the agenda of the reading teacher / temur & sezer mcenery, t., & hardie, a. (2011). corpus linguistics: method, theory and practice. cambridge university press. mcenery, t., & wilson, a. (2001). corpus linguistics. edinburgh university press. mohr, k. a., ding, g., strong, a., branum, l., watson, n., priestley, k. l., . . . lundstrom, k. (2017). reading the past to inform the future: 25 years of the reading teacher. the reading teacher, 71(3), 251-264. https://doi.org/10.1002/trtr.1636 naep. (2023, january 15). naep report card: reading. h t t p s : // w w w. n a t i o n s r e p o r t c a r d . g o v / reading/?grade=4 ngacbp. (2013). a governor’s guide to early literacy: getting all students reading by third grade. https://files.eric.ed.gov/fulltext/ed583168.pdf pearson, p. d. (1992). rt remembrance: the second 20 years. the reading teacher, 45(5), 378-385. https://www.jstor.org/stable/20200871 phillips, m. (1983). lexical macrostructure in science text [unpublished doctoral dissertation]. university of birmingham. phillips, m. (1985). aspects of text structure: an investigation of the lexical organisation of text. north-holland. phillips, m. (1989). lexical structure of text. university of birmingham. seidenberg, m. (2013). the science of reading and its educational implications. language learning and development, 9(4), 331-360. https://doi.org /10.1080/15475441.2013.812017 sinclair, j. (1991). corpus, concordance, collocation. oxford university press. stahl, n., & fisher, p. j. (1992). rt remembrance: the first 20 years. the reading teacher, 45(5), 370-377. https://www.jstor.org/stable/20200868 staiger, r. c., kress, r. a., larrick, n., cleland, d. l., stauffer, r. g., kline, l. w., . . . o'brien, d. g. (1992). the reading teacher 1948 to 1991: the editors retrospect. the reading teacher, 45(5). https:// www.jstor.org/stable/20200866 tognini-bonelli, e. (2001). corpus linguistics at work. john benjamins publishing. torgesen, j., houston, d., rissman, l., & kosanovich, m. (2007). teaching all students to read in elementary schools: a guide for principals. center on instruction. https://files.eric.ed.gov/ fulltext/ed521606.pdf vaclav , b., mcenery , t., & wattam, s. (2015). collocations in context a new perspective on collocation networks. international journal of corpus linguistics, 20(2), 139–173. https://doi. org/10.1075/ijcl.20.2.01bre wixson, k., peters, c., weber, e., & roeber, e. (1987). new directions in statewide reading assessment. the reading teacher, 40, 749-754. https://www.jstor. org/stable/20199614 international electronic journal of elementary education, december 2016, 9(2), 485-498. issn:1307-9298 copyright © iejee www.iejee.com generalized use of past tense verbs in children with autism following a sufficient exemplar training procedure sigmund eldevik a * ellie kazemi b greg elsky c a oslo and akershus university college of applied sciences, norway b california state university, usa c behavioral learning network, usa received: september, 2016 / revised: october, 2016/ accepted: november, 2016 abstract the purpose of this study was to validate a procedure for sufficient exemplar training that can potentially lead to generalized responding. based on this procedure, we taught four children with autism, generalized use of regular past tense verbs. we applied a non-concurrent multiple baseline design across the participants. the dependent variable was the number of verbs that needed to be trained for generalization to untrained verbs to occur. the number of verbs that needed to be trained varied considerably across participants as did the number of trials. we concluded that the procedure was validated and that it was easy to implement and it allows for individualization. we discuss the practical implications of our findings for teaching a wide range of skills using the same type of training procedure and data display strategies we provided in this study. keywords: sufficient exemplar training, autism, generalized responding, past tense introduction one of the many challenges for persons with autism is lack of generalization of skills across such dimensions as setting, type of material, wording of instructions, and different communication partners. another challenge is a lack of generalization from explicitly taught exemplars of a response class to untaught members of that class (e.g., labeling untaught exemplars of cars). furthermore, limited generalization within higher order operant classes may also be a challenge (e.g., imitating untaught exemplars of gross motor movements or matching stimuli that have not been explicitly taught). we have found two studies that taught the correct use of the past tense verbs in children with autism and/or intellectual disability. in one study, three adolescents with intellectual disability were taught generative use of regular verbs through imitation and * corresponding author: sigmund eldevik, oslo and akershus university college of applied sciences, department of behavioral science, po box 4, st. olavs plass, n-0130 oslo, norway. e-mail: eldevik@online.no, http://www.iejee.com/ international electronic journal of elementary education vol.9, issue 2, 485-498, december 2016 486 reinforcement procedures (schumaker & sherman, 1970). both past and present tense verbs were taught until the criterion was reached within four classes. one class was –t (as in stopped); a second class was -d (as in climbed), a third class was –ted (as in painted) and a fourth one was – ded (as in graded). following training, the researchers observed generative responding within these classes and that some of the children learned individual verbs faster as the experiment progressed. overgeneralization was also observed in that verbs that had been said correctly previously, in one of the classes, were said incorrectly but in accordance with the more recently trained class. in another study, a total of seven children with intellectual disabilities and autism were taught one or two sets of five regular past tense verbs through mei (greer & yuan, 2008). generalization of correct past tense use to novel regular verbs and overgeneralization to irregular verbs was observed in all of the participants. greer and yuan did not make any distinction between the verbs based on inflection, and children in their study did not seem to have any problems with responding correctly across inflection classes. although these studies successfully established the generalized use of the past tense, since the verbs were taught in sets, it allowed for limited individualization. it may also be that more verbs than needed were trained. several teaching procedures have been identified that promote generalization. one of the most commonly described in the literature, is the teaching of sufficient exemplars (cooper, heron, & heward, 2007; white et al., 1998). although the strategy for teaching sufficient exemplars is straightforward – train enough exemplars for generalization to occur in new, untrained exemplars – the specific procedures used to train sufficient exemplars differ across research studies. currently, there does not seem to be specific step-by-step recommendations for how to do this in practice. additionally, no recommendations have been provided as to what may be appropriate criteria for mastery for each exemplar, nor have recommendations been provided regarding how and if exemplars should be mixed during teaching. training sufficient exemplars is sometimes used interchangeably with training multiple exemplars (sometimes called multiple exemplar instruction; mei; see greer & yuan, 2008). however, some authors point out that there appears to be some differences between the two instructional methods. in mei, typically, several exemplars are trained concurrently and then, subsequent probes for generalization to untrained exemplars are conducted (white et al., 1998). in sufficient exemplar training however, it seems common to train one exemplar at a time to mastery and then conduct a probe on an untrained exemplar (or sometimes several untrained exemplars). there is some evidence to suggest that concurrent training, like in mei, leads to more rapid acquisition and more generalization to untrained stimuli, such as across different letter fonts (wunderlich, vollmer, donaldson, & phillips, 2014). however, concurrent training of many exemplars may sometimes be impractical (depending on the task at hand) and may also lead to teaching more exemplars than is actually needed for generalization to occur, wasting time and resources. the manner in which typically developing children acquire language appears to be more similar to sufficient exemplar training. detailed descriptive analysis of language development suggests that language (including the correct use of the verb tense) is taught through a number of three-term contingencies in interaction with parents (moerk, 1990). children may learn the past tense form of one verb in the kitchen one week, and the past tense form of another verb in the bathroom the next week. idiosyncratic learning of other verbs continues over the course of months or years and at some point the generalized use of past tense verbs is established. indeed, it is common for children to overgeneralize and apply the regular past tense form to irregular verbs (as in saying “runned” instead of ran). a sufficient exemplar training procedure / eldevik, kazemi & elsky 487 this process occurring in the natural environment – although slowly and unsystematic – is reminiscent of sufficient exemplar training. it is therefore conceivable that teaching sufficient exemplars of verbs more systematically may speed up acquisition. it is also conceivable that individuals with special needs, who have not generalized across the past tense form of verbs, may do so if they are taught more systematically. some of the most commonly used curriculum manuals for teaching children with autism suggest teaching a set number of items within a class (e.g. teach imitation of 10-15 gross motor movements, then move onto fine motor; teach matching of 15 objects, then move onto pictures (e.g., lovaas, 2003: sundberg, 2008, leaf & mceachin, 2008; maurice 1996). one problem with this strategy is that you cannot be sure that the child has learned enough exemplars for generalization to novel exemplars to occur. an alternative to teaching sets of exemplars (as in mei) is to teach only one exemplar at a time in succession. in the case of teaching the past tense, this will mean teaching one verb at a time and when this one verb is mastered, a probe of untrained verbs is done to see if generalization occurs. there are some possible benefits to this procedure. specifically, this procedure would allow the therapist to identify exactly when generalization to untaught verbs occurs, and to stop further teaching at that point. this could potentially save time and resources, by reducing the number of exemplars taught (as when you are teaching sets), and allows for a much greater degree of individualization. the next step in teaching would be guided by data for that individual and not simply by the completion of certain sets or a certain numbers of exemplars. variations of this procedure was successfully used to teach children with autism cooperative play (jahr, eldevik & eikeseth, 2000) and to answer wh-questions (jahr, 2001). both of these studies reported generalization across people and time following mastery in training. the aim of the present study was to validate a procedure for sufficient exemplar training where one verb is taught at a time, and where generalization probes are conducted after mastery of each verb. an important part of the procedure is applying behavioral criteria for when to teach the next verb and for determining if generalization has occurred. applying behavioral criteria allows for tailoring. each participant can be taught exactly the number of verbs that are needed for generalization to occur. the pace for introducing new verbs is also individualized in that each new verb needs to be mastered before the next one is introduced. in order to check other types of generalization we included tests for generalization across people and time. method participants and setting three boys and one girl with diagnosed with autism participated in this study. their age was between 6 and 20 years. the level of functioning was significantly below the average as measured by standardized assessments of intelligence and adaptive behavior. their everyday language mainly consisted of 2-5 word phrases. they labeled verbs in daily life and from pictures using the present tense in the simple (e.g., walk) or continuous form (e.g., walking). they did not use the past tense when they talked about what they had just done or what they did yesterday as reported by their parents and staff in semi-structured interviews (i.e., the expressive subdomain on vineland adaptive behavior scales-ii (sparrow, cicchetti, & balla, 2005) or when probed informally. all of the participants received intensive behavioral intervention, either at school or in their home. see table 1 for a more detailed description of the participants. international electronic journal of elementary education vol.9, issue 2, 485-498, december 2016 488 table 1. age of participants at intake. intellectual functioning and adaptive behavior levels are given as standard scores. weekly hours, and years in eibi programs along with their usage of past tense verb prior to the study, are also provided. characteristics chris roy charlie gretchen age 10 6 20 11 full scale iq a 44 na 47 na adaptive behavior composite b 62 74 55 61 communication 65 76 54 na daily living skills 69 75 58 na socialization 57 68 56 na weekly hours of eibi during study 15 20 10 20 years in eibi 8 2 17 8 regular past tense usage none none none some note. eibi = early intensive behavioral intervention; na = not available aroid, g. h. (2003). stanford-binet intelligence scales, fifth edition, technical manual. itasca, il: riverside publishing. bsparrow, s. s., cicchetti, d. v., & balla, d. a. (2005). vineland adaptive behavior scales: second edition. circle pines minnesota: ags publishing. dependent variable we collected trial-by-trial data on the participant’s responding during baseline, training, post-test, generalization across people and at follow-up. the dependent variable was the number of past tense verbs that needed to be taught before generalization to untaught verbs occurred. in addition, we counted the number of trials needed to reach the criterion for mastery of each single verb during the intervention, and to reach the criterion for mastery in generalization. the criterion for mastery when teaching a verb separately was three consecutive correct responses in a massed trial format, and then 5 consecutive correct responses in a random mix with other mastered verbs. the criteria for mastery in generalization, was correct responding on five consecutive untaught verbs. design we applied a non-concurrent multiple baseline design across participants to evaluate the effects of training. participants were assigned to baseline lengths of one day (participant 1; chris), one week (participant 2 and 3; roy and charlie), and four weeks (participant 4; gretchen). in each baseline session we probed the participants’ correct use of regular verbs in the past tense. we did this by going through a list of 20 verbs, probing each verb twice. the probes were conducted by showing the participant an action (e.g., bouncing a ball), and then after 1-2 seconds, asking, “what did i do?” the correct answer was to use the past tense form of the verb (e.g., “bounced” or “bounced the ball” or “you bounced the ball”). one-word answers were accepted as long as the correct verb tense was used and the answer was clearly pronounced. incorrect pronoun use or other grammatical mistakes a sufficient exemplar training procedure / eldevik, kazemi & elsky 489 (e.g., if the participant said “i bounced the ball!” instead of “you bounced the ball!”) were also accepted provided the participant used the correct past tense form of the verb. if the therapist was in doubt, the response was scored as incorrect. no feedback was given to the participants during this baseline testing. following the intervention, a probe session – similar to the probe sessions in baseline was conducted with two other therapists, one therapist that was familiar with the participant but not involved in the study and another therapist that did not normally work with participant. this was done to test for generalization across different persons. during the intervention phase, each new verb was probed, to see if the participant responded with the correct past tense without training. if the participant responded correctly only after training of a particular verb was introduced, and this was replicated across several verbs and participants, it would suggest that the training was the reason for the participants correct past tense usage. furthermore, it would suggest that training x number of verbs in this way, eventually can lead to generalization to new untrained verbs. this procedure also allows the ability to pinpoint the precise number of verbs needed to be trained before generalization to untrained verbs occurs. regular verb selection the regular verbs were selected from a list of all regular verbs. actions that could easily be demonstrated in a discrete-trial-teaching format were preferred (e.g., clapping, jumping). in addition, only verbs that the participant could label in the present tense and pronounce clearly were included. if the participant used the past tense correctly for any given verb on a consistent basis (i.e., correct on every probe in baseline), that verb was not included in the study. the list was therefore somewhat individualized. however, 16 of the 20 verbs included, were common for all participants. see table 2 for a list of the verbs included and how they were presented (the order of introduction varied somewhat across the participants). table 2. list of regular verbs included in the study and how they were presented. the presentation was followed by a 1-2 second pause and the question ‘what did i do?’ # verb how it was presented 1 clap clapping hands 3-4 times 2 jump jumping up and down 3-4 times 3 knock knocking 3-4 times on table with knuckles 4 wave waving 3-4 times with hand 5 bounce bouncing a ball 3-4 times on floor 6 kick kicking the wall 7 pat patting own arm 8 spray spraying the table with soap 9 smile smiling at participant international electronic journal of elementary education vol.9, issue 2, 485-498, december 2016 490 table 2 (cont.). list of regular verbs included in the study and how they were presented. the presentation was followed by a 1-2 second pause and the question ‘what did i do?’ # verb how it was presented 10 rip ripping a piece of paper 11 wipe wiping own mouth with paper 12 roll rolling the ball 2-3 meters on floor 13 cry pretending to cry 14 blink blinking with both eyes 15 fold folding a piece of paper in two 16 close closing a container 17 walk walking across the room 18 open opening a container procedure training sessions were embedded into the participants’ regular teaching sessions and conducted for approximately one hour per day. the participants’ therapists conducted all training and testing. the therapists had between 6 months and 7 years of experience with discrete-trial-teaching and the fidelity of their teaching was monitored regularly. the monitoring included several components of discrete trial teaching such as presenting instructions and teaching materials, prompting, shaping, delivering consequences and providing session structure. prior to this study, all of the therapists involved demonstrated at least 90% adherence to a proprietary checklist monitoring discrete trial teaching skills. sufficient exemplar training strategy the intervention phases started with probing the first verb on the list in the same way it was probed in baseline. the therapist showed the action and after 1-2 seconds asked, “what did i do?” after a maximum of two consecutive incorrect responses (no response was also scored as incorrect) a verbal modelling prompt was provided and the question repeated (“what did i do? clapped! what did i do?”). after a prompted trial, another trial was presented with the same verb. hence, the participant was always given the opportunity to respond correctly without a prompt after a prompted trial. we always prompted one-word answers (e.g., “clapped” rather than “you clapped”). the criterion for mastery when targeting a verb separately was three consecutive correct responses. when this was achieved, the verb was mixed randomly with 1-3 other mastered verbs from the list (except for the very first verb when no other mastered verbs were available). the criterion for mastery in the random mix phase was five consecutive correct responses, where the new verb had to be asked for at least once. when this criterion was met, the next new verb on the list was probed. if the participant responded incorrectly, the verb was taught following the same procedure as just described. if s/he responded correctly, the next verb on the list was probed or the new verb was done in a mix with other mastered verbs, before the next verb was probed. this procedure was followed until the participant responded correctly to five consecutive novel verbs on the first probe. at the end of each training session, the scores were displayed in a spreadsheet, where the a sufficient exemplar training procedure / eldevik, kazemi & elsky 491 therapist or supervisor could visually inspect the progress in the teaching and make adjustments if needed. the steps in the procedure are showed in table 3. table 3. the procedure for sufficient exemplar training followed in the current study. the first five steps are show with examples of score and the criterion for when to move to the next step. step # verb probe (sd) correct response if correct if incorrect 1 1 clap shows clapping, pause: "what did i do?" "clapped" probe next verb on list teach until 3 consecutive corrects in massed trial 2 2 jump shows jumping, short pause: "what did i do?" "jumped" probe next verb on list teach until 3 consecutive corrects in massed trial 3 1 and 2 random mix of 1 clap and 2 jump "jumped"/ "clapped" teach until 5 consecutive corrects in random order 4 3 knock shows knocking, short pause: "what did i do?" "knocked" probe next verb on list teach until 3 consecutive corrects in massed trial 5 1, 2 and 3 random mix of 1 clap, 2 jump and 3 knock "jumped"/ "clapped"/ "knocked" teach until 5 consecutive corrects in random order 6 4 wave shows waving, short pause: "what did i do?" "waved" probe next verb on list teach until 3 consecutive corrects in massed trial continue this procedure until 5 consecutive new verbs are correct when probed three out of the four participants had a token board and selected their backup reinforcer from pictures of preferred items kept in a folder (e.g., internet time, computer games, or puzzles). during training, all correct responses were followed with praise and a token. typically, the participants completed two token boards of 10 tokens before they got access to the backup reinforcer. for one of the participants (gretchen), only social praise followed correct responding. prompted responses were followed by praise only for all four participants. incorrect responses were not given any programmed consequence by the therapist, except for gretchen where an incorrect response was followed by an informational “no” or “try again”. international electronic journal of elementary education vol.9, issue 2, 485-498, december 2016 492 procedural integrity for the purposes of the present study, the therapists underwent additional training on the sufficient exemplar teaching procedure and data collection procedures. scoring was done using a datasheet that allowed each trial to be recorded as either correct, incorrect or prompted. we defined procedural integrity as following the intervention procedure on every teaching trial as outlined above (e.g., probing a new verb when the previous verb was mastered in a mix after training or on the first probe, prompting after a maximum of two incorrect responses). integrity was calculated by inspecting the trial-by-trial display of the teaching for each of the participants. we calculated integrity by counting the number of breaches in protocol and dividing it by the number of opportunities to breach. the average procedural integrity across all participants was 96% (range 93%-100%). inter-observer agreement in about 40% of the probes and training sessions, data were collected by both the therapist and an experimenter (first or third author) independent of each other. agreement was calculated by dividing the number of disagreements by the number agreements plus disagreements and multiplying by 100. agreement was high with an average of 96% (range 94-100%). results all four participants established the generalized use of past tense regular verbs following intervention. see figure 1. participant 1, chris, demonstrated no correct use of the past tense in the 18 verbs that were probed in baseline. chris started the intervention on the same day as the baseline probes were done. after being taught a total of seven verbs during the intervention phase, he started responding correctly when new verbs were probed and met the criterion for generalized responding (i.e., 5 consecutive correct responses on new verbs), with correct use of the past tense on verb numbers 8 through 13. this criterion was met after a total of 157 trials, and two days of teaching. on the posttest with the same 18 verbs that were tested in baseline, conducted immediately following the intervention phase, he got all correct. a test for generalization across people was conducted on the next day, including 11 of the 18 verbs in the baseline test. he used the past tense correctly on 10 out of 11 correct with a person not involved in the intervention. after about three months, a follow up test was done, that included 16 of the 18 verbs that were tested in baseline. he responded correctly on 10 of the 16 verbs probed. see figure 2. participant 2, roy demonstrated no correct use of the past tense in baseline sessions. he had two baseline sessions two weeks apart, each of them with the same 16 verbs. like chris, roy also needed to be taught 7 verbs before generalized responding to new untrained verbs was observed, and he reached the criterion of responding correctly to 5 consecutive novel verbs 8 through 13. this took 136 trials and 4 days of teaching. roy responded correctly to all 16 verbs in the post-test. a generalization test could not be performed in roy’s case, but a follow-up test was conducted after 3 months. no correct use of the past tense was observed at follow-up. see figure 3. participant 3, charlie showed no correct use of the past tense in the probes conducted prior to the intervention. like roy, charlie had two baseline sessions two weeks apart, but he had 18 verbs probed in each of the session. charlie needed training with 5 verbs before he started to generalize to novel verbs, and he met the criterion of responding correctly to verbs 6 through 13, and this took 59 trials, and 3 days of training. charlie also responded correctly to all 18 verbs in the post-test, in the test for generalization across people, and got 17 out of 18 correct in the follow-up test conducted 3 months later. see figure 4. a sufficient exemplar training procedure / eldevik, kazemi & elsky 493 participant 4, gretchen, demonstrated some correct use of the past tense in baseline. we did baseline tests once a week for 4 weeks. on the first test she got 7 out of 14 verbs correct, on the second 12 out of 17, on the third 7 out of 15 and on the fourth 10 out of 17. three verbs were consistently correct across all baseline measures and these were excluded from the study. gretchen only needed training on three verbs before she started to generalize and met criterion for generalized responding with correct responses to verbs 4 through 8. this required 39 trials and one day of teaching. on the post-test, the test for generalization across people and time she got 16 out of 17 correct. figure 1. the number of verbs in each condition are plotted against the top x-axis, and the number of weeks in baseline are plotted against the bottom x-axis. the criterion for mastery in training was 3 international electronic journal of elementary education vol.9, issue 2, 485-498, december 2016 494 corrects in a row in massed trials, followed by 5 corrects in a random mix of mastered verbs. the intervention phase was stopped when the participant responded correctly to 5 untrained verbs consecutively. the boxes with “probes” show the results of each verb probed during baseline, post-test, generalization across people, and at follow-up. a solid black circle indicates incorrect responding on the verb probed, whereas a white circle indicates correct responding. the number of trials needed to meet the criterion for mastery of each new verb during training are plotted against the y-axis, and number of verbs trained against the x-axis. when criterion for generalized responding across new verbs was met, this is highlighted in the gray circles (correct usage of at least 5 untrained verbs on first trial). figure2. this figure displays the training data for participant 1 chris. the number of training trials across verbs are plotted against the y-axis, and the verbs included in training are shown on the x-axis. a sufficient exemplar training procedure / eldevik, kazemi & elsky 495 the rectangles show the participants response coded as correct (grey), incorrect (black), or prompted (pattern). three corrects in a row means that criterion was met for mastery in massed trials and five correct responses on untrained verbs means that the criterion is met for generalization. figure 3. this figure displays the training data for participant 2 roy. the number of training trials across verbs are plotted against the y-axis, and the verbs included in training are shown on the x-axis. the rectangles show the participants response coded as correct (grey), incorrect (black), or prompted (pattern). three corrects in a row means that criterion was met for mastery in massed trials and five correct responses on untrained verbs means that the criterion is met for generalization. international electronic journal of elementary education vol.9, issue 2, 485-498, december 2016 496 figure 4. this figure displays the training data for participant 3 charlie. the number of training trials across verbs are plotted against the y-axis, and the verbs included in training are shown on the x-axis. the rectangles show the participants response coded as correct (grey), incorrect (black), or prompted (pattern). three corrects in a row means that criterion was met for mastery in massed trials and five correct responses on untrained verbs means that the criterion is met for generalization. figure 5. this figure displays the training data for participant 4 gretchen. the number of training trials across verbs are plotted against the y-axis, and the verbs included in training are shown on the x-axis. the rectangles show the participants response coded as correct (grey), incorrect (black), or a sufficient exemplar training procedure / eldevik, kazemi & elsky 497 prompted (pattern). three corrects in a row means that criterion was met for mastery in massed trials and five correct responses on untrained verbs means that the criterion is met for generalization. discussion we successfully taught four participants with autism to use regular verbs in the past tense following a sufficient exemplar training strategy. the strategy allowed the therapist to monitor performance on every new verb as it was introduced and to see exactly when transfer to novel untrained exemplars occurred. our results suggest that sufficient exemplar training may have some advantages over training in sets of stimuli as in mei or merely teaching a certain number of verbs which is sometimes suggested in intervention manuals. the protocol used in the present study allowed for a greater degree of individualization in the number of exemplars taught and the stepwise progression in training allowed for individual pacing since mastery of one step was required before the participant was allowed to move to the next step. this can save precious teaching time and avoids unneeded repetition. the display made it easy to conduct procedural fidelity checks. for instance, it was possible to see if prompts were given on several consecutive trials and if the mastered items were mixed with the target verb. anecdotal evidence from the therapists was that the display made it easier for them to understand the logic of the sufficient exemplar training strategy they employed. the specific strategy we employed in this experiment may hold some promise for promoting generalization both within a class and across settings. it is conceivable that the same type of strategy may be used to teach a host of other skills where transfer within a particular class would be logical. this could be other grammatical behavior such as singular and plural forms but also higher order classes such as imitation and matching could be taught following a similar strategy. indeed, the procedure shares the same logic as that employed by jahr, eldevik & eikeseth (2000) to teach children with autism generalized cooperative play, and jahr (2001) who taught children with autism to answer novel wh-questions. it is likely that the teaching of many exemplars (whether it is done as a serial or concurrent strategy) simulates normal language development and that the key element is learning sufficient exemplars. based on the literature, it is not clear how to conceptualize the fact that responding generalized to novel verbs and if it is a form of response generalization. stewart, mcelwee, and ming (2013) argued that this term is used inconsistently and that it does not provide a satisfactory account for how generative performances come about. instead, these authors proposed that generative language can be best explained as derived relational responding, and that the teaching of multiple exemplars is the underlying process responsible for this. other authors have described the generalized use of correct verb tenses as an acquired autoclitic operant (moore, 2008). it seems that further research and theoretical conceptualizations are needed to better understand the phenomenon of generative language. future studies should investigate whether skills acquired generalize across time and investigate whether generalization to other examples of the same verb will occur (e.g., generalization from opened a box to opened a door). in addition, the particular details of the protocol, such as the criterion for introducing the next probe, need to be validated across a variety of skills and participants. the trial-by-trial data collection and depiction should also be tried and compared with mei. international electronic journal of elementary education vol.9, issue 2, 485-498, december 2016 498 • • • acknowledgements we want to thank the gidske and peter jacob sørensens researchfund and the renee and bredo grimsgaards charity for financial support. we would also like to thank the lovaas institute for early intervention in culver city, los angeles and roy tønnesen at the department of autism, pedagogical psychological centre, bergen, norway for contributing to this study. references cooper, j. o., heron, t. e., & heward, w. l. (2007). applied behavior analysis (2nd ed.). upper saddle river, nj: pearson merrill prentice hall. greer, r. d., & yuan, l. (2008). how kids learn to say the darnedest things: the effect of multiple exemplar instruction on the emergence of novel verb usage. the analysis of verbal behavior, 24, 103-121. jahr, e. (2001). teaching children with autism to answer novel wh-questions by utilizing a multiple exemplar strategy. research in developmental disabilities, 22(5), 407-423. jahr, e., eldevik, s., & eikeseth, s. (2000). teaching children with autism to initiate and sustain cooperative play. research in developmental disabilities, 21(2), 151-169. leaf, r., & mceachin, j. (1999). a work in progress. new york: drl books. lovaas, o. i. (2003). teaching individuals with developmental delays: basic intervention techniques. austin, tx: pro-ed. maurice, c., green, g., & luce, s. c. (eds.). (1996). behavioral intervention for young children with autism: a manual for parents and professionals. austin, tx: pro-ed, inc. moerk, e. l. (1990). three-term contingency patterns in mother-child verbal interactions during first-language acquisition. journal of the experimental analysis of behavior, 54(3), 293-305. doi: 10.1901/jeab.1990.54-293 moore, j. (2008). conceptual foundations of radical behaviorism. ny: sloan: cornwall-on-hudson. roid, g. h. (2003). stanford-binet intelligence scales, fifth edition, technical manual. itasca, il: riverside publishing. schumaker, j., & sherman, j. (1970). training generative verb usage by imitation and reinforcement procedures. journal of applied behavior analysis, 5, 275-287. sparrow, s. s., cicchetti, d. v., & balla, d. a. (2005). vineland adaptive behavior scales: second edition. circle pines minnesota: ags publishing. stewart, i., mcelwee, j., & ming, s. (2013). language generativity, response generalization, and derived relational responding. the analysis of verbal behavior, 29, 137-155. sundberg, m. l. (2008). verbal behavior milestones assessment and placement program: the vbmapp. concord, ca: avb press. white, r. o., liberty, k. a., haring, n. g., billingsley, f. f., boer, m., burrage, a., sessoms, i. (1998). review and analysis of strategies for generalization. in n. g. haring (ed.), generalization for students with severe handicaps: strategies and solutions (pp. 15-45). seattle and london: university of washington press. wunderlich, k. l., vollmer, t. r., donaldson, j. m., & phillips, c. l. (2014). effects of serial and concurrent training on acquisition and generalization. journal of applied behavior analysis, 47(4), 1-15. doi:10.1002/jaba.154 © 2022 published by kura education & publishing. this is an open access article under the cc byncnd license. (https://creativecommons. org/licenses/by/4.0/) 199 copyright © www.iejee.com issn: 1307-9298 international electronic journal of elementary education january 2023, volume 15, issue 3, 199-208 autism and behavior analysis: from dissonance to dialogue karola dillenburgera,*, mickey keenanb abstract introduction questions about human behavior and diversity have captured the attention of scientists from a wide range of philosophical stances for centuries. while behavior analysts would argue that natural science is an enlightening perspective when applied to the understanding of the human condition, at times arguments within certain campaigns would prefer applied behaviour analysis (aba) to be banned or abolished. in this paper we discuss some of the roots of this stance and suggest a way forward based on what are basically complementary views on the goals enshrined in rights-based practice. exclusion in all its many guises is an ugly thing and, in this paper, we discuss consequences that arise for autistic individuals who have particularly high support needs (american psychiatric association, 2013; lord et al., 2022), if applied behavior analysis (aba)-based services were abolished or excluded (graber & graber, 2023). we argue that countering exclusion and ensuring real inclusion requires dialogue and genuine empathy. autistic self-advocates commonly report the consequences of discrimination, ableism, a normalizing agenda, being viewed from a medical rather than a social model, and misunderstandings concerning support needs (tarvainen, 2019). until the 1980s, autism was not officially recognised (volkmar et al., 1988) and profoundly autistic people (lord et al., 2022) were considered unreachable or unteachable and commonly confined to a lifetime in institutional care (gómez-durán et al., 2014). those who presented with challenging behavior were exposed to electroconvulsive therapy (ect) and/or physically and pharmacologically restraint (nielson et al., 2021; schnitzer et al., 2020). while this situation seems not to have changed in some countries (buivydaite et al., 2017; lipinski et al., 2022), in the 1960s, behavior analysts in the usa were the first professionals who were able to show that profoundly autistic individuals were not “unteachable,” but that they could be taught and keywords: natural science, behavior analysis, human diversity, autism diversity, discrimination received : 19 november 2022 revised : 20 january 2023 accepted : 8 march 2023 doi : 10.26822/iejee.2023.292 a,* corresponding author: karola dillenburger, queen’s university belfast, n. ireland. e-mail: k.dillenburger@qub.ac.uk orcid: https://orcid.org/0000-0002-3410-5949 b mickey keenan, ulster university, n. ireland. e-mail: mickeykeenan@me.com orcid: https://orcid.org/0000-0002-5579-9169 200 january 2023, volume 15, issue 3, 199-208 that they could learn (ferster & demyer, 1962). they applied the principles of behavior formulated by the science of behavior analysis to design environments that facilitated skills development for persons who had been exposed to long-term care in designated institutions. since then, the field of applied behavior analysis (aba; baer et al., 1968, 1987) has grown, particularly with regards to supporting autistic individuals (surgeon general, 1999). so much so, that today aba is viewed as providing the basis for “gold standard” interventions used to support individuals on the autism spectrum (koegel, 2011; lovaas, 1987; smith & eikeseth, 2011). with such a high profile, it is not surprising that for some, the abbreviation aba practically became a synonym for a method of autism intervention (chiesa, 2006; dillenburger & keenan, 2009). in the meantime, disability advocates achieved some success in their fight against discrimination, health disparities, and ableism (borowsky et al., 2021; narioredmond et al., 2019; tarvainen, 2019; thorne & barr, 2022), including extensive ratification of the united nations convention for the rights for persons with disabilities (uncrpd, 2006). as for autistic people, the fight for awareness, acceptance, and equal rights continues to be shaped by autistic self-advocates, mainly through the newly formed neurodiversity movement (leadbitter et al., 2021a, 2021b; silberman, 2015) as well as families of children with profound autism (unumb, 2013). but somewhere along the way, something has gone wrong. the special and supportive relationship between behavior analysts and profoundly autistic individuals and their caregivers that was built on deeply rooted care and trust in science appears to have been highjacked by an anti-aba movement that views the discipline of aba as abusive, causing post-traumatic stress, being ableist, and promoting a normalizing agenda (kupferstein, 2018; milton, 2012; sandoval-norton & shkedy, 2019; examples of shockingly bad practice by poorly trained professionals can be found on youtube). the focus of this paper is to discuss how those who have dedicated their lives and their science to helping people (i.e., behaviour analysts; cf., apba, 2017) are being singled out and accused as protagonists of all of society’s ills with regards to maltreatment of individuals with autism. while in his early writings milton (2012) could be viewed as one of the leading denigrators of aba, more recently he has sounded much more conciliatory and even seemed to hint at the solution when he discussed the double empathy problem (milton, 2018): …when people with very different experiences of the world interact with one another, they will struggle to empathise with each other. this is likely to be exacerbated through differences in language use and comprehension. (theory of double empathy section, para 1) in order to balance a debate that generally is dominated by the views of autistic self-advocates, we offer an insider view of the perspective of behavior analysts who aim to help profoundly autistic people. what follows is an overview of our personal story as behavior analysts and the struggles to address the imbalance in discussions that could prevent profoundly autistic individuals receiving the kinds of supports that our science can offer. recently, we learned that the 3-year-old child we met in 1997, and whose parents we introduced to the discipline of aba all those years ago, passed his driving test and he got a job in the open employment market. to be honest, that touched us deeply, more in fact than when he was awarded his master’s degree in american history a few years earlier. before we met him, he had been assessed by all the usual health and allied health professionals, none of whom (according to reports by his parents) were able to help him acquire even the most basic skills. according to their clinical assessment reports, he was never going to learn to speak in full sentences and was destined for an institution. together with his mother and other parents, we established the charity “parents education as autism therapists” (peat; the title was chosen by parents). during the subsequent 20+ years, peat helped hundreds of families build important life skills for their children, we co-authored the first book about aba and autism in europe (keenan, kerr, & dillenburger, 2000) that has been translated into german and japanese (keenan, kerr, dillenburger, et al., 2000), we co-produced a multilingual multimedia online aba training platform that has been translated into ten languages (german, spanish, dutch, italian, swedish, icelandic, norwegian, czech, portuguese, canadian french, with greek and japanese in progress; simple steps, 2013), and we collaborated in a range of large scale research projects (dillenburger et al., 2010, 2015; keenan et al., 2010; 2022) these are remarkable achievements inspired by a small parent-led charity that was established and provided services for over 20 years without any support from government. what is even more remarkable is that this charity was set up in northern ireland, in a context where their community has been torn apart by violent conflict over many years (dillenburger, 1991, 2007; fargas-malet & dillenburger, 2016), and their story continues to be one of struggle against prejudice, discrimination, and repression, as oftentimes the language used to caricature aba (as ableist, abusive, prejudicial, discriminatory, and pushing a normalizing agenda), is itself ableist, abusive, prejudicial, and discriminatory (e.g., baron-cohen, 2014; kupferstein, 2018; sandoval-norton & shkedy, 2019). 201 autism and behavior analysis: from dissonance to dialogue / dillenburger & keenan here is the problem. as behavior analysts, we think differently from most other people. as scientists, we avoid mentalistic pseudo-explanations for behaviour (apba, 2016; chiesa, 1994; green, 2016; skinner, 1953b; thyer, 2009; thyer & myers, 1999; thyer & pignotti, 2010). instead, our raison d’etre is the discovery of fundamental principles of behavior. we investigate the intimate relationship between environmental contingencies and behavior. the language we use to describe these relationships is full of what sounds like “jargon” to those who are not familiar with the science. a point to make here is that in any other science the terms scientists use may appear strange, cold, alien, even overly technical (keenan & dillenburger, 2000). this is because the terms scientists use are short-hand for discussions with other scientists, and therefore they have to be defined very carefully. as milton (2018) realised, there are “differences in language use and comprehension” (theory of double empathy section, para 1) that can hinder communication between people with different experiences, as would be the case between scientists and non-scientists. in fact, while the everyday language we inherit from our parents has its role in everyday life, it interferes with the precision required by science (skinner, 1953b) and thus does a poor job for scientists (chiesa, 1994; moore, 2013a). take for example, the term “behavior.” in everyday use of the term, it refers to "the way in which one acts, especially towards others,” oftentimes meaning “bad behavior” (e.g., "did the children behave?;" cambridge university press, 2020). in contrast, when behavior analysts talk about “behavior,” we define the term behavior as the “interaction of the organism with their environment,” both, historically as well as currently (cooper et al., 2007; phelps, 2007). we talk about lifelong “learning histories” as well as “cultural meta-contingencies” that impact on present behavior (glenn, 1988, 2004). this holistic view of behavior includes private behavior (what happens inside the skin, e.g., thinking and feeling; skinner, 1977; keenan & dillenburger, 2022 mentalism chapter) as well as public behavior (e.g., moving and speaking; skinner, 1953a, 1957). the image of a stream or river is helpful here. we cannot talk about a river without referencing both the water and the banks through which it flows. likewise, behavior analysts consider that the skin does not separate us from the world, but it connects us to it. this is a completely different way of viewing the world in which we live, with major implications for understanding the process of living where a typical lifespan can be over 2,208,988,800 s. called the behavioral stream, the process of living is viewed as a stream of experiences accumulated throughout one’s lifetime (keenan & dillenburger, 2022, behavioural stream and private events chapter). there is no formal “diagnosis” for our way of being, nor should there be. as behaviour analysts we are just different, with diverse views within our community. the science we employ to study behavior is attracting more people from around the world (see abai, 2022; apba, 2016). yet, we are being told by some (e.g., the neurodiversity movement) that we are wrong in what we are doing, even alleging that we are causing harm and abusing vulnerable children on the autism spectrum (sandoval-norton & shkedy, 2019). they demand that everyone (and that includes behavior analysts) conform to their normalizing agenda and their way of doing things (milton, 2012). as mentioned previously, when behavior analysts study the behavioral stream, the goal is to increase awareness of how behavior is affected by environmental contingencies. this is a formidable task and the methods that have been developed have enriched lives in a wide range of areas (see cambridge centre for behavior studies, 2022; heward et al., 2022). sharing the findings from behavior analytic research is not without its problems. when autistic children experience severe challenges, commonly parents seek guidance on how to design experiences for their children in order to facilitate the emergence of confidence and independence. in other words, parents need to learn how to implement “interventions” with precision and fidelity. behavior analysts have developed many strategies for designing bespoke experiences that straddle education and health issues. denying either the existence of the principles of behavior or their relevance to education and health is simply not an honest way to proceed. in contrast to north america, where there are laws to ensure aba-based interventions are available to those who need them (unumb, 2013), in europe the misinformation about aba disseminated by people not trained in the science has resulted in aba-based services generally not being endorsed by governments (keenan et al., 2022). in fact, some parents report that they have been threatened with the removal of all autism services if they continued with their aba-based home programs (dillenburger et al., 2015). intriguingly, these threats occur even though their children are making progress at home and are learning skills that they did not learn at school (larsson, 2021). keenan (2016) pointed out that this kind of anti-aba perspective implies that parents who seek to improve their awareness of how best to educate their children should apologise to their children. maybe they should say things like “we didn't know that by educating you, by preparing you for the future, we inadvertently did not ‘accept you as you are'?" (keenan, 2016, p. 10). or maybe they are supposed to say to their children: please forgive us for designing experiences for you to have fun, for toilet-training you, for helping you make friends, for supporting you to manage your self-injury, for teaching you to communicate, for giving you the 202 january 2023, volume 15, issue 3, 199-208 skills to make choices and convey your individuality, and for expressing our love for you in numerous other practical ways. we did not know that this could be interpreted negatively, as trying to change who you are or as trying to make you ‘normal’ (keenan, 2016, p .10). those who perpetuate myths about behavior analysis are missing the point completely if they consider these parental apologies to be necessary. on one hand they dismiss the whole science because they object to the idea of “deliberately doing something to change a person” (i.e., education). yet in the next breath, they propose to engage socially with a person in applied settings in the hope that the person be supported or empowered (i.e., changed). the additional layer of awareness provided by the science of behavior analysis in relation to the effects of environmental changes is considered an anathema. what is truly very sad about this situation is that it impacts directly on the benefits to be derived from awareness of how environmental contingencies contribute to the outcomes of education. the anti-aba stance generally is at odds with the science of behaviour analysis for a number of other reasons. behavior analysis embraces the full spectrum of neurodiversity and there is no better evidence than the way it addresses the differing educational needs of each person in applied settings (kazdin, 2010). to ensure that the focus is on the individual, we use single-system research designs to monitor changes in the behavioral stream (keenan & dillenburger, 2011) instead of using comparative group designs which commonly leave one group without support that may be desirable (e.g., nice, 2013). the result of the spread of misinformation is that a whole science has been branded as controversial (parker, 2015) when the real controversy should revolve around the deliberate dissemination of misinformation. the misrepresentation is so pervasive in europe that on one hand it is said that it would be wrong to invest in only one “thing” like aba, while on the other hand it is argued that it is acceptable to invest in one “thing” called an “eclectic approach,” even though there is evidence that aba-based interventions are more effective than eclectic interventions (howard et al., 2005, 2014). unfortunately, no-one addresses the ethical questions that arise from the role played by the dunning-kruger effect (benzel, 2022; hofer et al., 2022) when misinformation and the associated caricatures of aba form the basis of government strategies and policies (dillenburger et al., 2014). vyse (2022) is right when he says: it is one thing to form a social movement in an effort to gain greater acceptance of and better supports for a group of people in need. it is quite another to do so at the expense of another group who is also in great need. helping people on the autism spectrum should not be a zero-sum game with gains at one end of the spectrum requiring losses at another. unfortunately, the autism self-advocacy movement’s attacks on aba [applied behavior analysis] create just such a dilemma (the death of truth and freedom section, para 3). to ward off the onslaught of the anti-aba perspective (note: this language is often used by parents on the receiving end of misinformation), some have suggested that it might be best to change the name of aba. this suggestion is truly symptomatic of the stresses to which parents are exposed. what other science has to contend with a name change as a strategy for managing the aversive consequences produced by discrimination and prejudice. one of the authors (mk) explains: as a child who grew up in n. ireland, i am familiar with many facets of prejudice. raised by a protestant mother and having a catholic father, i experienced at first hand the damaging effects of misguided conclusions conceived in the shadow of myths. when visiting my grandparents, it was at times a surreal experience to cross from one perspective tainted by prejudice to the other perspective similarly tainted by a different kind of prejudice. i was exposed to exclusion and discrimination based on prejudice. in many walks of life during my childhood, irish people were not welcome, as expressed in the sign in the window “no irish”. never did i imagine, though, that in my professional life those scenes from my childhood would be replayed. i should have known better. behavior analysts are not welcome. the sign in the window should read “no aba’ or “no behavior analysts.” in shakespeare’s play romeo and juliet, juliet described the conflict between the rivalling families with the poetic words: “a rose by any other name would smell as sweet.” the families of my own parents, and the families described by shakespeare, were not able to break free from the experiences that shaped their beliefs in order to accommodate another with different experiences and consequently a different perspective. yet, many parents of children with autism have been able to break free from the misinformation and myths about aba and as a consequence their children have benefitted greatly (e.g., aba4all, 2022). unfortunately, we have seen parents pushed to the point of exhaustion in defence of aba in courts and tribunals fighting for their children’s rights to have access to aba-based interventions (blakemore, 2021; byrne & byrne, 2000). where else would this be allowed to happen? where are the investigative journalists who want a human interest story, one about how health and education authorities peddle misinformation about a science in which they are not trained, while parents do what they can to defend a science which they evidently understand better than those professionals (dillenburger et al., 2012)? the ethical drama that is played out in these situations simply is staggering and at the same time diabolical. 203 autism and behavior analysis: from dissonance to dialogue / dillenburger & keenan it is almost as if the parents are put on trial for being witches, inhuman creatures who are hell bent on harming their children. the truth is that these parents are heroes, mustering all of their love to defend the opportunity for their children to acquire skills that others believed they were unable to acquire. so much for equality of opportunity and disability rights. so much for the rights of their children to be regarded as people deserving of the opportunity to be guided by a science dedicated to bringing out the best in people through its accumulated understanding of the principles of behavior change. clearly, those who are against aba are not the only bastions of love for children. nevertheless, aba professionals who guide parents when educating their children continue to be maligned for doing so. it is worth stating that again in other words. parents who have used the discipline of aba to successfully empower their children and improve the quality of their children’s lives have been criticised for doing so. they have been criticised because some people proposed that the evil world of aba could never produce something positive. indeed, they would consider these parents to have been duped into adopting a perspective that emphasises the importance of normalizing children, and not accepting their uniqueness. one of the authors (mk) elaborates: when i hear that kind of misguided anti-aba rhetoric, i find myself straddling the mindsets of both sets of my grandparents (one set from a protestant background and one from a catholic background). it was never possible for them to accept that something positive could emanate from the other camp. of course, this was not always the case for them, but when it came to important matters that divided the community, they each held firm to how they had been taught to view their nemesis. my time growing up in a divided community has brought many stresses to me personally. but it has also educated me in showing how twisted logic is used to defend either position when it is challenged. there was a time in my childhood, for example, when one could have been challenged in the streets along these lines: “are you a catholic or a protestant?” “i am an atheist” “are you a catholic atheist or a protestant atheist” there is no normalizing agenda at the heart of aba. apart from this correction, there is another misguided criticism that needs to be rebuffed. it is said that using aba is tantamount to using a one-size-fits-all approach that ignores individual needs. this one-size-fits-all criticism is entirely misleading because there is indeed one thing that is used by the scientist/practitioner to guide the development of bespoke services, and it is called the scientific method. but this is entirely different from the kind of plug-and-play conceptualisation of one-size-fits-all conjured up by authorities who refuse to invest in aba. surely it cannot be wrong to employ professionals who use the scientific method in the 21st century for designing bespoke services. nurturing an individual using the scientific method so they can feel empowered by the acquisition of skills is an enlightened act of human kindness, not the act of a despot intent on removing their uniqueness (keenan & dillenburger, 2020). to conclude, here is an example that illustrates what can be achieved with good quality aba-based interventions. following it is a contrived example of the sorts of criticism that have been levelled by others in the face of something that does not sit well with preconceived ideas about the nature of aba. this case study was provided by dr. nichola booth (personal communication): a young boy, 8 years old, had a confirmed diagnosis of autism and was non-verbal. it was reported that he had excessive amounts of saliva that was impacting on his day-to-day social and educational experiences. at school he was being isolated from his peer group as he was rubbing the saliva over other children, teachers, classroom assistants and equipment. the same behaviors were happening at home to the point where his parents would not have visitors because their furniture was covered in dried saliva. his clothing was destroyed from being caked in dried saliva. after medication failed, the parents went back to their gp. it was at this stage that they were told the next step would be a referral to a surgeon for an invasive operation where a piece of his mouth would be snipped to reduce or eliminate the extreme levels of saliva. due to waiting times for this appointment, the parents accepted this and went home. while waiting for the hospital appointment the parents decided to access the services of a behavior analyst for support, but nothing related to the saliva issue. the aba professional came to the family home and carried out parental interviews and behavioral observations to determine what supports would be required. while carrying out the observation he noticed the behavior associated with excessive saliva production, especially the wiping of the saliva on all items and people within the room. this was raised as a potential issue to be addressed and it was then that the parents explained what had happened to date. the behavior analyst suggested that they could perhaps put a programme in place to see if this behavior could be reduced without the need for surgery. the parents agreed. following collection of data from additional observations, both at home and at school, a plan was developed. a sensory tray of all ‘goo-like’ items was made and placed in the main family room in the home. alongside this, a visual sign for the tray was made available across the home environment for the child to access. the parents were educated in how to identify when the excessive saliva related behaviors were about to occur – these included heightened vocalisations, 204 january 2023, volume 15, issue 3, 199-208 hand-stimming, and facial movements. when the parents saw these behaviors, they were immediately to prompt the child to point to the visual for the sensory table visual and then immediately redirect him to the sensory tray. he was then requested to ‘play’ and the parents modelled some of the playing behaviors for him with the items in the tray. this strategy was implemented successfully in the home with the saliva related behavior reducing significantly before it was introduced in the school setting. the end results was that surgery was no longer required and the behavior had decreased with the child being fully reintegrated back into school and other social events. a critique of this case study it is awful that the child was not accepted for who he was by both the medical professional and the aba professional! instead, they each attempted to normalize him, one using drugs and then proposing an invasive surgical procedure. the aba professional was more deceptive and hood-winked the child into changing his behavior, thereby refusing to acknowledge the child’s need to engage in his preferred activity! conclusion with so many parents of profoundly autistic children now championing aba while others try to stop all aba, it is clear that something is amiss. as can happen in any profession, some people have been on the receiving end of malpractice. in medical practice, some people have even been murdered by their doctors (gunn, 2010) and of course, the actions of such people must be restricted, but not at the expense of the entire discipline of medicine. the same has not been the case with aba. it seems that some malpractice has led to the condemnation of the whole science. in the midst of the struggle for equality, one very important point should not be overlooked. there is a strategy often fostered by an establishment in many walks of life called “divide et impera” [divide and conquer], an approach that julius cesar used successfully 22 centuries ago (razzetti, 2018). while autistic self-advocates and behavior analysts are at logger heads, the establishment does not need to change and can continue to discriminate unhindered. this paper is a call from behavior analysts to autistic self-advocates to heed milton’s (2018) call for the full appreciation the double empathy problem and join together against the real nemesis of autistic people, societal and political discrimination, inertia, and indolence. competing interests authors are irish doctoral-level behavior analysts and identify as somewhere on the spectrum of human diversity. they have no financial or non-financial interests that are directly or indirectly related to the work submitted for publication. no funding was received for research reported here. author contributions authors contributed equally to the paper. references aba4all. (2022). aba parent support group. applied behavior analysis available for all. https://www. abaa4all.com/ abai. (2022). association for behavior analysisinternational. website. https://www. abainternational.org/ american psychiatric association. (2013). dsm-5. diagnostic and statistical manual of mental disorders. http://bit.ly/2bexru3 apba. (2016). identifying applied behavior analysis interventions. white paper. association of professional behavior analyts. https://www. bacb.com/wp-content/uploads/apba-2017white-paper-identifying-aba-interventions1. pdf baer, d. m., wolf, m. m., & risley, t. r. (1968). some current dimensions of applied behavior analysis1. journal of applied behavior analysis, 1(1), 1310980. https://doi.org/10.1901/jaba.1968.1-91 baer, d. m., wolf, m. m., & risley, t. r. (1987). some stillcurrent dimensions of applied behavior analysis. journal of applied behavior analysis, 20(4), 313– 327. https://doi.org/10.1901/jaba.1987.20-313 baron-cohen, s. (2014). what scientific idea is ready for retirement? radical behaviourism. edge. https://www.edge.org/response-detail/25473 benzel, e. (2022). dunning-kruger effect. in world neurosurgery (vol. 160). https://doi.org/10.1016/j. wneu.2022.02.046 blakemore, l. (2021). the long road: defending the rights of children on the autism spectrum for access to the science of aba in the uk: in the community, the curriculum and the courtroom. centre for behaviour analysis, queen’s university belfast. https://www.youtube.com/ watch?v=bzn-zjwdeps&t=1317s borowsky, h., morinis, l., & garg, m. (2021). disability and ableism in medicine: a curriculum for medical students. mededportal : the journal of teaching and learning resources, 17. https:// doi.org/10.15766/mep_2374-8265.11073 205 autism and behavior analysis: from dissonance to dialogue / dillenburger & keenan buivydaite, r., newton, c. r., & prasauskiene, a. (2017). scoping review: autism research in baltic states—what is known and what is still to be studied. in review journal of autism and developmental disorders (vol. 4, issue 4). https:// doi.org/10.1007/s40489-017-0114-4 byrne, h., & byrne, t. (2000). mikey – dealing with courts, tribunals and politicians. in m. keenan, m. henderson, k. dillenburger, & k. p. kerr (eds.), applied behaviour analysis and autism: building a future together (pp. 208–218). london and philadelphia: jessica kingsley publishers. cambridge centre for behavior studies. (2022). cambridge centre for behavior studies. https:// behavior.org cambridge university press. (2020). cambridge dictionary: english dictionary, translations & thesaurus. in cambridge university press. chiesa, m. (1994). radical behaviorism: the philosophy and the science. in radical behaviorism: the philosophy and the science. chiesa, m. (2006). aba is not “a therapy for autism.” in applied behaviour analysis and autism: building a future together. cooper, j. o., heron, t. e., & heward, w. l. (2007). applied behavior analysis. applied behavior analysis, 2(2), 2–46. https://doi.org/10.1037/13937004 dillenburger, k. (1991). violent bereavement: widows in northern ireland. avebury. https://www. amazon.co.uk/violent-bereavement-widowsnorthern-ireland/dp/1856283003/ref=sr_1_1?ie= utf8&qid=1540580787&sr=8-1&keywords=dillenb urger+bereavement dillenburger, k. (2007). a behavior analytic perspective on victimology. international journal of behavioral consultation and therapy, 3(3), 433– 448. https://doi.org/10.1037/h0100817 dillenburger, k., akhonzada, r., & fargas, m. (2006). post-trauma: is evidence-based practice a fantasy? international journal of behavioral consultation and therapy, 2(1), 94–106. https:// doi.org/10.1037/h0100770 dillenburger, k., fargas, m., & akhonzada, r. (2007a). psychological impact of long-term political violence: an exploration of community service users. traumatology, 13(2), 15–25. https://doi. org/10.1177/1534765607302279 dillenburger, k., fargas, m., & akhonzada, r. (2007b). the pave project report: exploring the effectiveness of services for victims of the troubles in northern ireland. ofmdfm/qub. http://cain.ulst.ac.uk/ v i ct i m s /d o c s /g ro u p/p a ve/p a ve_ p ro j e ct_ report_07.pdf dillenburger, k., & keenan, m. (2009). none of the as in aba stand for autism: dispelling the myths. journal of intellectual and developmental disability, 34(2), 193–195. https://doi. org/10.1080/13668250902845244 dillenburger, k., keenan, m., doherty, a., byrne, t., & gallagher, s. (2010). living with children diagnosed with autistic spectrum disorder: parental and professional views. british journal of special education, 37(1), 1–25. https://doi. org/10.1111/j.1467-8578.2010.00455.x dillenburger, k., keenan, m., doherty, a., byrne, t., & gallagher, s. (2012). aba-based programs for children diagnosed with autism spectrum disorder: parental and professional experiences at school and at home. child and family behavior therapy, 34(2), 111–129. https://doi.org/ 10.1080/07317107.2012.684645 dillenburger, k., keenan, m., & gallagher, s. (2015). a seat at the table: families affected by autism. amazon kindle. https://www.amazon.co.uk/ seat-table-families-affected-autism-ebook/ dp/b00uoxf3f0/ref=sr_1_1?ie=utf8&qid=154049 0808&sr=8-1&keywords=dillenburger dillenburger, k., mckerr, l., & jordan, j.-a. (2014). lost in translation: public policies, evidence-based practice, and autism spectrum disorder. international journal of disability, development and education, 61(2), 134–151. https://doi.org/10.1 080/1034912x.2014.905059 fargas-malet, m., & dillenburger, k. (2016). intergenerational transmission of conflictrelated trauma in northern ireland: a behavior analytic approach. journal of aggression, maltreatment and trauma, 25(4), 436–454. https://doi.org/10.1080/10926771.2015.1107172 ferster, c. b., & demyer, m. k. (1962). a method for the experimental analysis of the behavior of autistic children. american journal of orthopsychiatry, 32(1). https://doi.org/10.1111/j.1939-0025.1962. tb00267.x glenn, s. s. (1988). contingencies and metacontingencies: toward a synthesis of behavior analysis and cultural materialism. the behavior analyst, 11(2). https://doi.org/10.1007/ bf03392470 206 january 2023, volume 15, issue 3, 199-208 glenn, s. s. (2004). individual behavior, culture, and social change. behavior analyst, 27(2). https:// doi.org/10.1007/bf03393175 gómez-durán, e. l., guija, j. a., & ortega-monasterio, l. (2014). medical-legal issues of physical and pharmacological restraint. medicina clinica, 142(suppl. 2). https://doi.org/10.1016/s00257753(14)70068-5 graber, a., & graber, j. (2023). applied behavior analysis and the abolitionist neurodiversity critique: an ethical analysis. behavior analysis in practice. https://doi.org/10.1007/s40617-023-00780-6 green, g. (2016). science, pseudoscience and antiscience theories in autism. different roads to learning. https://blog.difflearn. com/2016/07/14/pseudoscience-antisciencetheories-autism/ gunn, j. (2010). dr harold frederick shipman: an enigma. in criminal behaviour and mental health (vol. 20, issue 3, pp. 190–198). https://doi. org/10.1002/cbm.768 heward, w. l., critchfield, t. s., reed, d. d., detrich, r., & kimball, j. w. (2022). aba from a to z: behavior science applied to 350 domains of socially significant behavior. perspectives on behavior science, 45(2), 327–359. https://doi.org/10.1007/ s40614-022-00336-z hofer, g., mraulak, v., grinschgl, s., & neubauer, a. c. (2022). less-intelligent and unaware? accuracy and dunning–kruger effects for self-estimates of different aspects of intelligence. journal of intelligence, 10(1). https://doi.org/10.3390/ jintelligence10010010 howard, j. s., sparkman, c. r., cohen, h. g., green, g., & stanislaw, h. (2005). a comparison of intensive behavior analytic and eclectic treatments for young children with autism. research in developmental disabilities, 26(4), 359–383. howard, j. s., stanislaw, h., green, g., sparkman, c. r., & cohen, h. g. (2014). comparison of behavior analytic and eclectic early interventions for young children with autism after three years. research in developmental disabilities, 35(12), 3326–3344. https://doi.org/10.1016/j. ridd.2014.08.021 irish post. (2022). “no irish, no blacks, no dogs”. no proof? how the infamous “no irish, no blacks, no dogs” signs may never have existed. irish post. kazdin, a. e. (2010). single-case research designs: methods for clinical and applied settings (2nd ed.). oxford university press. keenan, m. (2016). aba and the nail that sticks out … aba – klinec, ktorý vytŕča….špeciálny pedagóg, časopis pre špeciálnopedagogickú teóriu a prax, 5. ročník, 2016, č. 1. p. 3-14 (slovakian journal of special education) keenan, m., & dillenburger, k. (2000). images of behavior analysis: the shaping game and the behavioral stream. behavior and social issues, 10, 19–38. https://doi.org/10.5210/bsi.v10i0.132 keenan, m., & dillenburger, k. (2020). behaviour analysis: a primer. celtic fringe productions. http://www. behaviouranalysis.eu.com keenan, m., & dillenburger, k. (2011). when all you have is a hammer ...: rcts and hegemony in science. research in autism spectrum disorders, 5(1), 1–13. https://doi.org/10.1016/j.rasd.2010.02.003 keenan, m., & dillenburger, k. (2020). nurturing. celtic fringe productions. https://www.youtube.com/ watch?v=tkvka5uk7rq&t=8s keenan, m., dillenburger, k., doherty, a., byrne, t., & gallagher, s. (2010). the experiences of parents during diagnosis and forward planning for children with autism spectrum disorder. journal of applied research in intellectual disabilities, 23(4), 390–397. https://doi.org/10.1111/j.14683148.2010.00555.x keenan, m., dillenburger, k., konrad, m.-h., debetencourt, n., vuksan, r., kourea, l., pancocha, k., kingsdorf, s., brandtberg, h. j., ozkan, n., abdelnour, h., da costa-meranda, m., schuldt, s., mellon, r., herman, a., tennyson, a., ayvazo, s., moderato, p., attard, n., … gallagher, s. (2022). professional development of behavior analysts in europe: a snapshot for 21 countries, behavior analysis in practice. behavior analysis in practice. keenan, m., kerr, k. p., & dillenburger, k. (2000). parents education as autism therapists. jessica kingsley publishers. keenan, m., kerr, k. p., dillenburger, k., & röttgers (translator), h. r. (2000). eltern als therapeuten von kindern mit autismus-spektrum-störungen: selbständigkeit fördern mit applied behaviour analysis. kohlhammer verlag. https://www. a m a zo n .d e/ e l te r n -t h e ra p e u te n k i n d e r n autismus-spektrum-störungen-selbständigkeitebook/dp/b00pwi24i6/ref=sr_1_2?ie=utf8&qid= 1538923294&sr=8-2&keywords=dillenburger koegel, r. (2011). o. ivar lovaas (1927-2010). american psychologist, 66(3), 227–228. https://doi. org/10.1037/a0022693 207 autism and behavior analysis: from dissonance to dialogue / dillenburger & keenan kupferstein, h. (2018). evidence of increased ptsd symptoms in autistics exposed to applied behavior analysis. advances in autism, 00–00. https://doi.org/10.1108/aia-08-2017-0016 larsson, e. v. (2021). is applied behavior analysis (aba) and early intensive behavioral intervention (eibi) an effective treatment for autism? a cumulative review of impartial reports. https:// doi.org/10.1542/peds.2007-2362. leadbitter, k., buckle, k. l., ellis, c., & dekker, m. (2021a). autistic self-advocacy and the neurodiversity movement: implications for autism early intervention research and practice. frontiers in psychology, 12. https://doi.org/10.3389/ fpsyg.2021.635690 leadbitter, k., buckle, k. l., ellis, c., & dekker, m. (2021b). autistic self-advocacy and the neurodiversity movement: implications for autism early intervention research and practice. frontiers in psychology, 12. https://doi.org/10.3389/ fpsyg.2021.635690 lipinski, s., boegl, k., blanke, e. s., suenkel, u., & dziobek, i. (2022). a blind spot in mental healthcare? psychotherapists lack education and expertise for the support of adults on the autism spectrum. autism, 26(6). https://doi. org/10.1177/13623613211057973 lord, c., charman, t., havdahl, a., carbone, p., anagnostou, e., boyd, b., carr, t., de vries, p. j., dissanayake, c., divan, g., freitag, c. m., gotelli, m. m., kasari, c., knapp, m., mundy, p., plank, a., scahill, l., servili, c., shattuck, p., … mccauley, j. b. (2022). the lancet commission on the future of care and clinical research in autism. in the lancet (vol. 399, issue 10321). https://doi. org/10.1016/s0140-6736(21)01541-5 lovaas, o. i. (1987). behavioral treatment and normal educational and intellectual functioning in young autistic children. journal of consulting and clinical psychology, 55(1), 3–9. milton, d. (2012). the normalisation agenda and the psycho-emotional disablement of autistic people. in autonomy, the critical journal of interdisciplinary autism studies (vol. 1, issue 1). milton, d. (2018). the double empathy problem . national autistic society, 3–2. https://www. a u t i s m . o r g . u k / a d v i c e a n d g u i d a n c e / professional-practice/double-empathy nario-redmond, m. r., kemerling, a. a., & silverman, a. (2019). hostile, benevolent, and ambivalent ableism: contemporary manifestations. journal of social issues, 75(3). https://doi.org/10.1111/ josi.12337 nice. (2013). national institute for health and clinical excellence autism in children and young people guideline consultation comments table 29 march 2013-10 may 2013. https://www. n i c e.o rg .u k /g u i d a n c e/c g 1 70/d o c u m e nts / autism-management-of-autism-in-childrenand-young-people-consultation-commentstable-and-responses2 nielson, s., bray, l., carter, b., & kiernan, j. (2021). physical restraint of children and adolescents in mental health inpatient services: a systematic review and narrative synthesis. journal of child health care, 25(3). https://doi. org/10.1177/1367493520937152 parker, s. (2015). autism: does aba therapy open society’s doors to children, or impose conformity? the guardian, 03(20). https://www.theguardian. com/society/2015/mar/20/autism-does-abatherapy-open-societys-doors-to-children-orimpose-conformity phelps, b. j. (2007). why we are still not cognitive psychologists: a review of why i am not a cognitive psychologist: a tribute to b. f. skinner. the behavior analyst, 30(2), 217–226. razzetti, g. (2018). there are two ways to lead people – divide and conquer is one of them. fearless culture. https://www.fearlessculture.design/ blog-posts/dividing-people-is-the-best-wayto-lead sandoval-norton, a. h., & shkedy, g. (2019). how much compliance is too much compliance: is longterm aba therapy abuse? in cogent psychology (vol. 6, issue 1). https://doi.org/10.1080/23311908.2 019.1641258 schnitzer, k., merideth, f., macias-konstantopoulos, w., hayden, d., shtasel, d., & bird, s. (2020). disparities in care: the role of race on the utilization of physical restraints in the emergency setting. academic emergency medicine, 27(10). https:// doi.org/10.1111/acem.14092 silberman, s. (2015). neurotribes: the legacy of autism and how to think smarter about people who think differently. allen & unwin. 208 january 2023, volume 15, issue 3, 199-208 simple steps. (2013). simple steps autism: the online teaching platform for the treatment of autism. parents education as autism therapists. https:// simplestepsautism.com/ skinner, b. f. (1953a). operant behavior. science and human behavior. https://doi.org/10.3390/ ijerph8093528 skinner, b. f. (1953b). science and human behaviour. in the macmillan company. skinner, b. f. (1957). verbal behavior. verbal behavior. skinner, b. f. (1977). why i am not a cognitive psychologist. behaviorism; published by cambridge center for behavioral studies (ccbs), 5(2), 1–10. smith, t., & eikeseth, s. (2011). o. ivar lovaas: pioneer of applied behavior analysis and intervention for children with autism. in journal of autism and developmental disorders (vol. 41, issue 3, pp. 375–378). https://doi.org/10.1007/s10803-0101162-0 surgeon general. (1999). mental health: a report of the surgeon general. national institute of mental health (u.s.). https://profiles.nlm.nih.gov/ps/ retrieve/resourcemetadata/nnbbhs tarvainen, m. (2019). ableism and the life stories of people with disabilities. scandinavian journal of disability research, 21(1). https://doi. org/10.16993/sjdr.632 thorne, j., & barr, g. (2022). then barbara met alan. bbc iplayer . https://www.bbc.co.uk/iplayer/ episode/m0015p4q/then-barbara-met-alan thyer, b. a. (2009). epistemology: a behavior analytic perspective. . humana.mente. https://www. h u m a n a m e n te . e u / i n d ex . p h p/ h m / i s s u e/ view/11/issue_11, 11(11), 45–64. thyer, b. a., & myers, l. l. (1999). on science, antiscience, and the client’s right to effective treatment. social work, 44(5). https://doi.org/10.1093/ sw/44.5.501 thyer, b. a., & pignotti, m. (2010). science and pseudoscience in developmental disabilities: guidelines for social workers. journal of social work in disability and rehabilitation, 9(2). https://doi.org/10.1080/1536710x.2010.493480 uncrpd. (2006). united nations convention for persons with disabilities. united nations. https:// www.un.org/development/desa/disabilities/ convention-on-the-rights-of-persons-withdisabilities.html unumb, l. (2013). keynote address by dr lorri unumb at centre for behaviour analysis (qub) conference. queen’s university belfast. h t t p s : // m e d i a s i te . q u b . a c . u k / m e d i a s i te / play/564a563b432440b68375d102170c337a1d volkmar, f. r., bregman, j., cohen, d. j., & cicchetti, d. v. (1988). dsm-iii and dsm-iii-r diagnoses of autism. american journal of psychiatry, 145(11), 1404–1408. https://doi.org/10.1176/ajp.145.11.1404 vyse, s. (2022). autism politics and the death of truth and freedom. skeptical inquirer. the magazine for science and reason. https:// skepticalinquirer.org/exclusive/autism-politicsand-the-death-of-truth-and-freedom/ . international electronic journal of elementary education, september 2016, 9(1), 87-108. 87 an investigation of primary school teachers’ pck towards science subjects using an inquiry-based approach* menşure alkiş küçükaydin a ** şafak uluçinar sağir a aamasya university, turkey received: 15 april 2016 / revised: 21 august 2016 / accepted: 29 august 2016 abstract in this study, the pedagogical content knowledge (pck) of four experienced primary school teachers was investigated within the “let’s solve the riddle of our body unit”. the pck investigation adopted a learning approach based on inquiry, content representation and pedagogical and professional-experience repertoires (pap-ers), and interview forms were used as data collection tools. during the course of the research, the findings obtained from observations made during a total of 18 course hours formed the basic data source of the study. according to the results of the study, in which descriptive and content analysis were used concurrently, primary school teachers lack subject matter knowledge, do not interrogate the pre-knowledge of students and some misconceptions exist regarding about blood moves and exercise with pulse. additionally, some deficiencies were detected in the curriculum, i.e., it offers non-inquisitional knowledge. furthermore, teachers employee assessment methods with traditional teaching methods and techniques. in the context of an inquiry-based learning approach, teachers appeared to believe that classroom activities were adversely affected by the physical conditions (class size, lack of laboratory etc.), students’ cognitive levels and parent profiles. the result of this study revealed that pck components affect one another. the pck findings pertaining to primary school teachers as it concerns the unit are briefly discussed and some suggestions about the development of pck are submitted. keywords: pedagogical content knowledge; inquiry-based approach; primary school teacher; science; teacher competencies introduction teacher competency is an effective factor on student behaviour and learning, and also plays a role in pedagogical progress and student learning. what pedagogical knowledge is and how teachers perceive their own pedagogy must be considered a separate component of pedagogical content knowledge. according to the pedagogical content knowledge (pck) * this paper was written from the first author’s ongoing doctoral dissertation under supervision of associate professor şafak uluçınar-sağır at amasya university, turkey. ** corresponding author: menşure alkış-küçükaydın, amasya university, institute of social sciences, amasya, turkey. phone: +90 530 328 58 15 e-mail: mensurealkis@hotmail.com 88 description by shulman (1987), it is important to arrange the content knowledge owned by the teacher according to the interests and needs of students. this can be done by using alternative approaches to such as analogy, demonstration and simulation and the transfer of knowledge. in accordance with hope and townsend (1983), a matter that must be considered at this point is being knowledgeable about what students think about, because misconceptions on the part of the teacher can have a negative effective in the preknowledge and comprehension deficiencies of the students. the instructional strategies used by the teacher during courses serve as indicators of being a specialist in the identification and elimination of existing misconceptions (magnusson, krajcik, & borko, 1999); therefore, pck has multiple dimensions. shulman (1987) continued to expand on pck definitions by adding different dimensions from the approach initiated by magnusson and others (1999). while emphasizing subject matter knowledge (smk), which is important in science learning and teaching, a complete consensus among the various models developed cannot be ensured (abell, 2007; smith, 1999; van driel, verloop & de vos, 1998). despite this, two primary components are agreed upon in pck studies; these focus on the knowledge pertaining to students’ understanding and the available research on smk (van driel et al., 1998). moving on from the basic elements, different models – in which different aspects related to pck are highlighted – have been developed (cochran, deruiter & king, 1993; cochran, king & deruiter, 1991; grossman, 1990; marks, 1990; tamir, 1998). these models suggest that components can be analysed independently or together. as a result, a complete definition of pck cannot be formulated. in order to classify as pck, shown below all components must be independently evaluated using a holistic approach. pck understand to be considered independently of each ingredient is essentially the same time evaluating a holistic approach. park and chen (2012) developed a pentagon model that highlights the importance of integrating components, thereby creating synthesis among all pck components (figure 1). figure 1. pentagon model of pck for teaching science (park & chen, 2012, p.925). in the pentagon model, knowledge of students’ understanding, orientation to teaching science, knowledge of instructional strategies for teaching science, knowledge of assessment of science learning and knowledge of assessment of science learning components are included and analysed as a whole in pck studies. this inclusion of all components in an equally-weighted manner provides strong consistency (park & chen, using an inquiry-based approach / alkış-küçükaydın&uluçınar-sağır 89 2012). to dominate the current curriculum of teachers and make the necessary adjustments and guidance in this respect is related to the curriculum knowledge component of pck (falk, 2011); this reveals the skills of the teacher as it concerns curriculum concepts (park & oliver, 2008b). the component of science education assessment reflects the status of learners using an approach that includes appropriate measurement tools and activities and methods in line with the current curriculum (park & oliver, 2008b). for students to understand the basic components of pck, teachers are required to have good smk and knowledge about student behaviours related to this knowledge (driel, jong, & verloop, 2002). when considered in terms of these components, good teaching strategies dismiss misconceptions about conceptions. science teaching strategies includes learning cycles, conceptual change strategies and the inquiry-based approach as a whole (park & oliver, 2008b). orientation information for science education covers instructional decisions. beliefs about the nature of science and faith for science education and pck knowledge is a component that cannot be discussed separately from smk, because it may include information about pck by making teaching experiences monitoring through the investigations of the smk (driel, jong, & verloop, 2002). indeed, several researchers have concluded that there is a deficiency among pck components due to about smk causes lack (cohen & yarden, 2009; jones & moreland, 2004; sperandeo-mineo, fazio & tarantino, 2005; veal & kubosko, 2003). hence, in pck studies, smk cannot be considered separately from these components. when considering the relevant literature studies, the effects of various experiences, applications and models on the development process of pck (henze, van driel & verloop, 2008; nilsson, 2008), the effect of smk on pck and their mutual interactions (rollnick, bennett, rhemtula, dharsey & ndlovu, 2008; sperandeo-mineo et al., 2005), as well as the pck status of teachers in general or about a certain subject were examined (lee, brown, luft & roehrig, 2007; lee & luft, 2008). overall, considering all of the studies involved, research about pck concerning primary school teachers, who present science lessons at the primary level, were included in abundance and it was observed that the relationship between pck components among these teachers have not been sufficiently examined. in addition, studies carried out that focus on specific subject matter in pck were individually assessed and a number of components were examined (mulhall, berry & loughran, 2003). in turkey, the middle school science curriculum was revised in 2013 and an inquiry-based approach was adopted. with this new approach, a basic vision expressed as "to train all students to be science literate individuals" (the ministry of national education, 2013) was developed. an inquiry-based approach plays an important role in obtaining desirable outcomes for student achievement. a “pedagogy of joint discovery” (levy & petrulis, 2012) approach is important for the development of critical thinking among students and for the advancement of intellectual and practical skills required for life. therefore, teachers play a role in the development of in-depth understanding among students, using the appropriate tools to do so and for transferring information; at the same time, students are responsible for their own learning, assessment and for designing a research and questioning process at the centre of their learning. as such, the pck framework of this study constitutes the details pertaining to how teachers employ a status of inquiry-based approach in their courses. from this perspective, the question, “how is the pck of primary school teachers geared toward the “let’s solve the riddle of our body unit” in terms of employing an inquiry-based approach?” constitutes the problem statement of the current study, based on the pentagon model. 90 method in this study – conducted to examine the pck of primary school teachers in the “let’s solve the riddle of our body unit” in terms of the inquiry-based approach – qualitative research and a case study were employed as a research design. the unit of analysis in this study was primary school teachers and included more than one case. thus, a “multiple case design” (yin, 2003, p.34) was employed for this research. the study was conducted among fourth grade primary school teachers and the study group was selected from teachers who worked in the village and town centres of tokat province (turkey) by using purposive sampling techniques. in purposive sampling, variables such as seniority/length of service of primary school teachers teaching fourth grade students and the campus of the school and how many times they taught the fourth grade were taken as a basis for volunteer. by doing so, it was assumed that the teachers with different cultural structures would participate in the study and that this would reveal the different dimensions of the problem under study. study participants are introduced below in detail and were observed during eight acquisitions and 18 course hours in total by placing cameras in class. the data related to pck were collected using other data collection tools. participants the group under study comprised three female and one male primary school teachers working in different areas of tokat province during the 2015-2016 academic year and teaching fourth grade. participants were given code names within the study group; their demographic knowledge and statements that reflected them are shown in table 1. when creating the study group, we preferred selecting fourth grade teachers who applied the 2013 revised science lesson curriculum. at the point of selecting participants, a criterionsampling technique was used. in criteria sampling, variables such as seniority/length of service of the primary school teachers teaching fourth grade students, the campus of the school and how many times they taught the fourth grade were taken as a foundation for volunteer. for this reason, some criteria such as teachers teaching at the fourth grade level and having at least five years of experience were taken into account. by doing so, it was ensured that divergently equipped teachers took part in the study and that as a result, different dimensions of the problem would arise. table 1. participant demographics code name zeynep ozge serkan esra gender female female male female age 31 35 60 53 professional seniority 8 years 14 years 37 years 32 years working time at school (years) 3 9 9 15 number of students 12 10 39 10 fourth grade teaching experience 3 6 8 9 graduation programmes faculty of education faculty of education graduate school of education graduate school of education zeynep had always taught in multi-grade classes. they taught fourth grade in combined classes and zeynep stated that she had taught fourth grade independently for the first time in the school where she is currently employed. zeynep described herself as a teacher who did not follow new methods and techniques and who engages in courses without preparation. using an inquiry-based approach / alkış-küçükaydın&uluçınar-sağır 91 ozge stated that she had taught fourth grade four times as a combined class and twice independently. as a teacher, she described herself as motherly and a person who ascribes importance to awards. serkan stated he had taught fourth grade students seven or eight times. as a teacher, he described himself as someone who valued his students. esra stated she had taught fourth grade students nine times in a consolidated class. as a teacher, she described herself as someone who saw herself as a friend to her students and as a teacher who takes an interest in their problems. data collection and analysis in the study, observations, interviews and document analysis were included as data collection methods and content representation; also included were pap-ers, interviews and several types of written materials applied by the teachers in their classes. content representation (core): content representation helps the teachers to demonstrate their teaching approach to the subject and questions the reasons for the specific approach applied. the core table developed by loughran et al. (2012) indicates big ideas about a subject in the top column, with teaching-related items below it. in this table, information such as big ideas, planning what to teach students related to these ideas, why the information is necessary for students, various ideas about the subject being taught, limitations and challenges concerning knowledge, awareness of students’ ideas, the factors involved in teaching the ideas and special education procedures are examined. the statements made in the content representation section were conveyed to lecturers who have researched the field of pck (two professors and one associate professor) via e-mail and expert opinions (two teachers) were obtained to test its intelligibility. following the proposed corrections, it was applied by giving the final version. pap-ers: loughran et al. (2012) suggest using pap-ers (pedagogical and professionalexperience repertoires) for determining pedagogical content knowledge. as a means of expression, pap-ers is also important as a methodology, evident by the different stages of teaching scenarios exhibited by teachers in pap-ers tables. therefore, based on the different formats (at that time, the students’ actions, lesson plans, etc.), the teacher's pedagogical knowledge and concept presentations can be revealed (loughran et al., 2000). though content representation and pap-ers cannot fully reveal the axioms of teachers, it is possible to argue that these methods demonstrate teachers’ beliefs about their pcks, as well as their reflections about their application (mulhall, berry & loughran, 2003). interview form: the interview form, prepared to reveal the pcks of the primary school teachers in science subjects and using the inquiry-based approach, questions were prepared in the context of orientation pertaining to science teaching, students’ understanding of science knowledge, teaching strategies, knowledge about teaching science, as well as knowledge about science learning assessments and curriculum knowledge. prior to the prepared questions being applied, expert opinions were collected; the questions were then applied to two primary school teachers and a pilot study was conducted. to gain expert opinions, we contacted a science educator professor, an associate professor and two assistant professors who have completed studies on pck. the face-to-face interviews were audio-recorded and then the data were converted into written form. the data obtained from the interviews were analysed using content analysis. in this study, both descriptive and content analyses were used. the data obtained from the observation form and core tables were subjected to descriptive analysis. data gathered from the interviews were converted into written form from voice recording. using an 92 inductive approach to interview form, for this was first coding and free coding was applied by the researcher, the obtained codes were collected through specific groups and finally, themes were noted. the nvivo 10.0 data analysis software was applied to save time during the data analysis process and to better organize the process. a short sample of applied codes is shown in appendix a. reliability and validity to ensure the reliability and validity of the study, a focused literature review was conducted; several data collection tools were employed and findings were checked for consistency once participants confirmed the findings. a working group was introduced and by describing the work environment, data were gathered from observations and interviews. during the analysis of interview questions, raw data were coded and emergent themes were given meaning by reviewing the literature. in addition to creating codes, teachers' statements were exemplified using one-to-one sentences (appendix a). after completing the coding process, codes and themes were sent to an independent coder. the codes of these two coders were then compared using the nvivo software and cohen's kappa coefficient was calculated. the reason for using this coefficient during calculations is that it allows for comparing codes applied by only two people (bazeley & jackson, 2015). during the study, 39 codes were gathered and 35 were in agreement. while code agreement was calculated as 35/39=0.89, cohen's kappa coefficient was determined as 0.81.according to bazeley and jackson (2015),a kappa-1 value shows perfect agreement, while other values close to this result are qualified as also having near perfect agreement. it is therefore possible to say that the applied codes in this study were reliable. findings the content representation forms completed by participant teachers regarding the "let’s solve the riddle of our body unit” are presented in appendix b. according to the data obtained from appendix b, teachers stated that they planned to teach students about the types of joint, bone and muscles related to the unit. however, these issues were not included in fourth grade science achievements. similarly, they stated that students will experience difficulty understanding the vessel types, which were not included among the for fourth grade achievements. teachers stated that they used mainly question-andanswer, discussion, experiments, observation, presentation, watching slides and reading and narration as methods and techniques. additionally, when the participant primary school teachers were asked to recommend alternative methods and techniques, they noted the optimal ways in the classroom conditions and that animated cartoons and presentations must be presented for students to understand the information better. when the documents that participant teachers used in their classes were analysed it was observed that they prepared too many quantitative questions. additionally, they gave weight to knowledge-level questions but did not focus on synthesis or higher level questions. only serkan prepared questions at the analysis level; ozge prepared questions from bloom taxonomy's initial three levels. apart from serkan, all the other teachers downloaded ready-made questions from education sites. apart from ozge, all other teachers included questions unlike of acquisition in their exams, which concerned bone types. the pap-ers analysis form was prepared using camera records of the primary school teacher participants, as well as the written documents they used in their classrooms (appendix c). when the pap-ers form and document examination was created, it was observed that the participant teachers used ready-made questions and curriculum unlike of acquisition in their classroom activities. additionally, it was observed that they had misconceptions about the subjects of tooth structure, blood circulation, breathing using an inquiry-based approach / alkış-küçükaydın&uluçınar-sağır 93 correctly and about the ingredients of the arterial and venous blood and joint structures, and as such, were likely unaware of students' misconceptions. zeynep used traditional teaching methods and techniques to teach the unit and could not evaluate sufficiently. she was unable to help her students sufficiently regarding the contraction and relaxation of muscles, vascular structures and blood circulation. additionally, she taught unlike the acquisition and as a result, had misconceptions about the structure of the tooth, blood circulation and breathing. since she was unprepared for the lesson, it was observed that she did not use inquiry-based teaching. in addition to not having enough information about blood structure, ozge did not allow her students to ask questions during her lessons. ozge always used educational videos; however, she simply let her students watch these without commenting on them and did not use an inquirybased approach. ozge was therefore inadequate in her teaching as it concerned content knowledge, which she stated during the interview. serkan was observed as entirely employing traditional teaching methods. he also stated during the interview that he did not know any inquiry-based methods and that was unable to make adequate assessments. additionally, he had misconceptions about joints. as he had inadequate information about the unit’s content, he did not move beyond using the textbook. esra used quite many analogies in her classes, conducted experiments and enabled children to effect research and ask questions. however, by teaching outside the content of the unit, esra was unable to sufficiently evaluate the research and questioning skills of students. teachers presenting in formation unlike the acquisition, as well as missing information, is associated with a lack of smk. it also emerged that, when evaluating science learning, traditional teaching methods and techniques were being used. as a result of the interviews conducted with the participating primary school teachers, the model presented in figure 2 was created. the model addresses personal and professional self-esteem, teaching methods and techniques, assessment information, programme information and the obstacles present in the learning environment. figure2.1. teachers’ personal and professional perceptions about themselves. 94 figure 2.2. teaching methods and techniques adopted by the teachers. figure2.3. teachers’ approaches to evaluating scientific understanding. figure 2.4. teachers’ thoughts about the programme. using an inquiry-based approach / alkış-küçükaydın&uluçınar-sağır 95 figure 2.5. obstacles present in the learning environment according to teachers. as shown in figure 2.1, the primary school teachers evaluated themselves teaching based on inquiry-based approach. according to two sub-dimensions, i.e., personal and selfesteem in terms of their. when the professional esteem dimension was examined, it was noted that teachers viewed themselves inadequately in terms of pedagogical components and conducted teaching based on a behavioural approach. teachers expressed their personal and professional features as follows: “i think of the students as my own children… i act as a motherly teacher...while teaching fourth grade subjects, i care about teaching one step further from the level of the students… i provide more information and sometimes this can be problematic.” (zeynep) “i always act like a mother to my students…they are like my sons and daughters and i believe rewarding them is very important for their education.” (ozge) “above all, i accept my students for who they are. this is because…i am a mother. depending on the situation, i sometimes act like their mother and sometimes their friend. of course, i set boundaries to our relations and behave accordingly. i am sincere. i am trying to be interested in all their problems, not only my students’ problems but also that of their families as much as this is possible.”(esra) “i view myself as a well-intentioned teacher… i place importance on my students… i love them and i try to teach them as best as i can.”(serkan) the participant teachers were asked whether there had been any changes to their teaching methods and techniques, compared to approaches that were in line with the revised programme; this resulted in the model shown by figure 2.2. according to this model, primary school teachers stated that they did not experience many changes and that the traditional teaching methods (question-and-answer, demonstrations, observation, taking notes and delivering lectures) were more practical and that pedagogically, inquiry-based approach did not affect them. serkan stated about the methods and techniques he used: “the students must take notes or the teacher must write on the board or the teacher must let students take notes as a means of summarizing the subject.” on the other hand, esra said that she had not changed her approach to teaching because she did not believe that new teaching methods and techniques were innovative. esra stated her ideas as follows: “i think new methods and techniques are simply old techniques with new names. of course i should try different techniques, but sometimes, what we say is “innovative” is simply a time consuming activity. i am not sure whether i can try new methods in crowded classrooms. therefore, instead of being innovative, i consider how much a method is practical and applicable.” primary school teachers were asked to evaluate science education in line with the changing programme and the new approach brought about by pck components, which delivered the model shown in figure 2.3. all of the teachers stated that they did not 96 employ the science education method as suggested by the approach based on inquiry and that they primarily used tests. on this subject, ozge stated: “i don’t know a matter of interest… i do not think that the students can learn much through the project and performance work in science lessons...the students’ successes is associated with teachers’ successes. if the students perform well in the exam, the teacher is also considered as having been successful.” serkan evaluated students’ science assessments according to changes in their behaviours. his ideas were expressed as follow: “the student reflects what he/she has learned similar to a mirror. to what extent can he/she reflect what they see, the acquisitions they have made and the changes in behaviour they have effected? it is a significant process when students reflect what they have learned through their behaviour.” the teachers’ thoughts about the new science curriculum (revised in 2013) and the basic approach adopted by it are shown in the model in figure 2.4. accordingly, the negative thoughts of teachers about the education programme outweighed their positive responses; they particularly did not like being easy of the acquisitions. however, they were satisfied about making allowance for quick and easy learning and being ready of all kinds of knowledge (such as internet) in their hand. ozge stated about the programme: “it is good…science lesson and a simplified one the acquisitions given to us let us to give something but i sometimes say that they not be so easy…i am experiencing difficulty getting used to this programme.” esra stated her ideas about the newly adopted programme and inquiry-based approach as follow: “when the aim of a new programme is given it very good but when i look at back to my students they cannot produce new knowledge by using the provided knowledge and were not able to solve the problem.” in line with the programme revised for the participant primary school teachers, they were asked about the cases that hindered their learning and education, which resulted in the above model (figure 2.5). accordingly, teachers primarily mentioned physical obstacles and highlighted parents’ profiles, which they prescribed importance to for the evaluation of science education. serkan’s statements about this matter were as follows: “the current conditions and facilities of the schools, the acquisitions and research studies are not sufficient for applying all of the innovations that have occurred for science lessons. you saw that we have to fit 39 students into a very small classroom and must teach accordingly.” according to ozge, obstacles to science teaching include school administration, crowded classrooms and the lack of a laboratory. ozge’s stated: “schools are difficult in the centre of tokat…because of the crowded classrooms. in villages it is difficult because of a lack of material. if i take a class with 35-40 students and try to give different materials to each of them within a big school, the school administration will be unhappy. i believe that it is necessary to plan the use of a laboratory for each lesson. consider the nature of fourth grade classes and their science lesson hour and that in some schools, both primary and elementary schools are together in the same building. city schools quickly become overcrowded but for schools in villages, classrooms are not as crowded… however, there is a different problem lack of materials.” results and discussion taking an inquiry-based approach to the participant primary school teachers in the context of the “let’s solve the riddle of our body unit”, this study conclude the presence of some deficiencies in both smk and pck. it was observed that teachers were not able to using an inquiry-based approach / alkış-küçükaydın&uluçınar-sağır 97 fully judge the subject of the unit and this case is effective on their pck. indeed, smk is a prerequisite in the development of pck. the deficiencies observed in smk led teachers to solve fewer problems and to use less effective teaching strategies. moreover, this prevented them from being adequate in terms of understanding students’ level of knowledge (kind, 2009). according to childs and mcnicholl (2007) and gess-newsome and lederman (1999), teaching strategies are affected by smk. in the literature, many studies were found to support the conclusion that a lack of smk affects pck components (cohen &yarden, 2009; kamen, 1996; matese, 2005). in addition, it was observed that several teachers conducted their classes without any prior preparation. being aware of students' misconceptions and understanding their level of knowledge is important for teachers in order to develop effective teaching plans; it is also effective for determining existing misconceptions among students and knowing why students behave in a certain manner (halim & meerah, 2002). this study shows that teachers held some misconceptions about the unit; as such, they lacked knowledge about students’ level of understanding. berg and brouwer (1991) conducted their study, have done with physics teachers and students are unaware of the misconceptions revealed that even led to the wrong student learning in their own misconceptions. smith and neale (1991) also tested similar findings about the knowledge of primary school teachers about light and shadow subject. teachers with deficient concept knowledge are insufficient at constructing information for students, at formulating appropriate questions, creating alternative explanations and structuring questioning. to discover whether science concepts are being understood in-depth, students’ understanding must be properly evaluated, while pinpointing misconceptions at the same time. among teachers weak concept knowledge leads to anxiety, less effective teaching and a decrease in self-sufficiency (czerniak & chiarelott, 1999). participant primary school teachers used traditional teaching methods and techniques during the unit and stated this to be the case during the interviews. only one of the teachers used primarily technology to present classes; however, while doing this, she left the students completely inactive, because the video presentations and animations that she allowed them to watch effectively replaced the teacher in the classroom, with students effectively becoming no more than spectators. this situation led to not many questions being asked, while in the classes of other teachers led to unanswered questions. knowledge of science teaching strategies can help teachers and students to better learn scientific concepts. however, incorrect analogies and examples revealed the misconceptions held by some teachers (gessnewsome & lederman, 1999). according to magnusson, borko and krajcik (1999), using teaching strategies correctly is important for ensuring the conceptual understanding of students. different activities conducted in the classroom can also have a positive effect on the comprehension challenges pertaining to the knowledge acquired by students (grossman, 1990; lederman, gess-newsome & latz, 1994). as seen in the information provided by the participant primary school teachers via the interviews and core, as well as the pap-ers analysis form, they were unable to evaluate science education. they also continued using traditional methods in the context of an inquiry-based approach. the evaluation methods used by teachers are an important element in the development of their pck, because teachers acquire knowledge regarding learning purposes through the evaluation methods they employ. instruction strategies can be monitored more closely through appropriate evaluation. some of the teaching materials employed by teachers supported their teaching strategies. thus, teachers’ knowledge of evaluation contributes to their professional development and is also important for the 98 development and configuration of students understanding the knowledge conferred to them (falk, 2011). as noted in many studies that support the correlation between the evaluation of knowledge and students’ understanding (atkin, coffey, moorthy, sato & thibeault, 2005; tabachnick & zeichner, 1999), pck components are interrelated. this study reached the same conclusion. in a study by bayer and davis (2011) involving science subjects and conducted with primary school teacher candidates, it was concluded that teachers had insufficient science evaluation knowledge. considering the component of science curriculum knowledge related to “let’s solve the riddle of our body unit”, the conclusion reached is that teachers were unaware of the acquisitions in the curriculum that they stated in the core, as was observed in the pap-ers analysis. indeed, in the interviews, teachers mentioned that they were not satisfied with only the simple knowledge acquisitions related to the revised programme and that they added additional acquisitions of their own. however, achieving success with the added acquisitions had not been successful. this study therefore concludes that the primary school teachers who participated in this study lacked the proper science curriculum components. this information is important in terms of teachers having a good command of their subject, changing and orienting activities and providing sufficient conceptual knowledge transfer (park & oliver, 2008b). in a study involving science teachers and pertaining to a breakdown of the ozone layer, bozkurt and kaya (2008) found similar conclusions. they investigated students’ understanding of the knowledge involved, their curriculum knowledge and teaching strategies knowledge pertaining to smk and pck. the study also concluded that candidate teachers had misconceptions about the subject being taught. smk is effective for the development of pedagogical strategies as it concerns teacher evaluations, curriculum applications and curriculum development (jones & moreland, 2005). in addition, in line with an inquiry-based approach, primary school teachers mentioned the physical conditions (class size and lack of laboratory) as factors affecting their classroom teaching. additionally, in the face-to-face interviews, teachers noted that they thought the revised programme to be important, but that they experienced difficulties implementing it. in a study performed with 215 secondary school science teachers that questioned their attitudes, beliefs and values toward scientific research (dibiase & mcdonald, 2015), teachers noted the importance of scientific inquiry, but did not have the skills to perform this themselves. they also noted an inability to evaluate students according to the inquiry-based approach and noted difficulties preparing lesson plans. according to the teachers, their students experienced difficulties understanding scientific concepts; teachers also thought the development of scientific process skills among students to be difficult. as a result, teachers believed scientific inquiry to be important; however, there were some difficulties putting this into practice. a study conducted by ramnarain and schuster (2014) involving five village and city teachers concluded classroom size, access the information source, school culture and the expectations of parents to be important in the development of pck; ideal classroom size and a wellconfigured laboratory triggered the use of an inquiry-based approach and supported the conceptual learning of students. in addition, teachers in schools that were in bad physical condition used tests as an evaluation tool more frequently and according to these teachers, parents with lower socio-economic status wanted their children to take down good notes rather than practising science, because they wanted their children to be employable. this led to teachers using traditional evaluation methods such as tests. the results of the current study show that teachers believe the inquiry-based approach to be useful, but that they view themselves as being inadequately informed about inquiry-based approach. in a study by dibiase and mcdonald (2015) about teachers’ beliefs, attitudes and values toward scientific research and questioning, teachers are shown to believe in the using an inquiry-based approach / alkış-küçükaydın&uluçınar-sağır 99 importance of inquiry, but that they did not believe themselves adequately skilful to prepare activities for such an approach. teachers stated that they were unable to assess their students according to this approach and that they found it difficult to prepare a lesson plan in line with this approach. according to the results obtained from the current research, participant primary school teachers are insufficient in the field of smk; they lack pck components, which negatively affect their teaching. accordingly, by conducting pck examinations in all science subjects, the misconceptions of primary school teachers can be defined and preventive, conceptual studies can be conducted. by organizing in-service training activities and providing specialist help, particularly concerning matters that are believed to be missing by the teachers’ applications can be initiated commonly. according to kramarski (2009), supportive programmes for teachers can contribute to their pedagogical knowledge. similarly, according to qablan and debaz (2015), these programmes can increase teachers’ science teaching skills, making them more effective in assessment, teaching and creating teaching plans. this will in turn have a positive impact on their pedagogies. at the conclusion of course for increasing candidate chemist teachers’ pck regarding the nature of science (nos) by demirdöğen,hanuscin, kondakçı-uzuntiryaki and köseoğlu (2015), one of the results was that teachers’ knowledge about nos and teaching strategies increased. in turkey, a debate on scientific developments can be provided by preparing relevant platforms, beginning with the process of restructuring universities. where the competencies of teachers are concerned, pck development during the past 10 years can be discussed. using these platforms, measures according to pck perceptions can be implemented, on the based opinions of experienced teachers. • • • references abell, s. k. (2007). research on science teacher knowledge. in s. k. abelland n. g. lederman (eds.), handbook of research on science education, pp.1105-1151. new jersey: lawrence erlbaum associates. atkin, j. m., coffey, j. e., moorthy, s., sato, m. & thibeault, m. (2005). designing everyday assessment in the science classroom. new york: teachers college press. avraamidou, l. & zembal-saul, c. (2005). giving priority to evidence in science teaching: a firstyear elementary teacher’s specialized practices and knowledge. journal of research in science teaching, 42(9), 965-986. doi: 10.1002/tea.20081 bazeley, p. & jackson, k. (2015). qualitative data analysis with nvivo (bakla, a. & demir, s.c., tr.). ankara: ani press. berg, t. & brouwer, w. (1991).teacher awareness of student alternate conceptions about rotational motion and gravity. journal of research in science teaching, 28, 3-18. doi: 10.1002/tea.3660280103. beyer, c. j. & davis, e. a. (2011).learning to critique and adapt science curriculum materials: examining the development of pre service elementary teachers’ pedagogical content knowledge. science education, 96(1), 130-157.doi: 10.1002/sce.20466. 100 bozkurt, o. & kaya, o. n. (2008). teaching about ozone layer depletion in turkey: pedagogical content knowledge of science teachers. public understanding of science, 17(2), 261-276. doi: 10.1177/0963662506071787. childs, a. & mcnicholl, j. (2007). investigating the relationship between subject content knowledge and pedagogical practice through the analysis of classroom discourse. international journal of science education, 29(13), 1629-1653. doi: 10.1080/09500690601180817. cochran, k. f., deruiter, j. a & king, r. a. (1993). pedagogical content knowing: an integrative model for teacher preparation, journal of teacher education, 44, 263-272. doi: 10.1177/0022487193044004004. cochran, k. f., king, r. a. & deruiter, j. a. (1991). pedagogical content knowledge: a tentative model for teacher preparation. east lansing, mi: national centre for research on teacher learning. (eric document reproduction service no. ed340683). cohen, r. & yarden, a. (2009). experienced junior-high-school teachers’ pck in light of a curriculum change: “the cell is to be studied longitudinally”. research in science education, 39, 131-155. doi: 10.1007/s11165-008-9088-7. corbin, j. & strauss, a. (2008). basics of qualitative research: techniques and procedures for developing grounded theory (3rd ed.).thousand oaks, ca: sage. czerniak, c. & chiarelott, l. (1999). teacher education for effective science instructiona social cognitive perspective. journal of teacher education, 41(1), 49-5. doi: 10.1177/002248719004100107. demirdöğen, b., hanuscin, d. l., kondakçı-uzuntiryaki, e. & köseoğlu, f. (2015). development and nature of pre service chemistry teachers’ pedagogical content knowledge for nature of science. research science education, 46(4), 575-612. doi: 10.10007/s11165-015-9472-z dibiase, w. & mcdonald, j. r. (2015). science teacher attitudes toward inquiry-based teaching and learning. the clearing house: a journal of educational strategies, issues and ideas, 88(2), 2938.doi:10.1080/00098655.2014.987717. driel, j. h-v., jong, o.-d. & verloop, n. (2002). the development of pre service chemistry teachers’ pedagogical content knowledge. science teacher education, 86, 572-590. doi: 10.1002/sce.10010. falk, a. (2011). teachers learning from professional development in elementary science: reciprocal relations between formative assessment and pedagogical content knowledge. science education, 96, 265-290. doi: 10.1002/sce.20473. gess-newsome, j. & lederman, n. g. (eds.). (1999). explaining pedagogical content knowledge. dordrecht. the netherlands: kluwer academic. grossman, p. l. (1990). the making of a teacher: teacher knowledge and teacher education, new york: teachers college press. halim, l. & meerah, m. s. (2002). science trainee teachers’ pedagogical content knowledge and its influence on physics teaching. research in science and technological education, 20(2), 215225.doi: 10.1080/0263514022000030462. henze, i., van driel, j. & verloop, n. (2008). development of experienced science teachers’ pedagogical content knowledge of models of the solar system and the universe. international journal of science education, 30(10), 1321-1342.doi: 10.1080/09500690802187017. hope j. & townsend, m. (1983). student teachers’ understanding of science concepts. research in science education, 13, 177-183.doi: 10.1007/bf02356704. jones, a. & moreland, j. (2004). enhancing practicing primary school teachers’ pedagogical content knowledge in technology. international journal of technology and design education, 14, 121140. doi: 10.1023/b:itde.0000026513.48316.39. using an inquiry-based approach / alkış-küçükaydın&uluçınar-sağır 101 jones, a. & moreland, j. (2005). the importance of pedagogical content knowledge in assessment for learning practices: a case-study of whole-school approach. the curriculum journal, 16(2), 193-206. doi: 10.1080/09585170500136044. kamen, m. (1996). a teacher’s implementation of authentic assessment in an elementary science classroom. journal of research in science teaching, 33(8), 859-877. doi: 10.1002/(sici)10982736(199610)33:8<859::aid-tea3>3.0.co;2-v. kind, v. (2009). pedagogical content knowledge in science education: perspectives and potential for progress. studies in science education, 45(2), 169-204. doi: 10.1080/03057260903142285. kolb, s. m. (2012). grounded theory and the constant comparative method: valid research strategies for educators. journal of emerging trends on educational research and policy studies (jeteraps), 3(1), 83-86. kramarski, b. (2009). developing a pedagogical problem solving view for mathematics teachers with two reflection programs. international electronic journal of elementary education, 2(1), 137-153. lederman, n. g, gess-newsome, j. & latz, m. s. (1994). the nature and development of pre service science teachers’ conceptions of subject matter and pedagogy. journal of research in science teaching, 31, 129-146. doi: 10.1002/tea.3660310205 lee, e. & luft, j. a. (2008). experienced secondary science teachers’ representation of pedagogical content knowledge. international journal of science education, 30(10), 1343-1363. doi:10.1080/09500690802187058. lee, e., brown, m. n., luft, j. a. & roehrig, g. h. (2007). assessing beginning secondary science teachers’ pck: pilot year results. school science and mathematics, 107(2), 52-60. doi: 10.1111/j.1949-8594.2007.tb17768.x. levy, p. & petrulis, r. (2012). how do first-year university students experience inquiry and research, and what are the implications for the practice of inquiry-based learning? studies in higher education, 37(1), 85-101. doi:10.1080/03075079.2010.499166 loughran, j., berry, a. & mulhall, p. (2012). understanding and developing science teachers’ pedagogical content knowledge. rotterdam/boston: sense publishers, tape 1, volume 12. loughran, j., gustone, r., berry, a., milroy, p. & mulhall, p. (2000). documenting science teachers’ pedagogical content knowledge through pap-ers. paper presented at the annual meeting of the american educational research association. new orleans, april, 2000. magnusson, s., krajcik, j. & borko, h. (1999). nature, sources and development of pedagogical content knowledge for science teaching. in j. gess-newsome and n.g. lederman (eds.), examining pedagogical content knowledge. the construct and its implications for science education, pp. 95-132. dordrecht. kluwer. marks, r. (1990). pedagogical content knowledge: from a mathematical case to a modified conception. journal of teacher education, 41(3), 3-11. doi: 10.1177/002248719004100302 matese, g. (2005). cognitive factors affecting teaching formative assessment practice, paper presented at the american educational research association, montreal, canada. ministry of national education. (2013). elementary science curriculum of institutions. talim terbiye kurulu başkanlığı, ankara: turkey. mulhall, p., berry, a. & loughran, j. (2003). frameworks for representing science teachers’ pedagogical content knowledge. asia-pacific on science learning and teaching, 4(2), 1-25. nillsoon, p. & loughran, j. (2012). exploring the development of pre-service science elementary teachers’ pedagogical content knowledge. journal science teacher education, 23, 699-721. doi: 10.1007/s10972-011-9239-y. 102 nilsson, p. (2008).teaching for understanding: the complex nature of pedagogical content knowledge in pre-service education. international journal of science education, 30(10), 12811299.doi: 10.1080/09500690802186993. park, s. & chen, y. c. (2012). mapping out the integration of the components of pedagogical content knowledge (pck): examples from high school biology classrooms. journal of research in science teaching, 49(7), 922-941. doi: 10.1002/tea.21022. park, s. & oliver, j. s. (2008a). national board certification (nbc) as a catalyst for teacher’s learning about teaching: the effects of the nbc process on candidate teacher’s pck development. journal of research in science teaching, 45(7), 812-834. doi: 10.1002/tea.20234. park, s. & oliver, j. s. (2008b). revisiting the conceptualization of pedagogical content knowledge (pck): pck as a conceptual tool to understand teachers as professionals. research science education, 38, 261-284. doi: 10.1007/s11165-007-9049-6. qablan, a. m. & debaz, t. (2015). facilitating elementary science teachers’ implementation of inquiry-based science teaching. teacher development: an international journal of teachers' professional development, 19(1), 3-21. ramnarain, u. & schuster, d. (2014). the pedagogical orientations of south african physical sciences teachers towards inquiry or direct instructional approaches. research science education, 44, 627-650. doi: 10.1007/s11165-013-9395-5. rollnick, m., bennett, j., rhemtula, m., dharsey, n. & ndlovu, t.(2008). substantive-level theory of highly regarded secondary biology teachers’ science teaching orientations. journal of research in science teaching, 42(2), 218-244. doi: 10.1002/tea.20046. shulman, l. s. (1987). knowledge and training: foundations of the new reform. harvard educational review, 57, 1-22. smith, d. & neale, d. (1991). the construction of subject matter knowledge in primary science teaching, (in) j. brophy (ed.) advances in research on teaching, vol.2 london, jai press. smith, d. c. (1999). changing our teaching: the role of pedagogical content knowledge in elementary science. in j. gess-newsome & n. g. lederman (eds.), examining pedagogical content knowledge: the construct and its implications for science education, pp.163-197. boston: kluwer. sperandeo-mineo, r. m., fazio, c. & tarantino, g. (2005). pedagogical content knowledge development and pre-service physics teacher education: a case study. research in science education, 36(3), 235-268. doi: 10.1007/s11165-005-9004-3. tabachnick, b. r. & zeichner, k. m. (1999). idea and action: action research and the development of conceptual change teaching of science. science education, 83(2), 309-322. doi: 10.1002/(sici)1098-237x(199905)83:3<309::aid-sce3>3.0.co;2-1. tamir, p. (1988). subject matter and related pedagogical knowledge in teacher education. teaching and teacher education, 4(2), 99-110. doi:10.1016/0742-051x(88)90011-x. vandriel, j. h., verloop, n. & de vos, w. (1998). developing science teachers’ pedagogical content knowledge. journal of research in science teaching, 35, 673-695. veal, w. r. & kubasko, d. s. (2003). biology and geology teachers’ pedagogical content knowledge of evolution. journal of curriculum and supervision, 18(4), 334-352. yin, r. k. (2003). case study research: design and methods (3rd ed.). united states of america: applied social research methods series v.5. international electronic journal of elementary education, september 2016, 9(1), …-…. 103 appendix a: sample coding table. theme sub-theme code quotations personal and professional self personal self patient “i wait for the student’s answer, i am patient” professional self to care prize “i cannot say i am skilful enough in terms of knowledge” valuing students “i don’t put strain on my student[s]. i don’t think they have to learn” obstacles in learning environment physical obstacles absence of laboratory “we can’t apply everything as needed as we don’t have a laboratory” lack of technology “we don’t have projector or internet access” curriculum information positive easy access to information “i think they are learning faster and easier, so when i get feedback, i am happy.” interactive learning “students have one-on-one interaction and access information themselves” planning “i…have to plan a lesson” negative teacher’s guide “we only guide students in science lessons” assessment information traditional assessment methods using activity “we include activities but…i can also use tests” test “i use tests too often” “what is really important to me is that i understand whether the student understands the lesson” expressing in sentence behavioural action “i must make sure that the topic makes sense to the students” teaching methods and techniques traditional teaching methods and techniques lecturing “i create definitions prior to the lesson” question-and-answer “we ask and answer questions to better understand the topic” alternative methods and techniques drama “overall ,children cannot easily keep in mind so we should concretize the topic like dramas” 103 104 appendix b: primary teachers’ content representation forms. big ideas a: bones, joints, muscles and skeleton all serve a function in our body’s support and movement b: skeletal and muscle health is important c: respiratory system is made up of nose, pharynx, larynx, trachea and lungs d: blood moves through the heart and vessels in our body e: there is a connection between exercise and pulse what you expect students to learn about this idea -skeletal structure and functions -bone structure and types -functions of muscles in support and movement -types of joints and muscles -that the skeleton, joints and muscles provide body support and movement -positive and negative elements in skeleton and muscle health -what can be done for health? how to protect the skeleton and muscle -the importance of physical exercise on the skeletal system in addition to nourishment -respiratory system organs, their functions, sequences and where they are located in the body -circulatory system organs -how blood flows through the body -what does blood consist of? -functions of circulatory system organs -effects of exercise on breathing and the circulatory system -as you do more exercise your pulse rate increases -relationship between sport and leading a healthy lifestyle -that when you exercise the body expends more energy and breathing increases why is it important for students to know this? -learning body’s support and movement systems -being healthy and supporting better growth -the importance of retaining skeletal and muscle health for the future -to know how to maintain a healthy lifestyle -important in terms of learning to breathe in the correct manner -in terms of protecting themselves from illnesses -keeping away from anything that may harm respiration organs -to know that blood is vital to the body and for caring about nutrition -for knowing the functions of blood, vessels and the heart within their bodies -for learning the effect of sports on the body’s systems and making sport a way of life -to have knowledge about the relationship between exercise types and pulse (for first aid) -to learn what type of sport they can partake in and how they should conduct these activities for a healthy lifestyle what else do you know about this idea (that you do not intend students to know yet) -muscle systems, convulsion and relaxation -the relationship between exercise-pulse-breathing -respiratory system organs’ health and illnesses in case of unhealthy systems -vessel types (artery, vena and capillary) difficulties/limitations connected to teaching this idea -to be able understand muscle structure and their convulsion and relaxation -collaboration between muscles, skeleton and joints for movement -learning orders of organs -movement of blood intracorporeal from bottom to top -movement of blood through vessels using an inquiry-based approach / alkış-küçükaydın&uluçınar-sağır 105 knowledge about students’ thoughts and how this influences your teaching of this idea -knowing that the skeleton consists of bones and that it helps movement -that the body is made up of bones and muscles; being healthy -benefits of sport -what is the importance of being healthy? -breathing oxygen in and breathing carbon dioxide out -where are the lungs? -what is the shape of the trachea? -how the heart pumps blood and blood moves throughout the body -what is the colour of blood? -vessels transport blood -that pulse rate is different for children and adults -what are the types of exercise? -where can pulse be controlled from? other factors that influence your teaching of this idea -question-and-answer -discussion slides monitoring -experimenting -observation -research and practice reading, lecturing -brainstorming -question-and-answer -discussion slides monitoring -experimenting -observation -research and practice reading, lecturing -role-play question-and-answer, -discussion slides monitoring -experimenting -observation -research and practice reading, lecturing -learning through discovery question-and-answer -discussion slides monitoring -experimenting -observation -research and practice reading, lecturing -showing and making question-and-answer -discussion slides monitoring, -experimenting -observation -research and practice reading, lecturing teaching procedures (and particular reasons for using these to engage with this idea). -conduct questionnaire for older people -animation -watch cartoons -illustrate breathing on human body model showing and making -exemplify circulation by conducting an experiment -have children design their own models with cables while teaching the circulation system showing and making, -children can be given a list of sporting activities that they can do and the benefits of these activities specific ways of ascertaining students’ confusion around this idea (include likely range of responses) -laboratory method -animated cartoons -research, experiments, showing slides about topics -animated cartoons -research, experiments, showing slides about topics -animated cartoons -research, experiments, showing slides about topics -animated cartoons -research, experiments, showing slides about topics -animated cartoons -research, experiments, showing slides about topics 106 appendixc: pap-ers analysis table of primary teachers. education repertory presentation/teacher zeynep ozge serkan esra preparation before lesson yes yes incorrect information about the unit -a tooth is bone -true breathing occurs by lifting shoulders up and down -circulation occurs as a result of applying force -deoxygenated blood involves only carbon dioxide oxygenated blood involves only oxygen -blood components are air and nutrition -one of the functions of blood circulation is to assist growth -a tooth is bone -penguins do not have joints -joint movements occur with the help of the brain checking students’ prior knowledge “what is a skeleton? what do you do to stay healthy?” teaching methods and techniques -discourse -question-and-answer -demonstration -brainstorming -video -showing animations and slides -presentation -question-and-answer ask children to write summary -presentation, -question-and-answer -ask children to read a book and write a summary -presentation -question-and--answer -let children lecture -write a summary -giving research homework -simulation evaluating students’ knowledge -through test question-and-answer -through test -question-and-answer -through test -question-and-answer -through test -question-and-answer application -delivering oral presentation applied teaching methods and techniques -a doll made of play dough -skeletal model -tent -lung model -stethoscope -skeletal model made of paper -animal skull and piece of meat -skeleton made of play dough tea, chicken wings lung model respiratory system board -circulation system board unanswered student questions “how can our muscles be healthier?” students are not given an opportunity to ask questions “what is keratin?” using an inquiry-based approach / alkış-küçükaydın&uluçınar-sağır 107 given information outside of the lesson acquisition -placement of anvil, stirrup and hammer -functions of shinbone and scapula -structure of smooth and skeletal muscles -joint and bone types types of joints types of joints student expressions that cannot be corrected “we can compare blood to a bus that travels throughout a city continuously. passengers continuously get on and off the bus, just as blood always changes inside the vessels” international electronic journal of elementary education, september 2016, 9(1), …-…. 601 www.iejee.com this page is intentionally left blank 108 http://www.iejee.com/ © 2022 published by kura education & publishing. this is an open access article under the cc byncnd license. (https://creativecommons. org/licenses/by/4.0/) 213 copyright © www.iejee.com issn: 1307-9298 international electronic journal of elementary education january 2023, volume 15, issue 3, 213-223 adulthood begins in preschool: meaningful curriculum in support of increased independence for individuals with autism peter f. gerhardta,*, shanna bahryb, natalie m. driscollc, jessica cauchid, brian k. masone, madhura deshpandef abstract introduction a meaningful curriculum is one that is individualized, inclusive of the individual’s unique needs and interests, and focused on building independence in current and future environments. a meaningful curriculum addresses an individual’s needs and prioritizes instructional programs based on what is, potentially, most important to their lives and not simply a somewhat arbitrary list of isolated skills. a meaningful curriculum is one that puts as much emphasis on skill acquisition outside of the classroom as it does on skill acquisition within the classroom or school. unfortunately, the use of meaningful curricula to educate autistic students would seem to be something of a rarity, which may help to explain the consistently poor outcomes that individuals with autism and related disorders experience in adulthood. in this article, the authors make recommendations that may help practitioners to mitigate these outcomes by providing instruction in curricula that place a focus on adaptive behavior skills, the intersection of the individual, respectful intervention, and an emphasis on these important topics beginning in preschool, and increasing in importance and complexity across the lifespan. overall, individuals with an autism spectrum disorder (asd) have poor adult outcomes when compared to same-aged peers in every area typically evaluated including employment, living arrangements, social and community participation, access to services, physical and/or mental health, and safety (roux et al., 2015). such outcomes have shown little improvement over time (newman et al., 2010) and are worse than those of adults with other disability labels or identities (e.g., intellectual disability [id], learning disability [ld], speech-language impairment, and emotional disturbance; roux et al., 2015). on an annual basis, approximately 70,000+ autistic teens in the united states become adults (autism speaks, n.d.) but lack the necessary skills to successfully transition to an independent life after high school (gerhardt & lainer, 2011). on a macro-level, poor outcomes result in higher financial keywords: autism, autism spectrum disorder, adolescence, adulthood, transition, curriculum, assessment, quality of life received : 20 december 2022 revised : 23 january 2023 accepted : 8 march 2023 doi : 10.26822/iejee.2023.294 a,* corresponding author: peter f. gerhardt, endicott college, beverly, ma, the epic school, paramus, nj, usa. e-mail: pgerhardt@epicschool.org orcid: https://orcid.org/0000-0003-1234-1753 b shanna bahry, endicott college, beverly, ma, meaningful hope, east providence, ri, usa. e-mail: sbahry@endicott.edu orcid: https://orcid.org/0000-0002-4347-018x c natalie m. driscoll, endicott college, beverly, ma, seven hills foundation, worcester, ma, usa. e-mail: ndriscol@endicott.edu orcid: https://orcid.org/0000-0003-3261-0172 d jessica cauchi, endicott college, beverly, ma, usa. atlas behaviour consultation, oakville, on, canada. e-mail: jcauchi@endicott.edu orcid: https://orcid.org/0000-0001-6084-1846 e brian k. mason, endicott college, beverly, ma, usa. hamilton-wentworth district school board, hamilton, on, canada. e-mail: bmason@endicott.edu orcid: https://orcid.org/0009-0000-4110-6133 f madhura deshpande, endicott college, beverly, ma, usa. momentum learning services, sydney, nsw, australia. e-mail: mdeshpande@mail.endicott.edu orcid: https://orcid.org/0009-0001-7163-4401 214 january 2023, volume 15, issue 3, 213-223 costs to the family and to society at large (dell’armo & tasse, 2018; farley et al., 2018; howlin & magiati, 2017). upon leaving high school, autistic youth experience a drop in available services and supports that is generally referred to as a service cliff (roux et al., 2015). while federal law (i.e., individuals with disabilities act; idea, 2004) mandates the provision of special education inclusive of a plan to transition from school to the world of adult services, there is no similar obligation post-high school; this leaves families to fend for themselves when seeking appropriate services (roux et al., 2015; shattuck et al., 2020). sadly, there is a backlog of service needs that easily exceeds the resources currently available (gerhardt & lainer, 2011). over the next decade, autism speaks (n.d.) estimates that an additional 707,000-1,116,000 teens in the us will enter the world of adulthood, only further exacerbating this problem. to address this problem, the role that parents, teachers, aides, behavior analysts, and related service providers need to play becomes nothing short of critical leading up to, and during, the transition to adulthood. while every student's transition plan should be individualized, every plan should provide attention to three major outcomes including employment, independent living, and happiness (e.g., ticani & bondy, 2014). unfortunately, it seems that a combination of poor preparation leading up to adulthood and limited access to effective, individualized adult services, hinders progression toward such positive outcomes (shattuck et al., 2020; snell-rood et al., 2020). despite the fact that individuals on the spectrum can, and often do, make valuable contributions to the communities in which they live, they are often denied access to this opportunity. providing a meaningful curriculum is one way to reduce such barriers and increase meaningful access. what makes a curriculum meaningful? a meaningful curriculum is one that is individualized, inclusive of the individual’s unique needs and interests, and focused on building independence in current and future environments (ayres et al., 2011). a meaningful curriculum addresses an individual’s needs and prioritizes instructional programs based on what is, potentially, most important to their lives. as positive adult outcomes can be predicted, in part, based on the adaptive behavior repertoire of an individual (ayres et al., 2011; dell’armo & tassé, 2018; mazefsky et al., 2008; mazzotti et al., 2016), instruction in adaptive skills lies at the core of any meaningful curriculum. adaptive behavior is, essentially, any and all skills and abilities that allow independent functioning in their day-to-day life in the environment in which they live (heward, 2005). adaptive skills are “so central to adult life that it would not be an understatement to say that good adaptive behavior skills will get a person through times of no academic skills better than good academic skills will get a person through times of no adaptive behavior” (gerhardt et al., 2013, p. 167). adaptive behavior should therefore be a priority in programming starting at a young age and increase in focus as individuals get closer to adulthood. what is meaningful for some will not be meaningful for all and, as such, practitioners need to take the unique needs of each individual into consideration when confronted with general education standards. strict adherence to the general education standards requires practitioners to target goals that will likely have little to no relevance to the adult lives of individuals. for example, learning to identify saturn when you have not yet learned to brush your teeth would probably not be considered a meaningful goal for many (ayres et al., 2011). there is a balance to goal development, and while academic skills may be part of a meaningful curriculum for some individuals, others may need to focus more on adaptive behavior competencies instead (although even the more “academic” student still needs to acquire meaningful adaptive behavior skills). in addition to academic skills and skills specific to the autism spectrum disorder (asd) diagnostic criteria, other important adaptive skill domains include but are not limited to, self-care/hygiene, dressing, health maintenance, safety, cleaning/care of the home, cleaning/care of clothing, social skills and niceties, mealtime skills, community engagement, self-management, and leisure/recreation. a truly individualized and meaningful curriculum fulfills each individual’s right to an appropriate education (ayres et al., 2011; ayres et al., 2012; bahry et al., 2022b). a meaningful curriculum should promote a balance between teaching high-value (high preference/ high relevance) and low-value (low preference/high relevance) skills. a skill that is highly valued is more likely to be maintained over time, so should be prioritized in a meaningful curriculum (gerhardt et al., 2013). however, low-value skills might also be important if they promote engagement, safety, or enjoyment in an indirect way. for example, showering may be a lowvalue skill to an individual, but mastery of the skill may lead to decreased body odor and increased neat appearance which may help promote engagement in social situations or the ability to get and retain a job (which both may be of high value). at the end of the day, applied skills are simply useful skills. if the skill is not valued, it will not be used; if the skill is not used often enough, the individual will likely, over time, lose their ability to display the skill. a meaningful curriculum can be conceptualized as an antecedent-based behavioral intervention. when 215 adulthood begins in preschool / gerhardt, bahry, driscoll, cauchi, mason & deshpande caregivers and practitioners proactively set goals for young individuals with autism and other related disorders that are meaningful, the future risk of severe challenging behavior, abuse, and harm is reduced (e.g., ala'i-rosales 2019). gaps in adaptive skills can be identified and targeted within the context of a meaningful curriculum, which can help to lower dependency on others across the lifespan and thus help to mitigate the risks discussed (saulnier & klaiman, 2022). meaningful intervention and challenging behavior a high prevalence of challenging behavior exists within the asd population, specifically with such dangerous behavior as self-injury and aggression (e.g., davies & oliver, 2016). often these behaviors develop as a result of skill deficits in other areas, such as communication. in a large population-based sample in the united states, self-injurious behavior (sib) was reported to occur in more than 30% of children with asd (soke et al., 2016). another large sample study found aggression to be reported in approximately 1 in 4 individuals with asd, with a significant association found between aggression and overall cognitive level (i.e., iq; hill et al., 2014). the display of challenging behavior can, and often does, restrict the activity of adolescents and adults on the spectrum. community-based instruction (cbi) often gets put on hold, as do job training opportunities due to the presence of challenging behavior. as such, a meaningful curriculum needs to adequately address these challenges to allow for increased skill acquisition in other important areas. despite having over 60 years of behavior analytic research focusing on the assessment and intervention of challenging behavior (ala'i-rosales et al., 2019), the vast majority of this research has been conducted in well-controlled research environments or clinics and not in a more typical environment (e.g., the classroom or the community). the complexity of autism education beyond the controlled environment (i.e., the real world) does not easily lend itself to wellcontrolled study conditions. within the context of a meaningful curriculum, intervention with challenging behavior requires: 1. reductions in aggression or self-injury that result in positive changes to the individual’s life reductions in the frequency of a particular behavior that does not result in increased community access or employment training opportunities for adolescents or young adults represents only part of the desired outcome. 2. the development of a set skills that enables the individual to manage their own behavior and control relevant aspects of their environment this could include, antecedent strategies such as behavioral relaxation/self-calming, functional communication training and support, context-appropriate refusal skills, the ability to accept delayed reinforcement, curricular modifications congruent with the individual’s preferences, and environmental modifications that support greater access to positive reinforcement. 3. a data-based justification that the behavior in question has a negative impact on the individual’s quality of life or wellbeing all of us engage in stereotypic behavior. the difference lies in our ability to recognize the social contingencies relevant to stereotypy and differentiate between “time in” conditions (when we can engage in stereotypy) and “time out” conditions (when we cannot engage in stereotypy). any intervention targeting stereotypy needs to focus on teaching the individual how to recognize and respond to the two conditions. 4. recognition of each individual’s right to be angry, irritated, or annoyed this is only a problem when anger, irritation, or annoyance result in aggression or selfinjury. being pissed off is not a challenging behavior. intersection of the individual as previously discussed, a central feature of a meaningful goal is that it presents as meaningful to the student in question. in other words, meaningful goals require a much greater degree of individualization than do non-meaningful goals. individualization of goals requires an ongoing evaluation of the intersection of the person and the environment, considering such parameters as personal preferences, dislikes, interests, and idiosyncrasies. when providing instruction either inside or outside of the classroom an understanding of the intersection between the person, their skill repertoire, and the requirement of the environment becomes even more critical. for example, a student may be more willing to independently apply deodorant if they personally pick the form (e.g., spray, roll-on) of the deodorant and its scent. tthis is an example of programming at the intersection of the individual; how deodorant feels or smells to them when applied can turn a meaningful but non-preferred skill into a meaningful, preferred skill. as an educator or other professional, this should not be an alien concept as you practice it every day in your own life. for example, when you go to check out at the supermarket, there exists an intersection between your skill set and the environment: do you use self-checkout or go to a cashier lane? if you have only a few items, or you feel comfortable with self-checkout you could go there. similarly, if you do not like waiting in line at the checkout, you may choose to do your shopping before going to work when there are fewer people at the supermarket. either option represents a personal response to the intersection of the environment, a person's skill repertoire, and personal preferences. 216 january 2023, volume 15, issue 3, 213-223 for students with autism, particularly those with a limited skill set indicating preferences, the relevant intersections may be difficult to identify. in terms of meaningful curriculum, however, this only serves to further highlight the importance of individually determined cbi starting early on. for example, a goal to teach a person to respond with their current location when texted by a known person has value in terms of a personal safety goal, but initially may have limited meaningfulness to the individual. if that same individual loves chocolate cake, beginning instruction by sending them texts telling them where to find a piece of chocolate cake may make the process of “responding to a text” more valuable to the individual and, therefore, more meaningful. this systematic process is not easily accomplished, however, if the skill is to maintain, it has to acquire some personal meaning. the simple truth is that if instruction in meaningful goals does not happen when an individual is in school, it is unlikely to ever happen, given the highly limited availability of supports and services in adulthood (shattuck et al., 2020). goals must be socially valid to the student, support team, and the immediate community in order for instruction to be successful (schwartz & baer, 1991). social validity, as outlined by wolf (1978), is the degree to which programming is acceptable to the student and to others. ideally, society should be validating programming at every step of the planning process, including the goals selected, the procedures used, and the outcome of programming (stokes & baer, 1977; wolf, 1978). the regular assessment of social validity can, and should, include the individual receiving services to whatever extent possible, as well as parents, caregivers, and other community members (stokes & baer, 1977). meaningful programming that has been created can be validated using a number of already established measurement tools (e.g., bernstein, 1989; fawcett, 1991; gresham & lopez, 1996; kazdin, 1980), with adjustments being made to programs as needed based on their results. a recommended beginning to identifying social validity early on is to use person-centered planning. person-centered planning person-centered planning is a process designed to allow individuals with disabilities to participate more actively and directly in their transition planning including attention to the person’s preferences for a life they would consider to be meaningful based on their individual strengths, abilities, aspirations, and preferences (collings et al., 2019). person-centered planning has been found to promote more positive adult outcomes for people with various disabilities labels (robertson et al., 2007). autistic youth, however, can experience difficulties in participating due to communication and social deficits (hagner et al., 2014). as such, embedding instruction in selfdetermination skills within a curriculum is an important component of individualized, person-centered intervention, especially because these skills often do not develop without specific instruction for individuals with disabilities (stancliffe et al., 2000; wehmeyer et al., 1996). as wehmeyer and abery (2013) stated, “selfdetermined people are, in essence, actors in their own lives, rather than being acted upon” (p. 399). such individuals make their own decisions, set their own goals, and create plans to meet these goals (martin et al., 2019). the critical skill of self-determination has been shown to improve the likelihood of meaningful outcomes in adulthood, specifically in the areas of employment, postsecondary education, and independent living (field et al., 1998; lachapelle et al., 2005; powers et al., 2012; wehmeyer & schwartz, 1997). self-determined behavior, including making personally relevant choices, has also been identified as a core dimension of quality-of-life and happiness (shalock & verdugo, 2012, wehmeyer & abery, 2013). any meaningful curriculum should therefore prioritize communication skills instruction while concurrently assessing various indices of happiness (e.g., smiling, laughing, yelling while smiling) as well as unhappiness (e.g., frowning, crying, yelling without smiling) to determine and honor preferences (dillon & carr, 2007; green & reid, 1996). techniques rooted in the principles of applied behavior analysis (aba) have demonstrated our ability to clearly define and systematically increase, a number of indices of happiness even for individuals with profound multiple disabilities and limited to no communication skills (dillon & carr, 2007; green et al., 1997; green & reid, 1996; ivancic et al., 1997; lancioni et al., 2002). the fact that data support that practitioners can identify and promote the acquisition of behaviors associated with improved quality of life and happiness, supports an argument in favor of meaningful curriculum and intervention on a practical and professionally ethical basis. how to teach: effective interventions for meaningful goals instruction in meaningful curricula is provided in the environment in which the skill is most likely to be used (gerhardt et al., 2013). skills targeted for acquisition that appear to be meaningful, lose that designation when taught outside of the relevant context (brown et al., 1976). practitioners cannot solely teach skills in classrooms or school hallways and hope for the best. for example, teaching an individual to cross the street while in the classroom is unlikely to generalize to the community without additional instruction in the community. as individuals get older, the proportion of instruction delivered in schools and clinics should be reduced and replaced to the greatest extent 217 adulthood begins in preschool / gerhardt, bahry, driscoll, cauchi, mason & deshpande possible with instruction in the context in which the skill will ultimately occur (bahry et al., 2022a; gerhardt & bahry et al., 2022; gerhardt et al., 2013). long-term planning is crucial because individuals will only be in the classroom for a finite number of years, but will be out in the real world for the rest of their lives. because adaptive behavior skills, in general, tend to be complex skills, they may require potentially complex intervention. in those cases, interventions based on the principles of aba (baer et al., 1968) represent a set of evidence-based strategies with documented effectiveness in promoting the acquisition of adaptive behavior skills. these interventions include, but are not limited to, modeling, chaining, shaping, differential reinforcement, token economies, behavioral momentum, self-management, and functional communication training (fct; gerhardt & lainer, 2011). respectful intervention and meaningful curriculum a meaningful curriculum is an essential component of a future life of dignity and respect. as brown and colleagues (1979) noted, “the more functional skills individuals with disabilities have in their repertoires, the more they can do for themselves, the more privacy, choice and dignity they have and the fewer social, emotional, financially and other pressures they place upon others” (p. 4). continuously evaluating goals to promote the most independence possible by imposing the fewest restrictions necessary to maintain safety is, therefore, recommended. winett and winkler (1972) warned against, “a rigid preoccupation with order and control and where children are required to be still, to be silent, and to obey” (p. 499). this warning is of increasingly greater importance as individuals age across the school years. individuals have a right to instruction and support in exercising these freedoms, such as the right to choose and refuse and to make decisions about their goals and instruction (bannerman et al., 1990). respectful intervention also includes ensuring that practitioner behavior, including the language used during instruction, promotes dignity. the way practitioners speak and behave toward the individual they support is of central importance in the instructional process. recommendations include using language that the individual and/or their family prefers (e.g., person-first vs. diagnosis-first language), speaking directly to the individual when in front of them instead of about them, avoiding pejorative language (e.g., “low-functioning”), and speaking about people in a normative way (e.g., using respectful language like ms./mr. if it is customary in a given context; reid et al., 2017). the form of respectful intervention needs to continually evolve to reflect the age of the student or client. providing intervention that is respectful of the learner also includes obtaining assent to treatment. while assent is considered an ethical obligation of many professionals including behavior analysts (bacb, 2022), as well as within medicine (olszewski & goldkind, 2018; bakić-mirić & bakić, 2008) it is typically not well studied and difficult to define. assent may be defined using terms like therapeutic alliance (e.g., goldiamond, 1974) or rapport, but however defined, assent should include both vocal-verbal measures as well as behavioral indicators of agreement to intervention. while there is much work to be done in the development of methods of measurement of assent (morris, 2021), as well as recommendations regarding how often to obtain assent and how to manage dissent in required intervention, the increased focus on, and discussion about assent in intervention is important and encouraging. dignity of short-term and long-term risk within the context of a meaningful curriculum, individuals on the spectrum should be afforded what is known as the dignity of risk. appropriate and reasonable risk-taking can and should be incorporated into a meaningful curriculum. for example, an individual will not acquire the skills necessary to independently navigate a mall if the instructional opportunity is never provided under the guise of personal safety or the absence of any advocacy for the skill to be taught. for some individuals with disabilities, concerns about balancing risk and benefit may be minimized when staff are well-trained in the assessment process (e.g., driscoll et al., 2022). however, not providing instruction in meal preparation due to the possibility of the person burning themselves on the stove fails to take into account the frequency that neurotypical peers burn themselves or their food during cooking. withholding access to risk is a way of infantilizing individuals with disabilities (perske, 1972). as individuals move through school and, eventually, enter adulthood, practitioners need to include instruction in the competencies associated with managing dignity of risk to whatever extent possible while minimizing the possibility of harm or trauma. clinical judgment (e.g., leaf et al., 2019; leaf, et al, 2016) including an in-depth understanding of the individual’s learning history, family concerns and preferences, and individual preferences should be used as a guide. currently, the majority of adults on the autism spectrum remain dependent on their families and/ or providers for the management of their hygiene, medical care, finances, home, and living, and daily living tasks (e.g., howlin & magiati, 2017; roux et al., 2015; shattuck et al., 2020). adults with disabilities who live in staffed residences and attend day programs have less emphasis placed on highly qualified staffing support (lowe et al., 1998; gerber et al., 2011; smidt et al., 2007; 218 january 2023, volume 15, issue 3, 213-223 gormley et al., 2019) and some studies have shown that psychotropic drugs are frequently and excessively prescribed to manage challenging behaviors of adults diagnosed with autism with comorbidity of intellectual disabilities in residential settings (robertson et al.2000; deb et al., 2015; bowring et al., 2017; lim et al., 2021; felce et al., 2011). unfortunately, adults with disabilities are not often making decisions for themselves due to their perceived capacity, which is described by mcswiggen and colleagues (2016) as a judgment about the intellectual, decisional, effective, and practical skills to make a particular decision for a person’s life, health, and wellbeing. when capacity is determined to be insufficient, a legal guardian is often tasked with making decisions on the behalf of the individual (dudley & goins, 2003). increasing independence in skill areas such as communication, safety, medical interventions, hygiene, meal planning and preparation, budgeting, and other useful skill domains can help support individuals to lessen or completely avoid dependency on others to make decisions for them in adulthood (i.e., build their capacity to make decisions themselves). in building these skills, individuals in supported environments can begin to take back independence and subsequently reduce relinquishment of decisionmaking to guardians, residential care providers, staff in workspaces and/or day programs, and even family members. systems-based approach to meaningful programming creating meaningful programming requires a sometimes significant shift in the current practices of a given learning environment. significant changes in organizational settings often require a systems-based approach for success (e.g., standen, et al, 2020). in order to develop a process that can support this, intentional moves generally need to be made at a myriad of levels of a school or school district (i.e., classroom, teacher, administration) the administration level training on meaningful curriculum, including the practices discussed in this paper, should focus on school principals and/ or executive directors. gaining buy-in from directorlevel professionals is critical to promote a top-down implementation process. these strategies can help to develop a positive and inclusive school culture that prioritizes long-term outcomes. the teacher level may be one of the most important pieces, given that this is the level where goals are written. coaching and feedback can help to improve and shape behavior in teaching practices (gavoni & weatherly, 2019), which reasonably could include practices related to a focus on meaningful programming. the educational team supporting the teacher needs to play a role in identifying meaningful goals and the development of systems for data-based decision-making. at the classroom/direct-care level, there are many practitioner (e.g., assistant teachers, paraprofessionals) behaviors that contribute to the practice of meaningful programming. when instructors have the perception that they are effective in their teaching practice, this has been shown to have a large effect size related to positive outcomes for students (hattie, 2013). instilling the perspective that programming developed is going to meaningfully affect individuals’ lives long-term can be a helpful addition to this perception. summary the goal of any meaningful curriculum is to increase the student’s personal independence across multiple environments. personal independence can be defined as the degree of congruence (or match) between the skills in an individual’s repertoire and the social, communication, social, safety, and mobility demands of the environment in which they live, work, or play. typically developing individuals acquire these skills through a combination of in vivo modeling, shaping, chaining, trial and error, etc. some more “specialized” skills (e.g., riding a bike, playing an instrument, doing the backstroke) may require a degree of formalized instruction combined with ongoing practice. for most people with autism, however, that naturalistic combination of processes is generally insufficient to acquire even basic skills. research (e.g., dell’armo & tasse, 2019) indicates that individualized instruction in skills collectively referred to as adaptive behavior is associated with more positive outcomes in adulthood. adaptive behavior, however, is an incredibly complex and diverse array of skills and behavioral competencies, the boundaries of which are defined by the intersection of age in years and the physical, geographic, socio-cultural, economic, personal, and health-related demands of the environment. this is at the core of meaningful curriculum and respectful intervention. various aspects of a meaningful curriculum and respectful intervention have been laid out in some detail earlier in this paper. however, there is a meaningful curriculum “hack” that has not been discussed, and that is the use of the phrase “in order to.” when writing instructional goals or objectives end the goal with “in order to” and then complete the sentence. for example, jeremiah is a fictional 13-year-old who lives in a rural community. jeremiah has co-occurring diagnoses of autism and an id. an instructional goal for may be “jeremiah will independently ride a 2-wheeled bicycle for a minimum of 60-minutes.” as that stands it would appear to be a very time-intensive instructional 219 adulthood begins in preschool / gerhardt, bahry, driscoll, cauchi, mason & deshpande goal. however, if we add a completed “in order to,” we end up with, “jeremiah will independently ride a 2-wheeled bicycle for a minimum of 60-minutes in order to join his family on long bike rides and promote a healthy lifestyle as he ages.” this addition explains in plain language why this goal could be considered meaningful to jeremiah. on the other hand, if jeremiah had the goal, “jeremiah will balance his checkbook using the bank provided ledger and a calculator in order to…” in order to what? to do something in the most difficult way possible when using an app or downloading an account statement online would provide the same outcome much more quickly and easily? just as importantly, assume jeremiah masters the calculator goal, where does the skill go next? does he report a negative balance to a parent? does he advocate at the bank to correct their error? or, maybe, he then transfers money from one account to another to cover the deficit? in any case, the addition of “in order to” works to assist parents, teachers, and other educational professionals to identify initial goals and propose the long-term application of the goal in a meaningful way. as was stated earlier, this is presented as a hack and not a fool-proof method. neither should it be considered a substitute to the recommendations provided earlier in this paper. effective intervention requires the merging of evidence-based practice and intervention with individualized, meaningful curriculum applied in the right context. while the concept of meaningful curriculum has significant face validity (i.e., it makes sense), the research base in support of this approach remains at an early stage. however, until that research is available, it is probably worth repeating that “teaching the wrong skills well is no better than teaching the right skills poorly.” in general, the field of education and related services is good at increasing skill proficiency. now we need to be proficient with targeting the right skills. footnotes 1a note about terminology: throughout this paper, the terms “autism,” “on the autism spectrum,” “asd,” “person with autism,” and “autistic person” are used interchangeably. while the authors recognize that amongst the clinical, medical, and neurodiverse community there are preferences and conventions in terminology use, the selection of terminology in this paper is based on grammar and stylistic needs and does not reflect a particular terminological intent. references ala’i-rosales, s., cihon, j. h., currier, t., ferguson, j. l., leaf, j. b., leaf, r., mceachin, j., & weinkauf, s. m. (2018). the big four: functional assessment research informs preventative behavior analysis. behavior analysis in practice, 12(1), 222– 234. https://doi.org10.1007/s40617-018-00291-9 autism speaks. (n.d.) autism statistics and facts. https:// www.autismspeaks.org/autism-statistics-asd. ayres, k. m., douglas, k. h., lowrey, k. a., sievers, c. (2011). i can identify saturn but i can’t brush my teeth: what happens when the curricular focus for students with severe disabilities shifts. education and training in autism and developmental disabilities, 46(1), 11-21. ayres, k. m., douglas, k. h., lowrey, k. a., sievers, c. (2012). the question still remains: what happens when the curricular focus for students with severe disabilities shifts? a reply to courtade, spooner, browder, and jimenez (2012). education and training in autism and developmental disabilities, 46(1), 14-22. baer, d. m., wolf, m. m., & risley, t. r. (1968). some current dimensions of applied behavior analysis. journal of applied behavior analysis, 1(1), 91–97. https://doi.org/10.1901/jaba.1968.1-91 bahry, s. n., cauchi, j., driscoll, n.m., & gerhardt, p. f., (2022a). meaningful curriculum and functional intervention for adults with autism. in j. b. leaf, j. h. cihon, j. l. ferguson, & p. f. gerhardt (eds.), handbook of quality of life for individuals with autism spectrum disorder (pp. 313 332). springer. bahry, s., gerhardt, p.f., weiss, m.j. leaf, j.b., putnam, r.f., & bondy, a. (2022b). the ethics of actually helping people: targeting skill acquisition goals that promote meaningful outcomes for individuals with autism spectrum disorder. behavior analysis in practice, advanced online publication. https://doi.org/10.1007/s40617-02200757-x bakić-mirić, n. m., & bakić, n. m. (2008). successful doctor-patient communication and rapport building as the key skills of medical practice. facta univers, 15(2), 74-9 bannerman, d. j., sheldon, j. b., sherman, j. a., & harchik, a. (1990). balancing the right to habilitation with the right to personal liberties: the rights of people with developmental disabilities to eat too many doughnuts and take a nap. journal of applied behavior analysis, 23(1), 79-89. https:// doi.org/10.1901/jaba.1990.23-79 220 january 2023, volume 15, issue 3, 213-223 behavior analyst certification board. (2020). ethics code for behavior analysts. https://bacb. com/wp-content/ethics-code-for-behavioranalysts/ bernstein g. s. (1989). social validity and the report of the aba task force on right to effective treatment. the behavior analyst, 12(1), 97. https:// doi.org/10.1007/bf03392484 bowring, d. l., totsika, v., hastings, r. p., toogood, s., and mcmahon, m. (2017) prevalence of psychotropic medication use and association with challenging behaviour in adults with an intellectual disability. a total population study. journal of intellectual disability research, 61: 604– 617. doi: 10.1111/jir.12359. brown, l., nietupski, j., & hamre-nietupski, s. (1976). criterion of ultimate functioning. in thomas, m.a. (ed.), hey, don’t forget about me: education’s investment in the severely, profoundly, and multiply handicapped (pp. 2-15). the council for exceptional children. retrieved from https:// mn.gov/mnddc/parallels2/pdf/70s/76/76-hdfcec.pdf brown, l., mclean, b., nietupski, s. h., pumpian, i., certo, n., & gruenewald, l. (1979). a strategy for developing chronological age appropriate and functional curricular content for adolescents and young adults with significant disabilities. journal of special education, 13(1), 81-90. https:// doi.org/10.1177/002246697901300113 collings, s., dew, a., dowse, l. (2019) ’they need to have walked in our shoes’: what people with intellectual disability say about national disability insurance scheme planning. journal of intellectual and developmental disability, 44(1), 1-12. https://doi.org/10.3109/13668250.2017.1 287887 deb, s., unwin, g. and deb, t. (2015), trajectory of psychotropic prescribing in adults with intellectual disability. journal of intellectual disability research, 59 (1), 11-25. https://doi. org/10.1111/jir.12119 dell’armo k. a., & tassé m. j. (2018). the role of adaptive behavior and parent expectations in predicting post-school outcomes for young adults with intellectual disability. journal of autism developmental disorders, 49(4), 1638-1651. https://doi.org/10.1007/s10803-018-3857-6 dillon, c. m., & carr, j. e. (2007). assessing indices of happiness and unhappiness in individuals with developmental disabilities: a review. behavioral interventions, 22(3), 229-244. https:// doi.org/10.1002/bin.240 driscoll, n. m., rothschild, a. w., luiselli, j. k., goldberg, s., crawley, j., fofanah, d., & wangaga, j. (2022). brief report: a pilot investigation of safety concerns among direct service providers for adults with intellectual and developmental disabilities. journal of developmental and physical disabilities, 1-8. dudley, kenneth & goins, r. (2003). guardianship capacity evaluations of older adults. journal of aging and social policy, 15(1), 97-115. 10.1300/ j031v15n01_06. farley, m., cottle, k. j., bilder, d., viskochil, j., coon, h., & mcmahon, w. (2018). mid-life social outcomes for a population-based sample of adults with asd. autism resources, 11(1), 142-152. https://doi. org/10.1002/aur.1897 fawcett s. b. (1991). social validity: a note on methodology. journal of applied behavior analysis, 24(2), 235–239. https://doi.org/10.1901/ jaba.1991.24-235 felce, d., perry, j., lowe, k. and jones, e. (2011), the impact of autism or severe challenging behaviour on lifestyle outcome in community housing. journal of applied research in intellectual disabilities, 24(2), 95-104. https://doi. org/10.1111/j.1468-3148.2010.00571.x field, s., martin, j., miller, r., ward, m., & wehmeyer, m. (1998). self-determination for persons with disabilities: a position statement of the division on career development and transition. career development for exceptional individuals, 21(2), 113-128. https://doi. org/10.1177/088572889802100202 gavoni, p., & weatherly, n. (2019). deliberate coaching: a toolbox for accelerating teacher performance. learning sciences international. gormley, l., healy, o., o’sullivan, b., o’regan, d., grey, i., and bracken, m. (2019) the impact of behavioural skills training on the knowledge, skills and well-being of front line staff in the intellectual disability sector: a clustered randomised control trial. journal of intellectual disability research, 63: 1291– 1304. https://doi. org/10.1111/jir.12630. gerber, f., bessero, s., robbiani, b., courvoisier, d.s., baud, m.a., traoré, m.c., blanco, p., giroud, m. and galli carminati, g. (2011), comparing residential programmes for adults with autism spectrum disorders and intellectual disability: outcomes of challenging behaviour and quality of life. journal of intellectual disability research, 55: 918-932. https://doi.org/10.1111/ j.1365-2788.2011.01455.x 221 adulthood begins in preschool / gerhardt, bahry, driscoll, cauchi, mason & deshpande gerhardt, p. f., & bahry, s. n. (2022). neglected aspects of transition planning and implementation for autistic adolescents and young adults. in j. b. leaf, j. h. cihon, j. l. ferguson, & p. f. gerhardt (eds.), handbook of quality of life for individuals with autism spectrum disorder (pp. 107 120). springer. gerhardt, p.f., & lainer, i. (2011). addressing the needs of adolescents and adults with autism: a crisis on the horizon. journal of contemporary psychotherapy, 41, 37-45. https://doi.org/10.1007/ s10879-010-9160-2 gerhardt, p. f., zawacki, j., & satriale, g. (2013). adaptive behavior in adolescents and adults with autism spectrum disorders. in p.f. gerhardt & d. crimmins (eds.), social skill and adaptive behavior intervention with learners with autism (pp. 159-170). brookes publishing. green, c. w., gardner, s .m., & reid, d. h. (1997). increasing indices of happiness among people with profound multiple disabilities: a program replication and component analysis. journal of applied behavior analysis, 30(2), 217-228. https:// doi.org/10.1901/jaba.1997.30-217 green, c. w., & reid, d. h. (1996). defining, validating, and increasing indices of happiness among people with profound multiple disabilities. journal of applied behavior analysis, 29(1), 67-78. https://doi.org/10.1901/jaba.1997.30-217 gresham, f. m., & lopez, m. f. (1996). social validation: a unifying construct for school-based consultation research and practice. school psychology quarterly, 11, 204–227. https://doi. org/10.1037/h0088930 hagner, d., kurtz, a., may, j., & cloutier, h. (2014). person-centered planning for transition-aged youth with autism spectrum disorders. journal of rehabilitation, 80(1), 4-10. hattie, j. (2013). visible learning for teachers: maximizing impact on learning. routledge: ny heward, w. i. (2005). exceptional children: an introduction to special education. prentice hall. hill, a. p., zuckerman, k. e., hagen, a. d., kriz, d. j., duvall, s. w., van santen, j., nigg, j., fair, d., & fombonne, e. (2014). aggressive behavior problems in children with autism spectrum disorders: prevalence and correlates in a large clinical sample. research in autism spectrum disorders, 8(9), 1121–1133. https://doi.org/10.1016/j. rasd.2014.05.006 howlin p, & magiati, i. (2017). autism spectrum disorder: outcomes in adulthood. current opinion in psychiatry, 30(2):69-76. https://doi.org/10.1097/ yco.0000000000000308 individuals with disabilities education act (idea), 20 u.s.c. § 1400 (2004). https://sites.ed.gov/idea/ ivancic, m.t., barrett, g.t., simonow, a., & kimberly, a. (1997). a replication to increase happiness indices among some people with profound multiple disabilities. research in developmental disabilities, 18(1), 79-89. https://doi.org/10.1016/ s0891-4222(96)00039-x kazdin, a. e. (1980). acceptability of alternative treatments for deviant child behavior. journal of applied behavior analysis, 13, 259–273. https:// doi.org/10.1901/jaba.1980.13-259 lachapelle, y., wehmeyer, m. l, haelwyck, m.c., courbois, y., keith, k. d., schalock, r., & walsh, p. n. (2005). the relationship between quality of life and self-determination: an international study. journal of intellectual disability research, 49(10), 740-744. https://doi.org/10.1111/j.13652788.2005.00743.x lancioni, g. e., o’reilly, m. f., campodonioc, f., & mantini, m. (2002). increasing indices of happiness and positive engagement in persons with profound multiple disabilities. journal of developmental and physical disabilities, 14, 231-237. https://doi. org/10.1023/a:1016020320752 leaf, j. b., leaf, r., mceachin, j., taubman, m., ala'irosales, s., ross, r. k., smith, t., & weiss, m. j. (2016). applied behavior analysis is a science and, therefore, progressive. journal of autism and developmental disorders, 46(2), 720 731. https://doi.org/10.1007/s10803-015-2591-6 leaf, r., leaf, j. b., & mceachin, j. (2019). clinical judgment in aba: lessons from our pioneers. different roads. lim, r., moffat, a.k., young, r. and kalisch ellett, l.m. (2021), use of medicines in adults with autism spectrum disorder in australia. journal of pharmacy practice and research, 51(5), 410414. https://doi.org/10.1002/jppr.1745 lowe, k., felce, d., perry, j., baxter, h., & jones, e. (1998). the characteristics and residential situations of people with severe intellectual disability and the most severe challenging behaviour in wales. journal of intellectual disability research, 42(5), 375–389. https://doi.org/10.1046/ j.1365-2788.1998.00154.x 222 january 2023, volume 15, issue 3, 213-223 martin, j. e., pulos, j. m., & sale, p. (2019). assessing and teaching critical self-determination skills to transition-age youth with disabilities. in p. wehman & j. kregel (eds.), functional curriculum for elementary and secondary students with special needs (4th ed.), pp. 53-74. pro-ed, inc. mazefsky, c. a., williams, d. l., & minshew, n. j. (2008). variability in adaptive behavior in autism: evidence for the importance of family history. journal of abnormal child psychology, 36(4), 591-599. https://doi.org/10.1007/s10802-0079202-8 mazzotti, v. l., rowe, d. a., sinclair, j., poppen, m., woods, w. e., & shearer, m. l. (2016). predictors of postschool success: a systematic review of nlts2 secondary analyses. career development and transition for exceptional students, 39(4), 196215. https://doi.org/10.1177/2165143415588047 mcswiggan, s., meares, s., & porter, m. (2016). decision-making capacity evaluation in adult guardianship: a systematic review. international psychogeriatrics, 28(3), 373-384. doi:https://doi. org/10.1017/s1041610215001490 morris, c., detrick, j. j., & peterson, s. m. (2021). participant assent in behavior analytic research: considerations for participants with autism and developmental disabilities. journal of applied behavior analysis, 54(4), 1300–1316. https://doi. org/10.1002/jaba.859 newman, l., wagner, m., cameto, r., knokey, a., & shaver, d. (2010). comparisons across time of the outcomes of youth with disabilities up to 4 years after high school: a report of findings from the national longitudinal transition study (nlts) and the national longitudinal transition study-2 (nlts-2). sri international. retrieved from https://nlts2.sri.com/reports/2010_09/ nlts2_report_2010_09_complete.pdf olszewski, a. e., & goldkind, s. f. (2018). the default position: optimizing pediatric participation in medical decision making. the american journal of bioethics, 18(3), 4-9. perske, r. (1972). the dignity of risk and the mentally retarded. mental retardation, 10(1), 24-27. powers, l. e., geenen, s., powers, j., pommier-satya s., turner, a., dalton, l., & swank, p. (2012). my life: effects of a longitudinal, randomized study of self-determination enhancement on the transition outcomes of youth in foster care and special education. children and youth services review, 34, 2179-2187. https://doi.org/10.1016/j. childyouth.2012.07.018 reid, d.h., rosswurm, m., rotholz, d.a. (2017). no less worthy: recommendations for behavior analysts treating adults with intellectual and developmental disabilities with dignity. behavior analysis in practice, 11(1), 71-79. https:// doi.org/10.1007/s40617-017-0203-y robertson, j., emerson, e., gregory, n., hatton, c., kessissoglou, s. and hallam, a. (2000), receipt of psychotropic medication by people with intellectual disability in residential settings. journal of intellectual disability research, 44: 666-676. https://doi.org/10.1111/j.13652788.2000.00307.x robertson, j., emerson, e., hatton, c., elliott, j., mcintosh, b., swift, p., krinjin-kemp, e., towers, c., romeo, r., knapp, m., sanderson, h., routledge, m., oakes, p., joyce, t. (2007). person-centered planning: factors associated with successful outcomes for people with intellectual disabilities. journal of intellectual disability research, 51, 232 243. https://doi.org/ 10.1111/j.1365-2788.2006.00864.x roux, a. m., shattuck, p., rast, j. e., rava, b. a., anderson, k. a. (2015). national autism indicators report: transition into young adulthood. a.j. drexel autism institute, drexel university. retrieved from https://drexel.edu/autismoutcomes/ p u b l i c a t i o n s a n d re p o r t s / p u b l i c a t i o n s / national-autism-indicators-report-transitionto-adulthood/ saulnier, c. a., & klaiman, c. (2022). assessment of adaptive behavior in autism spectrum disorder. psychology in the schools, 59, 1419– 1429. https://doi.org/10.1002/pits.22690 schwartz, i. s., & baer, d. m. (1991). social validity assessments: is current practice state of the art?. journal of applied behavior analysis, 24(2), 189–204. https://doi.org/10.1901/jaba.1991.24-189 shalock, r. l., & verdugo, m. a. (2012). a leadership guide for today’s disabilities organizations. brookes publishing. shattuck, p. t., garfield, t., roux, a. m., rast, j. e., anderson, k., hassrick, e. m., & kuo, a. (2020). services for adults with autism spectrum disorder: a systems perspective. current psychiatry reports, 22(3), 1-12. https://doi.org/10.1007/s11920-020-1136-7 smidt, a., balandin, s., reed, v. and sigafoos, j. (2007), a communication training programme for residential staff working with adults with challenging behaviour: pilot data on intervention effects. journal of applied research in intellectual disabilities, 20: 16-29. https://doi. org/10.1111/j.1468-3148.2006.00336.x 223 adulthood begins in preschool / gerhardt, bahry, driscoll, cauchi, mason & deshpande snell-rood, c., ruble, l., kleinert, h., mcgrew, j. h., adams, m., rodgers, a., odom, j., wong, w.h., & yu, y. (2020). stakeholder perspectives on transition planning, implementation, and outcomes for students with autism spectrum disorder. autism, 24(5), 1164-1176. https://doi. org/10.1177/1362361319894827 soke, g. n., rosenberg, s. a., hamman, r. f., ringerlin, t., robinson, c., carpenter, l., giarelli, e., lee, l., wiggins, l. d., durkin, m. s., & diguiseppi, c. (2016). brief report: prevalence of self-injurious behaviors among children with autism spectrum disorder—a population-based study. journal of autism and developmental disorders, 46(11), 3607-361. https://doi.org/10.1007/s10803016-2879-1 stancliffe, r. j., abery, b. h., & smith, j. (2000). personal control and the ecology of community living settings: beyond living-unit size and type. american journal on mental retardation, 105(6), 431-454. https://doi.org/10.1352/08958017(2000)105<0431:pcateo>2.0.co;2 stokes, t. & baer, d. (1977). an implicit technology of generalization. journal of applied behavior analysis, 10(2), 349-367. https://doi.org/10.1901/ jaba.1977.10-349 ticani, m., & bondy, a. (eds). (2014). autism spectrum disorders in adolescents and adults: evidencebased and promising interventions. the guildford press. wehmeyer, m. l, & abery, b. h. (2013). selfdetermination and choice. intellectual and developmental disabilities, 51(5), 399-411.https:// doi.org/10.1352/1934-9556-51.5.399 wehmeyer, m. l., kelchner, k., & richards, s. (1996). essential characteristics of self-determined behavior of individuals with mental retardation. american journal on mental retardation, 100(6), 632-642. wehmeyer, m. l., & shwartz, m. (1997). self-determination and positive adult outcomes: a follow-up study of youth with mental retardation or learning disabilities. exceptional children, 63(2), 245-255. https://doi.org/10.1177/001440299706300207 winett, r. a., & winkler, r. c. (1972). current behavior modification in the classroom: be still, be quiet, be docile. journal of applied behavior analysis, 5(4), 499-504. https://doi.org/10.1901/jaba.1972.5499 wolf, m. m. (1978). social validity: the case for subjective measurement or how applied behavior analysis is finding its heart. journal of applied behavior analysis, 11(2), 203-214. https://doi.org/10.1901/ jaba.1978.11-203 international electronic journal of elementary education january 2023, volume 15, issue 3 copyright © www.iejee.com issn: 1307-9298 editorial copyright © www.iejee.com issn: 1307-9298 dear iejee readers, it’s a great pleasure for me to present this special issue about the scientific strengths and practical challenges that characterize applied behavior analysis (aba) in our time. near thirty active researchers within the field of autism/autism spectrum disorder (asd) express their ideas, experiences and achievements in the field of aba. they share their concerns about their observations in the field. through this special issue, they are exposing themselves to the public room. they are doing this while we are celebrating autism awareness month of april. the field of aba has a strong scientific knowledge base. it is not only involved in, but it has been the main contributor to the establishment of twenty-eight evidence-based practices (ebps) for children and youth with asd/autistic children and youth. with these ebps one can give a helping hand to individuals at different ages to improve their socially significant behaviors; social, communicative, functional and adaptive skills. i do not have any doubt that the topics and concerns they are addressing will contribute to eye-opening and strengthen scientific awareness and scientific seriosity in the field. it will not be an exaggeration to claim that the thirteen papers have the potential to start a paradigm shift in the field of applied behavior analysis toward more ethical, more progressive and more reflective practices with fidelity, by everybody, for individuals with autism / autistic individuals. thanks to drs joseph h. cihon, justin b. leaf and ellie kazemi for their editorialship for this valuable special issue of iejee. dr. kamil özerk, professor of education editor-in-chief, iejee international electronic journal of elementary education, 2015, 7(3), 281-300. issn:1307-9298 copyright © iejee www.iejee.com assessing the promise of a supplemental reading intervention for at-risk first grade students in a public school setting kouider mokhtari  the university of texas at tyler, united states joanna l. neel the university of texas at tyler, united states forrest kaiser caldwell elementary school, tyler, texas, united states hong-hai le the university of texas at tyler, united states received: 9 april 2015 / revised: 10 may 2015 / accepted: 25 may 2015 abstract in this exploratory quasi-experimental case study, we assessed the promise of a yearlong supplemental reading intervention with a small pilot group of at-risk first grade readers in an elementary school setting. using standardized measures of reading proficiency, we found that after 47 hours of one-on-one tutoring instruction, students read significantly more proficiently than did non-tutored students in a matched group of first grade peers in the same school. these results are encouraging in light of literacy research documenting the impact of one-on-one tutoring by qualified tutors of at-risk early grade readers. we used lessons learned from this pilot study to help inform and direct the necessary revisions and refinements of future reading interventions with the goal of building the school’s capacity to support the literacy development of at-risk readers so that they can catch up with their typically developing peers. keywords: response to intervention, early literacy instruction, one-on-one tutoring, at-risk readers introduction students arrive at school around 7:00 a.m. parents drop them off by the cafeteria, where they get breakfast and meet their tutors. denica waits for six-year old keyonte each tutoring day (names are pseudonyms). keyonte usually runs 10-15 minutes late. when she picks him up from the cafeteria, they  kouider mokhtari the university of texas at tyler, 3900 university blvd tyler, texas 75799, united states, phone: (903) 566-7177 e-mail: kmokhtari@uttyler.edu http://www.iejee.com/ international electronic journal of elementary education vol.7, issue 3, 281-300, 2015 282 usually talk about his morning or the books he read the night before. these conversations help set the stage for the daily tutoring lessons. denica intentionally asks open-ended questions to help engage keyonte in purposeful and meaningful conversations. most morning conversation starters are centered on breakfast or his clothes (he often has on matching sneakers and jackets, or new jeans with cool logos). gradually, their conversation becomes more centered on books and characters that are interesting to him. once upstairs in the school library, they would settle in for their daily literacy lesson. keyonte often works best when seated side by side at a table with denica. working on the floor proved to be too much of a distraction. sitting beside him allows denica to easily see how he is reading and point out key words and phrases during picture walks. students like keyonte are often regarded as “struggling” or “at-risk” readers. most enter first grade with low literacy skills, and are therefore considered unprepared to fully engage in formal school-based literacy activities. on the other hand, these students come to school with an array of talents and home experiences that are critically important for their school success. in our study of what works for struggling first grade readers, we show how schools can bring about significant improvements in reading performance outcomes among these students, by catching them before they fall, providing them with intensive one-on-one instruction, and expert teaching will be necessary. for many of these students, as marie clay (2005) observed: “it is the individual adaptation made by the expert teacher to that child’s idiosyncratic competencies and history of past experiences that starts him on the upward climb to effective literacy performances.” (p. 63). what research says about how to effectively reach and teach at-risk early grade readers when it comes to what works when teaching children to read in the early grades, we know one thing for certain: there are no silver bullets. on the other hand, a review of several decades of reading research and long-term data give us a better sense of what works — and, for the most part, it’s what we’re not doing, which, according to reading experts, includes early detection of reading difficulties, intensive instruction, and expert teaching for all children, especially for those who come from ethnically diverse and/or low-income families (e.g., au, 2011, clay, 2005; taylor, 2007). for purposes of this article, we will focus on two main lines of literacy research and instruction. the first one examines the characteristics of effective reading instruction programs that have been shown to help at-risk readers catch up to their grade levels. the second line of research examines the degree to which these programs are similarly effective for all children, including those who come from ethnically diverse and economically disadvantaged backgrounds. what does it take to effectively reach and teach early grade at-risk readers? literacy researchers and practitioners generally agree that it is possible to prevent reading problems for most children when they are provided with supplementary instructional support in the form of effective early and intensive literacy interventions (e.g., clay, 2005; snow, burns, & griffin, 1998; wasik & slavin, 1993). some researchers have shown that almost all first grade children can learn to read, including those who enter school with low levels of literacy and who in the past would have failed to learn to read in first grade (e.g., taylor, critchley, paulsen, macdonald, & miron, 2002). a recent u.s. department of education report concluded, after a review of evidence from available randomized controlled studies, that one-on-one tutoring by qualified tutors for at-risk readers in grades 1–3 is effective (institute of education sciences, 2003, p. iii). the report authors further noted “one-on-one tutoring of at-risk readers by a well-trained tutor yields an effect size of about 0.7. this means that the average tutored student reads more assessing the promise of a supplemental reading intervention / mokhtari, neel, kaiser & lee 283 proficiently than approximately 75 percent of the untutored students in the control group.” (p. 19). the above findings appear at odds with students’ reading performance in many united states schools as revealed by state and national tests of reading proficiency. report cards such as the national assessment of educational progress (naep) indicate that since 2007, nearly two out of three 4th grade students in the u.s. have had reading proficiencies below the level needed to do grade level work adequately (national center for educational statistics [nces], 2011). state assessments consistently show that as many as 30% of first grade students, on average, enter school with low levels of reading and writing. the number of students in need of reading assistance is much greater for students of lowincome families, students with disabilities, and students representing culturally, linguistically, and racially different backgrounds. so why are there such high numbers of underachieving readers when the research evidence shows that reading problems are preventable for the majority of young children? the answer to this question is complex but depends, to a great extent, on what schools are doing or not doing to prevent and address students’ reading difficulties. in our work with struggling readers in school settings, we often find that there is a gap between what is known about best practices in literacy instruction and what happens daily in practice, particularly in classrooms that have a high percentage of underachieving readers. the answer also depends on whether schools have the means and expertise to put in place systems for identifying children at-risk of reading difficulties, providing effective literacy instruction in the preschool and early grades, and supporting the professional learning and development of teachers. consistent with these observations, literacy researchers have argued for several decades that “few students in the united states regularly receive the best reading instruction we know how to give” (allington, 2011), and that classroom literacy instruction seldom reflects best practices as identified in the research (taylor, peterson, pearson, & rodriguez, 2010). richard allington, a leading literacy researcher who has spent many years studying exemplary elementary classroom teachers, has argued that as a literacy community, we know how to teach nearly every child to read by first grade. unfortunately, few schools are doing what they need to do to help students most at-risk of reading failure. in an article published in educational leadership, allington and gabriel (2012) outline six elements of effective reading instruction that they assert “do not require much time or money—just educators’ decision to put them in place” (p. 1). the key to reaching the goal of teaching every child to read by first grade depends on providing opportunities for every child to experience these research-based elements of reading instruction every day. according to allington and gabriel (2012), in order to help all students become competent, independent readers and writers, classroom teachers should (a) give students an opportunity to read something that appeals to their interests and needs, (b) read something they can accurately read and understand, (c) write something that is meaningful to them, (d) talk about what they read or write with someone, and (e) hear a fluent adult reader read aloud every day. drawing from research on effective reading instruction during the last four decades, as well as her own research examining the “how” as well as the “what” of effective elementary reading instruction practices, taylor and colleagues concur that many of the classroom literacy instruction practices she observed in thousands of classrooms over a period of several years are inconsistent with research-based instruction practices (taylor, 2007; taylor et al., 2010). they argue that to reach the goal of helping all children in the elementary grades succeed in reading to their fullest potential, teachers and administrators within schools should make a concerted effort to work together to develop international electronic journal of elementary education vol.7, issue 3, 281-300, 2015 284 and deliver a sound school-wide reading program. she further notes that while schools know that a wealth of information is available to help them move closer to helping every child become a reader, putting all the relevant pieces together remains a challenge. moreover, she points out that ongoing professional development in which teachers work together within their buildings to reflect on their practices is an important first step in achieving this goal. researchers, policy makers, and practitioners generally agree that the first three years of classroom instruction are critically important for preventing students from falling behind and preventing reading failure. during these critical years, schools lacking the expertise and/or the resources to put in place a system for providing expert reading instruction for all students are likely to create a pool of students who will become struggling readers. by and large, during the past several decades, schools have made substantial progress in addressing reading difficulties by designing effective early reading intervention programs. one of the most notable examples of successful early intervention programs is the reading recovery model, which uses one-to-one tutoring for struggling readers in grades 1-3 (clay, 2005, pinnell, lyons, deford, bryk, & seltzer, 1994; pinnell, fried, & estice, 1990). another successful program is success for all, which has a track record of providing successful school-wide tutoring interventions for students at-risk of reading difficulties (slavin, madden, karweit, livermon, & dolan, 1990). programs such as these, and others, use literacy practices that are supported by research evidence and that have been shown to work well for at-risk readers. additionally, with new initiatives such as response to instruction (rti), which is a part of the 2004 reauthorization of the individuals with disabilities act, research indicates that it is possible to substantially reduce the number of students classified as learning disabled. this legislation, according to johnston (2011) and allington (2010) enables schools to (a) provide increasingly intensive tiered instruction to help ensure that students having difficulty learning to read are provided with the requisite expert instruction, and (b) identify students who continue to have reading difficulties after receiving intensive reading instruction. the most commonly used form of rti has three tiers of instruction ranging from conventional classroom reading instruction (tier 1), to supplementary expert instruction delivered in small group settings (tier 2), to targeted instruction provided in one-on-one tutorial settings for students most in need of reading assistance. since the enactment of the idea legislation, there has been a great deal of interest within schools and districts to put in place tiers of instruction systems aimed at significantly reducing the number of students experiencing reading difficulties. to address the needs of students who are most at-risk of reading difficulties (i.e., those who are in the third tier of instruction), many schools have put in place various types of extended-day programs depending on their needs and resources. allington (2012) describes four of the most commonly used extended school-day designs as follows: 1) school-based remedial assistance with expert reading instruction. in this design, eligible students work with reading and/or special education teachers for an hour or more after school to accelerate literacy development. 2) school-based tutoring with trained community volunteers, high school, or college students. designs such as this often consist of only once or twice weekly sessions, although some do provide daily instructional support. 3) school-based homework help/child care/recreation with paraprofessional or volunteer support. in this design, eligible students receive mostly homework assistance with corresponding recreational activities. assessing the promise of a supplemental reading intervention / mokhtari, neel, kaiser & lee 285 4) community-based homework help/child care/recreation. there are actually fewer school-based than community-based after school programs currently operating. these programs, which are similar to school-based programs, are sponsored by organizations such as ymca, boys and girls clubs, churches, and other community groups (allington, 2012, pp. 179-180). these designs vary in terms of target audience, staffing expertise, instructional focus, and intensity of instruction. research on the effectiveness of these program designs is rather mixed. for instance, in one study, wasik and slavin (1993) found that programs using certified teachers resulted in significantly higher gains than programs using noncertified staff. however, other researchers (e.g., davidson & koppenhaver, 1993; inverzini rosemary, juel, & richards, 1997; wasik, 1998) found that programs using non-certified personnel were as effective as those using certified teachers. the key to success in these programs, according to allington (2012), appears to be related to “providing non-certified personnel with strong training, structured tutorials, and ongoing supervision” (p. 181). have reading programs been equally effective for nearly all at-risk readers? looking beyond the reading programs that have been shown to be effective in helping atrisk readers catch up to their grade level, a growing number of literacy researchers and teacher educators have expressed concern that effective programs have not always been comparably successful for all at-risk readers, especially for those children who come from ethnically diverse and economically disadvantaged backgrounds (e.g., au, 2011; cochransmith, 2004; compton-lilly, 2007, 2009, 2011; gorski, 2013; ladson-billings, 2009). these children, they argue, are often overrepresented among at-risk readers in schools. they bring a vast range of abilities, practices and life experiences that are often quite different from the practices and experiences they encounter at school. in light of these circumstances, one should not assume that a program or intervention that proves to be effectiveness for one group of students will be equally effective for other groups of readers. while serving as a reading recovery teacher trainer in a high-poverty school in wisconsin, compton-lilly (2011) examined how well the program serves african american children, including children who did not complete the 20-week interventions or who were otherwise hindered by policies, which disadvantaged children who bring diverse life experiences to reading recovery classrooms. she found that there was a 20% difference in success rates between african american children and european american children when we considered all the children served in reading recovery. these authors argue that recognizing the funds of knowledge these children bring to school is an important first step to helping them become successful readers and writers. in his book, reaching and teaching students in poverty: strategies for erasing the opportunity gap, gorski (2013) provides an insightful review of the research and instructional practices that hold promise for working with these children around literacy, an excellent analysis of why economic inequities exist and persist among public school students, and an overview of practical classroom-tested guidance for teachers and leaders who care enough to make a difference. the present study we collaborated with an elementary school on the design of a supplemental or tier 3 before-school tutoring program, with the goal of enhancing the reading skills of a small pilot group of at-risk, first-grade students. our proposed supplemental reading intervention program is fairly similar to the second program design described above in that it uses trained pre-service teachers and provides a highly structured and closely supervised reading intervention program. international electronic journal of elementary education vol.7, issue 3, 281-300, 2015 286 the overall purpose of the study was to evaluate the effectiveness of this supplemental reading intervention on students’ reading achievement outcomes. specifically, we were interested in finding out whether participation in this supplementary tutoring program results in significant improvements in students’ reading achievement as measured by scores obtained on nationally normed measure of reading proficiency. instructional/research setting we conducted the pilot study in a local area elementary school located in a mid-size city in the southwestern united states. opened in 2000, the target school is a fine arts magnet school operating within a large independent school district (approximately 18,000 students) in the southwestern united states. the school has 675 students in kindergarten through 5th grade. student population is 36% african american, 32% hispanic, and 27% white with an economically disadvantaged rate of 65%. forty-two full-time teachers and 20 support staff serve these students. the school has five first grade classrooms with an average of 24 students per classroom. of the 120 or so students in first grade, about 3040% were designated as needing assistance in reading. study participants participants in this case study consisted of 12 first grade at-risk readers who entered first grade with low levels of reading and writing skills. students were selected for participation in the study based on teacher recommendations and student performance on district benchmark assessments data using the texas primary reading inventory test battery, which placed them in the lowest performing quartile among all first grade students in the school. of the 12 students identified for tutoring, 5 were hispanic (4 males, 1 female) and 7 were african american (6 males, 1 female). although none of the students were identified as having a reading or learning disability, one student repeated first grade, one was identified as having attention deficit hyperactivity, and one was an english learner. additional information about student demographics is provided in table 1. in the section below, we provide a brief profile of each of the student participants. j.a. is seven years old. he likes to draw and enjoys spending time with his family. his mother works at a day care, and he spends a lot of time with his granny. he enjoys mysteries and scary stories with monsters, and he likes playing math games at school. in his tutoring sessions, it is often difficult for him to focus and he gets off-task easily if he is not actively engaged in an activity. initially, he had low confidence in his reading ability, but has visibly gained confidence through this program. six-year-old v.a. enjoys playing at the playground. she lives at home with her mom, dad, and sisters. v.a prefers short, funny books, particularly about animals. at home, she does not have any books in her room but her sisters read chapter books so they have a few books around. during her tutorial sessions, v.a struggles to read fluently. she began tutoring with low confidence in reading and was very shy, but quickly gained confidence through her reading and enjoys choosing the book she would like to read during tutoring sessions. l.b. is six years old. she spends a great deal of time drawing, playing games on the computer, and socializing with her siblings. she currently lives with her grandmother and aunt and is the youngest of her many siblings. l.b says she enjoys reading, particularly animal books, picture books, and funny stories. at school, she enjoys playing games, especially math games. at home, her grandmother occasionally reads to her before bedtime. during tutoring sessions, l.b struggles to stay engaged. her confidence level in reading is high during familiar reading, and is hesitant to attempt reading through more difficult texts. assessing the promise of a supplemental reading intervention / mokhtari, neel, kaiser & lee 287 e.c. is seven years. he loves playing soccer. at school, e.c enjoys learning, playing with his best friend at recess, learning math, and participating in science activities. he says he likes to read about mysteries, and books about super heroes, animals, sports, and cars. at home, he reads mostly with his granny but there aren’t any books at his house. e.c struggles mainly with phonemic awareness skills such as blending sounds. overall, e.c has a desire to learn and finds reading enjoyable. seven-year-old n.c. enjoys playing with his friends, and playing games on the computer. he claims that he loves to read, no matter the genre, and likes to write stories. at home, there aren’t many books to read except in his mimi’s room. during his tutoring sessions, it is evident that he struggles to read words and sentences at age-appropriate levels fluently, which poses problems for him when trying to understand what he reads. h.c. is seven years old. his favorite activities include going to waterparks, watching t.v. and playing video games. h.c lives with his mom but visits his dad (non-english speaking) on the weekend. he also has an older brother who is at the same elementary school. he likes to be able to read books that are funny. h.c says that his mother sometimes reads to him at home on occasion. in tutoring sessions, h.c requires glasses to read, which he often forgets, and he struggles with word decoding and reading fluency. his confidence in his reading is low. j.h. is seven years old. he likes playing on the computer and with video games. his favorite character is the hulk. j.h prefers reading non-fiction and books about superheroes and he reads with his mom in the evening with the few books he has at home. he gets easily distracted during tutoring sessions and finds it hard to focus on what he is reading. six-year-old m.j. enjoys football, spiderman, and playing with his younger siblings. he lives with his mom, who is an accountant, and his dad, a policeman, and has an older brother in 3rd grade and a newborn brother. in his tutoring sessions, m.j. has trouble making connections between letter sounds in words longer than five letters and struggles with comprehension when reading grade level materials. j.m. is eight years old. he likes watching t.v. and drawing, and has interests in bugs and animals. he lives with his mom and has older sisters who are also in elementary school. j.m. rides the bus to school and is sometimes late to school because of it. he likes his tutor and looks forward to coming to school as a result. i.o. is seven years old. he likes playing on the playground, tag, and soccer. he lives at home with his parents and two brothers. i.o. prefers to read books about spiders, soccer, animals, cars, and super heroes. at home, he reads with his parents and brothers and has several dr. seuss books in the house. i.o. often arrives late to his tutoring sessions, which reduces the amount of time his tutor gets to spend with him. c.r. is six years old who enjoys drawing pictures and learning about fish. at home, he reads mostly with his mother. while they do not have books in his house, his mother frequently reads magazines and his grandmother reads newspapers. c.r. prefers to read books about sports and super heroes. he has excellent attendance at school and likes getting help in reading. e.s. is a six-year-old boy who struggled with reading but has made great progress throughout the year. he enjoys engaging in literacy conversations and reading books at home with his siblings, parents, aunts, uncles, and grandparents. he reports that his living room only has two or three books, although he has access to magazines at home and he like computer-reading games. he also likes math and science because he thinks they are cool and with reading, the pictures in the books let him know what he is reading. during international electronic journal of elementary education vol.7, issue 3, 281-300, 2015 288 tutoring sessions, he especially likes to read about sports, superheroes, animals, car and trucks, book in series, and funny stories. instructional framework for purposes of this study, we developed a research-based instructional framework to help us organize and manage instruction. our literacy instruction framework incorporates established instructional design characteristics in terms of content, organization, and management of literacy instruction. it also takes into account student needs as well as the needs of our tutors. for instance, we wanted to design a framework that incorporates the literacy needs of our target group of students with respect to early language and literacy skills. in addition, we developed a flexible the framework to enable our tutors to adapt lessons that are relatively easy to implement and evaluate, and to enables us to closely supervise and monitor the degree to which components of the framework are implemented as intended to the extent possible. our instructional framework includes three key components including (a) a 20-minute reading or re-reading of an easy or familiar text, (b) a 20-minute shared or interactive reading of a self-selected text, and (c) a 20-minute interactive writing segment targeting specific writing skills and strategies. together, the three components of the framework take approximately 50-55 minutes to complete. appendix a provides a descriptive outline of the key components of the framework with sample activities. consistent with the recommendations delineated in the newly developed publishers’ criteria for the common core state standards in english language arts and literacy, grades k–2 (coleman & pimentel, 2012), all curriculum materials used when tutoring our target students have three important characteristics: first, students have access to a mix of literary and informational texts that vary in terms of topics, length, complexity, and genres. we wanted to help ensure that students have opportunities to read texts that are rich and accessible on their own and with others in order to help build their knowledge, experience, and enjoyment of reading. second, we made a concerted effort to select materials that appeal to students’ interests and needs. we wanted to provide students access to interesting reading materials that motivate them to read so that they can do so independently in and outside of school. third, we provided a sufficient number of leveled reading materials during each tutoring session so as to enable students to read texts they can read on their own as well as texts that are more complex that they may be able to read with tutor guidance. tutors introduce higher-level texts and present them through readalouds, shared readings, and other tutor-assisted strategies. instructional delivery pre-service teachers pursuing degrees at a local university received training as literacy tutors and delivered the tutoring lessons three times a week for a period of 10 weeks during the regular fall and spring semesters of one school year. tutor training took place as a part of a semester-long reading assessment and instruction course that pre-service teacher candidates complete in partial fulfillment of the requirements of a bachelor’s degree in elementary education. this course is designed to help pre-service teachers strengthen expertise in identifying students’ reading difficulties and designing instruction aimed at helping students improve their early grade reading skills. the course includes a supervised field experience, which requires teacher candidates to apply what they learn about reading assessment and instruction in a local elementary school setting. tutor training our approach to preparing effective tutors is guided by the ecological context of our atrisk participating readers. we challenge our tutors to think critically and reflectively about assessing the promise of a supplemental reading intervention / mokhtari, neel, kaiser & lee 289 productive ways of supporting the reading and writing development of these students, many of whom come to school with low levels of literacy. we want our tutors to think carefully about why they do what they do before rather than after thinking about what they do and how they do it. when tutors explore the underlying reasons why a particular child entered first grade with lower than expected reading skills, for instance not knowing all the letters of the alphabet, they are more likely to be constructively responsive to the needs of that child. they are also likely to become more thoughtful and reflective about their teaching. we planned and implemented our tutor training in three closely integrated steps. these steps include an initial training phase during the first four weeks of the semester, a 30-minute debriefing phase, which takes place immediately following each tutoring session, and an individual consultation phase, which takes place throughout the semester depending on the needs of individual tutors. during the first four weeks of the semester, tutors receive instruction in approximately 1.5 hours twice a week for four weeks. during this time, tutors also learn about the school setting in which tutoring takes place, meet with the school principal and teachers, and learn about the students they will be assigned to tutor. during the tutor training sessions, instruction and coaching typically consists of close reading and discussion of evidence-based literacy practices, using assessment data to inform instruction, and organizing instruction in tutorial settings using a common instructional framework. supporting materials for the first component include a mix of readings such as “what at-risk readers need” (allington, 2011),“fad: filtering, analyzing, and diagnosing reading difficulties” (mokhtari, niederhauser, beschorner, & edwards, 2011), and “what’s a tutor to do? (roller, 2006). during the debriefing sessions, tutors engage in a guided tutoring lesson study and reflection with a focus on what worked well, and what needs improvements, as well as sharing of ideas and resources. individual consultation is initiated either by the instructor or by the student depending on perceived needs. tutors keep a reflective journal, which often serves as a source for identifying challenges, questions, or issues for discussion. research design doing research in real-world school settings is complex. our study takes place in a school setting with ecological constraints, which makes it not feasible or ethical to conduct randomized controlled experiments, the ‘gold standard’ for establishing what works. for purposes of our study, because our target student participants were identified as at-risk readers with a unique set of needs and backgrounds, we were not interested in just finding out whether our intervention generally works for these students as a group, for doing so does little to tell us under what circumstances it does or does not work. more importantly, our goal was to use the data obtained from the intervention to better understand the conditions under which individual children, not just groups, succeed or fail to learn to read. we collected student and tutor assessment data at various times during the year to help us determine the promise of the literacy tutoring intervention for producing the outcomes it is intended to produce. we collected preand postassessment data for all first grade students in the school. doing so enabled us to make comparisons in reading achievement outcomes across three groups of students: at-risk readers who needed and received reading assistance (at-risk tutored), at-risk readers who needed but did not receive supplemental reading instruction (at-risk non-tutored), and typically developing readers who did not need nor receive supplemental reading assistance (typically developing peers). because we combined two approaches in our research design, (we international electronic journal of elementary education vol.7, issue 3, 281-300, 2015 290 collected data multiple times and examine student performance across groups), our design can best be described as a quasi-independent group design with one equivalent group and one non-equivalent group design. when selecting our non-tutored comparison group, we used propensity score matching, a quasi-experimental technique, recommended by guo & fraser (2010) to find students similar to the intervention students in terms of their background characteristics, using information from school databases such as gender, ethnicity, school-administered reading benchmark test scores, participation in free or reduced lunch, and other demographic characteristics. we also randomly selected a group of typically developing readers from the rest of first grade students who did not need assistance in reading. data collection we assessed students’ reading achievement outcomes at the beginning and end of the school year using the gates-macginitie reading test (macginitie & macginitie, 2004). this test is a standardized nationally norm-referenced, general reading ability test, which assesses foundational literacy skills including vocabulary and reading comprehension skills. it is commonly used test of reading ability with adequate technical adequacy as measured by reported reliability (reliability coefficients range from .89 to .93 for vocabulary and .87 to .94 for comprehension) and validity data (high correlations reported for studies correlating gates reading tests with other tests of reading comprehension). in order to determine the extent to which tutors adhered to the key elements of the instructional framework, we collected fidelity of implementation data in two ways. first, tutors completed a fidelity of lesson implementation protocol for each of the lessons they delivered (see appendix a). fidelity of lesson implementation consisted of checking off lesson components completed as-is, modified, or not completed. second, two members of our team observed each tutor for at least one lesson on two occasions during the school year using a lesson observation protocol (see appendix b). these data helped us monitor and assess the functioning of the tutoring lessons in action, collect implementation fidelity data, and make the necessary adjustments in the design of the intervention so as to enhance its overall functioning. we reviewed 100% of the lesson implementation fidelity forms completed by tutors and sought reliability for coding the observation protocols made by two members of our team for each tutor observed. an examination of the tutor fidelity of lesson implementation indicated that our tutors implemented lessons as intended or with slight adaptations about 85% percent of the time. inter-rater reliability between two lesson observers was 89%. data analysis we collected student and tutor assessment data throughout the year to help us determine the promise of the literacy tutoring intervention for producing the outcomes it is intended to produce. in an attempt to determine whether the reading intervention had an impact on at-risk tutored students, we analyzed the data obtained in three different yet complementary ways. first, we examined the amount of tutoring time each student received over the course of the school year, and we analyzed the progress in reading proficiency achieved by students individually and as a group from fall to spring taking into account the total tutoring time invested. second, we examined the progress achieved by tutored students in comparison to a matched group of at-risk non-tutored peers and in comparison to all typically developing first grade peers in the same school. we analyzed the data obtained by using analysis of covariance (ancova), which enabled us to control for students’ initial level of reading assessing the promise of a supplemental reading intervention / mokhtari, neel, kaiser & lee 291 ability using students’ gates-macginitie fall semester scores as a covariate, and provided us with a cleaner measure of effect of our instructional intervention. finally, with a relatively small number of students in our study, and in light of the fact that students’ reading development changes over time, but not necessarily in the same way or at the same rate, we reviewed each student’s growth individually, and examined student performance in relation to national proficiency standards so as to determine whether the student is on track to reach proficiency, or remains at-risk of reading failure. results the results obtained are presented in three complementary ways: as the amount of tutoring time each student received over the course of the school year, the progress in reading proficiency achieved by students individually and as a group from fall to spring taking into account the total tutoring time, the progress achieved by tutored students in comparison to a matched group of at-risk non-tutored peers and typically developing first grade peers in the same school. please note that our data analyses are based on a total of 10 participating at-risk students since one of the students moved in the middle of the year, and one student completed the program but did not participate in all assessments. these results are presented in tables 1-2 and figures 1. table 1 presents group demographics as well as the amount of time spent by the at-risk first grade students who received tutoring the fall and spring semesters of the school year. table 1. student demographic profiles at-risk tutored at-risk non-tutored typical peers (n=12) (n=11) (n=10) ______________________________________________________________________________ mean age (sd) 6.67 (.65) 6.72 (.64) 6.5 (.53) gender male 8 7 6 female 4 4 4 ethnicity african-american 7 6 2 caucasian 0 1 6 hispanic 5 4 3 special need english learner 1 1 0 special needs 1 1 0 __________________________________________________________________________________________________________ how much one-on-one instructional time did each at-risk student receive over the course of the school year? as table 1 shows, the ten first graders received an average of 17.85 hours of instructional time during the fall semester and 28.95 hours of tutoring during the spring semester for a total average of 47.33 hours (sd=4.58) of tutoring or roughly 2840 minutes. individual tutoring time ranged from a low of 42 hours (2520 minutes) to a high of 53.95 (3195 minutes). the average tutoring time our students received falls within the recommended 44-80 hours range of instruction needed to substantially reduce the incidence of reading failure in a school system by accelerating at-risk students’ reading proficiency to average levels of performance (allington, 2012; clay, 2005). what proportion of tutored students made sufficient progress in reading proficiency after 47 hours of one-on-one reading instruction? an examination of the students individual reading progress data show that 9 out of 10 of the at-risk tutored students achieved significantly higher extended scale scores on the international electronic journal of elementary education vol.7, issue 3, 281-300, 2015 292 gates-macginitie test in the spring 2013 when compared to their performance on the same test in the fall 2012. these data are fairly consistent with literacy tutoring research suggesting that when tutored by a well-trained tutor, the average at-risk reader should be expected to read more proficiently than approximately 75 percent of the untutored students in the control group (institute of education sciences, 2003). are at-risk tutored first graders learning beginning reading skills at about the growth rate one would expect? at the end of first grade, the main concern is whether each student has developed adequate beginning reading skills to get a good start in reading. to determine whether a student has made good gain during grade 1, assessment experts recommend using national curve equivalents (nces), which are normalized standard scores with a mean of 50 and a standard deviation of 21.06. nces measure progress in reading by describing a student’s position within the norming group at successive times during the year or grade levels. as a general rule of thumb, experts agree that a student who maintains about the same nce from fall to spring or earns a total score on a test level less than 7 nces has not changed relative to the achievement of students in the norming group. a student with an nce score of 50 is roughly at grade level. table 2 displays the average nce scores of students in our three groups. an examination of these data indicates that at-risk students made gains of nearly 20 nces between fall and spring semesters while the scores of at-risk non-tutored students actually declined by nearly two points from 30.1 to 28.6. typically developing first grade peers gained nearly 18 points from fall to spring. while it is evident that the nce growth scores of the at-risk group of students were lower than typically developing peers, the reading progress made provide evidence that the reading intervention has made a significant difference in the reading skills of at-risk tutors. table 2. comparison of reading growth rate of at-risk tutored, at-risk no-tutored, and typically developing non-tutored first grade peers measures fall winter spring m (sd) m (sd) m (sd) gates-macginitie reading test (nce)* at-risk tutored peers 21.75 (14.0) 37.9 (14.36) 40.3 (18.7) at-risk non-tutored peers 30.1 (15.7) 30.8 (19.8) 28.6 (19.3) typically developing peers 60.0 (26.6) 78.4 (22.19) 77.67 (21.2) ____________________________________________________________________________ * nce= national curve equivalent how proficiently did at-risk tutored students read when compared to at-risk non-tutored peers and to typically developing first grade students in the same school? table 3 presents the means and standard deviations pertaining to the reading progress made by the three groups of students at three points in time during the school year (i.e., fall, winter, & spring). table 3 also includes the means that were adjusted for the effects of the covariate. following field (2009), we used these means rather than the original means to more accurately interpret the group differences reflected in our ancova analysis. assessing the promise of a supplemental reading intervention / mokhtari, neel, kaiser & lee 293 table 3. comparison of reading proficiency of at-risk tutored, at-risk non-tutored, and typically developing non-tutored first grade peers measures fall winter spring m (sd) m (sd) m (sd) gates-macginitie reading test (ess)* original means adjusted means** at-risk tutored peers 288.3 (22.6) 343.0 (24.9) 371.7 (43.5) 393.83 (12.6) at-risk non-tutored peers 306.4 (28.2) 329.1 (35.5) 342.3 (43.9) 351.09 (11.1) typically developing peers 367.3 (56.7) 433.1 (54.8) 456.6 (50.3) 421.19 (14.8) notes: *ess=extended scale score; ** means adjusted for the effect of the covariate using a test of between-subject effects, we found an overall significant group effect of our reading intervention after controlling for initial group differences in reading ability, f(2,26) = 8.002, p = .002). an examination of the adjusted means (and contrasts) for the three groups shows that the at-risk tutored students had higher adjusted means after 47 hours of one-on-one reading instruction (m= 393.83; sd= 12.6) than did their at-risk matched peers who did not receive any tutoring (m= 351.09; sd= 11.1). however, as table 2 shows, the adjusted means of the typically developing peers (m= 421.19; sd= 14.8) was higher than both the at-risk tutored non-tutored. we verified these results using the sidak corrected post hoc pairwise comparisons among the three group means and found that the at-risk tutored students outperformed the at-risk non-tutored students, (p = .04). the post hoc tests further showed that the typically developing students outperformed the atrisk non-tutored students (p = .001) but not the at-risk tutored students (p = .218) although they had a higher adjusted mean score. figures 1 provide a visual depiction of these differences. figure 1. comparison of reading growth of at-risk tutored, at-risk non-tutored, & typically developing non-tutored peers discussion the findings of this study are encouraging. our pilot quasi experiment generally shows that it is indeed possible to significantly advance the early grade reading achievement international electronic journal of elementary education vol.7, issue 3, 281-300, 2015 294 outcomes of at-risk first grade students when we provide them with a sufficient amount of individualized instruction by well-trained tutors. we analyzed the results using a suite of procedures to determine whether our pilot intervention achieved its intended outcomes, and functioned well in a local area school setting. taken together, our findings revealed several promising outcomes. first, we found that 9 of 10 of the at-risk tutored students achieved significantly higher extended scale scores on the gates-macginitie test at the end of the intervention when compared to their performance on the same test before the intervention. these data are consistent with literacy tutoring research suggesting that when tutored by a well-trained tutor, the average at-risk reader should be expected to read more proficiently than approximately 75 percent of the untutored students in the control group (institute of education sciences, 2003). second, we found that students who received tutoring read more proficiently after 47 hours of instruction when measured by gates-macginitie, a standardized norm-referenced test of reading ability. these students read more proficiently at the end of the intervention than did non-tutored students in a matched group of first grade peers in the same school. however, when compared with the performance of typically developing readers, we found that these students received lower average reading proficiency scores than typically developing peers in the same school, although this difference was not significantly different. third, we examined student performance to determine whether at-risk tutored first graders’ beginning reading skills were at or about the growth rate one would expect. using average nce scores of students in our three groups of readers, we found that at-risk students made gains of nearly 20 nces between fall and spring semesters while the scores of at-risk non-tutored students actually declined by nearly two points from 30.1 to 28.6. typically developing first grade peers gained nearly 18 points from fall to spring of the same year. while it is evident that the nce growth scores of the at-risk group of students were lower than typically developing peers, the reading progress made provide evidence that the reading intervention has made a significant difference in the reading skills of atrisk tutors. when considering all aspects of this pilot study, we find that while these results are quite encouraging, especially in light of literacy research documenting the impact of oneon-one tutoring by qualified tutors of at-risk readers in grades 1-3, an achievement gap remains when comparing the reading proficiency of tutored students to that of their typically developing first grade peers who did not need extra assistance in reading. this is not too surprising since our students entered first grade with a significantly larger gap in literacy achievement than did typically developing peers. closing this reading achievement gap will take additional instructional time in the form of one-on-one and/or small group instruction, which will help accelerate to average levels of performance the progress of children who show early signs of reading difficulty. some of these children’s reading progress typically falls within the lowest 20% of the enrollment in similar school settings. in light of these findings, it is worth noting that in order to help maintain the progress at-risk students made during the school year, opportunities need to be provided for them to read and write during the summer months. research has shown that students in primary and elementary grades lose much of their reading ability when they do not read during the summer months when school is not in session. this reading loss has been shown to affect these students’ reading performance when they return to school in the fall. research has also shown that students from lower socio-economic backgrounds suffer greater summer reading loss than do students from upper socio-economic levels. the assessing the promise of a supplemental reading intervention / mokhtari, neel, kaiser & lee 295 likelihood of summer reading loss is therefore more real for students who are poor and who have poorly developed language skills (allington & mcgill-franzen, 2003). taking a rearview mirror look at our yearlong journey, we learned valuable lessons from this pilot study that helped inform and direct the necessary revisions and refinements of the intervention with the goal of building the school’s capacity to support the literacy development of at-risk readers so that they can catch up with their typically developing peers. we are eager to share these lessons with primary and elementary grade reading teachers and school leaders, who may be interested in putting in place reading intervention programs aimed at improving reading performance among underachieving students in the early grades. lessons learned in assessing what worked well for us in this pilot program, we attribute the improvement in reading proficiency among our target students to five closely inter-related contributors. first, we wanted to identify students who entered school with low literacy skills fairly early during the year and in first grade. even though the school had underachieving readers in second through fifth grade, we wanted to design a reading intervention program that specifically targets students entering first grade. investing in first grade reading development will is more likely to have an impact on reducing the incidence of reading failure in subsequent grades. second, drawing from research and practices documenting what has worked particularly well in tutoring programs such as reading recovery, success for all, and others, we wanted to provide these students with a sufficient dose of intensive instruction that is likely to result in improved reading achievement outcomes for these students. although we know that the amount of instructional time needed to help close the achievement gap varies a great deal depending on student needs, we used the recommended margin of 44-80 hours of instruction as a general target (e.g., allington, 2012) in designing our reading intervention for these at-risk students. third, we worked conscientiously to help ensure that our tutors, who were pre-service teacher candidates, were effectively prepared for their tutoring roles and responsibilities. as we indicated in the tutor training section above, our training focused on the challenges, issues, and questions that our target students were experiencing at that time. tutors received intensive training during the first four weeks of the semester on evidence-based literacy practices, using assessment data to inform instruction, and organizing instruction in tutorial settings using a common instructional framework. this training was followed by daily debriefing sessions, which took place immediately following each tutoring session, and individual consultation, which took place throughout the semester depending on the needs of individual tutors. this model of tutor preparation was experiential and focused primarily on the school context in which tutoring took place. finally, we attribute the impact of this pilot reading intervention to our schooluniversity collaborative relationship, which proved essential in terms of identifying students in need of reading assistance, access to pertinent assessment data, providing an environment conducive to tutoring sessions, and enabling excellent communication among tutors, parents, and the first grade teachers of these students. international electronic journal of elementary education vol.7, issue 3, 281-300, 2015 296 acknowledgement the anderson-vukelja-wright endowment fund provided support for the design and implementation of this project. • • • references allington, r. l., (2011). what at-risk readers need. best of educational leadership, 68, 40-45. allington, r., and gabriel, r. (2012). every child, every day. educational leadership, 69 (6), 10-15. allington, r., and mcgill-franzen, a. (2003). the impact of summer setback on the reading achievement gap. the phi delta kappan, 85(1), 68-75 allington, r.l. (2010). essential readings on struggling readers. newark, de: international reading association. allington, r.l. (2012). what really matters for struggling readers: designing research-based programs. new york: longman. au, k. (2011). literacy achievement and diversity: keys to success for students, teachers, and schools. new york, ny: teachers college press. clay, m. m. (2005). literacy lessons designed for individuals: part one, why? when? how? auckland, new zealand: heinemann. cochran-smith, m. (2004). walking the road: race, diversity, and social justice in teacher education. new york, ny: teachers college press. coleman, d., & pimentel, s. (2012). revised publishers’ criteria for the common core state standards in english language arts and literacy, grades k–2. the national governors association center for best practices and the council of chief state school officers. washington, dc. compton-lilly, c.f. (2007). re-reading families: the literate lives of urban children four years later. new york, ny: teachers college press. compton-lilly, c.f. (2009). literacy practices of african american children: three case studies. in g. liu (eds.), multicultural families, home literacies, and mainstream schooling, (pp. 29 49). charlotte, nc: information age publishing company. compton-lilly, c.f. (2011). counting the uncounted: african american students in one reading intervention. journal of early childhood literacy. 11(1), 3 24. davidson, j. & koppenhaver, d. (1993). adolescent literacy: what works and why. new york: garland. field, a. (2009). discovering statistics using spss. london, england: sage publications. gorski, p. (2013). reaching and teaching students in poverty: strategies for erasing the opportunity gap. new york: teachers college press. guo, s. & fraser, m.w. (2010). propensity score analysis: statistical methods and applications. los angeles: sage publications. holdaway, d. (1979). the foundations of literacy. portsmouth, nh: heinemann institute of education sciences (2003). identifying and implementing educational practices supported by rigorous evidence: a user friendly guide. washington, dc: u.s. department of education. invernizzi, m., rosemary, c. a., juel, c., & richards, h. (1997). at-risk readers and community volunteers: a three year perspective. journal of the scientific studies of reading, 3, 277-300. assessing the promise of a supplemental reading intervention / mokhtari, neel, kaiser & lee 297 johnston, p.h. (2011). response to intervention in literacy: problems and possibilities. the elementary school journal, 111 (4), 511-534. ladson-billings, g. (2009). the dreamkeepers: successful teachers of african american children. san francisco, ca: jossey-bass. macginitie, w. h., macginitie, r. k., maria, k., dreyer, l. g., & hughes, k. e. (2000). gates-macginite reading tests (4th edition). rolling meadows, il. riverside publishing. mokhtari, k., niederhauser, n., beschorner, b., & edwards, p. (2011). f.a.d.: a basic procedure for filtering, analyzing, and diagnosing students’ reading & writing difficulties. the reading teacher, 64 (8), 631-635. national center for education statistics (2011). the nation's report card: reading 2011 (nces 2012–457). national center for education statistics, institute of education sciences, u.s. department of education, washington, d.c. pinnell, g. s., fried, m. d., & estice, r. m. (1990). reading recovery: learning how to make a difference. the reading teacher, 43(4), 282–295. pinnell, g. s., lyons, c. a., deford, d. e., bryk, a. s., & seltzer, m. (1994). comparing instructional models for the literacy education of high-risk first graders. reading research quarterly, 29(1), 8–39. roller, c. m. (1998). so… what’s a tutor to do? newark, de: international reading association. slavin, r. e., madden, n. a., karweit, n. l., livermon, b. j., & dolan, l. (1990). success for all: firstyear outcomes of a comprehensive plan for reforming urban education. american education research journal, 27(2), 255–278. snow, c.e., burns, m.s., & griffin, p. (eds.) (1998). preventing reading difficulties in young children. washington, d.c: national academy press. taylor, b. m., critchley, c., paulsen, k., macdonald, k., & miron, h. (2002). learning to teach an early reading intervention program through internet-supported professional development. retrieved from eir® website: http://www.earlyinterventioninreading.com/pdfs/taylor_research2.pdf. taylor, b.m. (2007). the what and the how of good classroom reading instruction in the elementary grades. minneapolis, mn: university of minnesota center for reading research. taylor, b.m., peterson, d. s., pearson, p.d., & rodriguez, m.c. (2010). looking inside classrooms: reflecting on the “how” as well as the “what” in effective reading instruction. in r.l allington, essential readings on struggling learners. newark, de: international reading association. trelease, j. (2013). the new read-aloud handbook. new york, ny: penguin. wasik, b. a. (1998). volunteer tutoring programs in reading: a review. reading research quarterly, 33, 266-292. wasik, b. a., & slavin, r. e. (1993). preventing early reading failure with one-to-one tutoring: a review of five programs. reading research quarterly, 28(2), 178–20. international electronic journal of elementary education vol.7, issue 3, 281-300, 2015 298 appendix a literacy lesson framework lesson component description time frame reading familiar text session begins by having the child re-read a familiar text with the goal of building reading fluency and boosting self-confidence (clay, 1995; pinnell, fried, & estice, 1990). tutor conducts a running record as the child reads. 15 minutes 2.5 minute break interactive reading shared book reading and writing is interactive experience whereby the child participates in guided reading and writing activities, thus allowing them to learn about how language works and to see themselves as readers and writers (snow, griffin, & snow, 1998; holdaway, 1979). 20 minutes 2.5 minute break interactive writing session concludes with the tutor modeling reading of (and writing about) a challenging new text. this is an opportunity for the child to read and write about texts that are rich and accessible in terms of content so as to help build their knowledge, experience, and enjoyment of reading and writing (trelease, 2006). 20 minutes assessing the promise of a supplemental reading intervention / mokhtari, neel, kaiser & lee 299 a s i s appendix b tutor self-rating of lesson implementation fidelity protocol please take a few minutes following each lesson to share insights regarding lesson implementation. tutor: ______________________ start time: ______ a.m. p.m. tutee:_______________________ end time: ______a.m. p.m. open-ended comments: 1. please describe aspects of this lesson that worked particularly well. 2. please describe aspects of this lesson that did not work well. 3. please describe what you will do next to address the aspects of the lesson that did not work as planned. implemented comments familiar reading (15 minutes) 2.5 minute break interactive reading (20 minutes) 2.5 minute break interactive writing (20 minutes) modified as is no no as is modified modified as is no international electronic journal of elementary education vol.7, issue 3, 281-300, 2015 300 a s i s appendix c lesson fidelity of implementation observation protocol observer: ______________________________ tutor: _______________________ start time: ______ a.m. p.m. tutee: _______________________ end time: ______ a.m. p.m. overall lesson quality: overall lesson pacing: 3= outstanding 3= too fast 2= fair 2= about right 1= needs improvement 1= too slow open-ended comments: 1. describe aspects of the lesson observed that are particularly strong. 2. describe aspects of the lesson observed in need of improvement. 3. describe recommendations for improvement. implemented comments familiar reading (15 minutes) 2.5 minute break interactive reading (20 minutes) 2.5 minute break interactive writing (20 minutes) modified as is modified no modified no as is as is no international electronic journal of elementary education, december 2016, 9(2), 473-486. issn:1307-9298 copyright © iejee www.iejee.com a parent-oriented approach to rapid toilet training * dai doan a karen a. toussaint a ** a university of north texas, usa received: august, 2016 / revised: october, 2016/ accepted: november, 2016 abstract the current evaluation assessed the effectiveness of a rapid toilet training procedure for three young males with autism. the evaluation extended the research on rapid toilet training procedures by assessing parents’ preference to include two common toilet training components, a urine alarm and positive practice. in addition, we assessed child challenging behaviors during intervention. all parent participants’ elected not to use the urine alarm, and one parent elected to discontinue the implementation of positive practice techniques. all child participants engaged in challenging behavior with the initiation of toilet training. the toileting intervention was successful as all three participants increased successful self-initiations for the toilet and decreased accidents across home and clinic settings. all parents provided favorable social validity ratings of the treatment. findings suggest that clinicians should partner with parents to develop individualized toileting interventions that are appropriate and effective. keywords: autism, rapid toilet training, social validity, challenging behavior. introduction successful toileting is a developmental milestone generally achieved within three years of age (schum, et al, 2002). successful toileting improves an individual’s quality of life as an individual can independently toilet, and it decreases the risk of urinary and bowel problems associated with incontinence (bakker, vangool, vansprundel, vanderauwera & wyendale, 2002; barone, jasutkar, & schneider, 2009). toileting also provides indirect benefits, such as allowing individuals to fully participate in social activities, and reduces stigmatism associated with incontinence (cicero & pfadt, 2002). however, for individuals diagnosed with autism spectrum disorder (asd), toileting is often significantly delayed or simply not achieved (williams, oliver, allard & sears, 2003). azrin and foxx (1971) developed one of the most well-known and effective toileting procedures known as the rapid toilet training (rtt) method (kroeger & sorensenburnworth, 2009). rtt procedures were originally evaluated with nine, institutionalized * author note: this research was completed in partial fulfillment of the m.s. in behavior analysis at university of north texas for the first author. ** corresponding author: karen toussaint, department of behavior analysis, university of north texas, denton, tx 76203. e-mail: karen.toussaint@unt.edu http://www.iejee.com/ international electronic journal of elementary education vol.9, issue 2, 473-486, december, 2016 474 adults with intellectual disabilities. within four days of the program, toileting accidents decreased to nearly zero, and maintenance probes demonstrated participants maintained toileting success at 140 days post-treatment. the rtt procedure consists of multiple components, including: (a) frequently-scheduled toileting opportunities, (b) increased fluid intake, (c) use of a urine alarm, (d) positive reinforcement for continent voids and for staying dry during scheduled checks, (e) and restitutional overcorrection and punishment contingent upon toileting accidents. subsequent adaptions of the rtt procedures have been have been successfully applied to both children and adults with and without intellectual developmental disabilities, including asd (didden, sikkema, bosman, duker, & curfs, 2001; foxx & azrin, 1973; kircaali-i̇ftar, ulke-kurkcuoglo, cetin, & unlu, 2009). these adaptions have largely retained the original components of azrin and foxx’s (1971) rtt procedures but with modification. for example, leblanc, carr, crossett, bennett, and detweiler (2005) evaluated a rtt toileting program for three children with autism and made several modifications to the original procedures. first, caregivers participated in the toileting program alongside clinicians in order to promote maintenance and generalization of toileting skills in the home setting. in lieu of restitutional overcorrection procedures, leblanc et al implemented a positive practice procedure that consisted of stating “no wet pants” and quickly guiding participants to the toilet for a brief sit and repeating the procedure four additional times. also, the researchers conducted functional communication training to facilitate independent requests to toilet. finally, they employed an intensive sit schedule, but the sit schedule progressively adjusted so that participants were prompted to sit on the toilet less frequently with successful urinations. as a result of the program, all three participants achieved continence that maintained at a 1-month follow-up, and two of the three participants independently requested to toilet. despite the success of rtt procedures, researchers have raised concerns regarding the inclusion of components that may be considered restrictive consequences for individuals with developmental disabilities. caregivers may consider punishment procedures, such as restitutional overcorrection and time-out, unacceptable to use and difficult to implement (cicero & pfadt, 2002; post & kirkpatrick, 2004). in addition, overcorrection procedures may produce negative emotional side effects in children during toilet training; however, these undesirable side effects are often anecdotally reported and not directly measured (matson & ollendick, 1977). given concerns with restitutional overcorrection procedures, positive practice techniques have been suggested as a substitute. positive practice techniques focus on teaching the responses an individual should engage in rather than decreasing inappropriate behaviors, as in restitutional overcorrection (carey & bucher, 1983). researchers have successfully utilized variations of positive practice components during toilet training programs. cicero and pfadt (2002) evaluated a toilet training procedure with three children with autism where teachers delivered a brief reprimand and then quickly guided the participants to the toilet for an opportunity to eliminate. leblanc et al. (2005) implemented a positive practice procedure that consisted of stating “no wet pants” and quickly guiding participants to the toilet for a brief sit and repeating the procedure four additional times. in both evaluations, all participants achieved continence within two weeks of training. although positive practice techniques may be considered relatively less intrusive than restitutional overcorrection procedures, caregivers may have similar concerns. positive practice techniques are considered a punishment procedure as they are implemented contingent upon toileting accidents to decrease the future probability of accidents (kroeger & sorensen-burnworth, 2009). a parent-oriented approach to rapid toilet training / doan & toussaint 475 to increase the acceptability of toileting procedures for families, researchers have also recommended eliminating special equipment requirements, such as the urine alarm (cicero & pfadt, 2002). eliminating special equipment from toileting protocols may decrease burdens on the family by eliminating the financial costs of purchasing an alarm as well as providing a simplified toileting program. urine alarms are often included in toileting programs to alert the trainer that an accident is occurring so that any consequences can be implemented (azrin & foxx, 1971). in addition, the alarm sound may also startle the participant and temporarily stop the flow of urine which allows for a subsequent, successful completion of the void in the toilet. it may also help children to discriminate when an accident is occurring by increasing its saliency. however, previous evaluations have successfully toilet trained children without the use of a urine alarm which highlights that urine alarms are not necessary to achieve toileting success (post & kirkpatrick, 2004; simon & thompson, 2006; tarbox, williams, & friman, 2004). numerous toileting programs exist that have been modified to improve both caregiver and child satisfaction while maintaining toileting success; however, caregivers have not typically provided input or exercised control over the treatment procedures during the course of the program. given that one of the defining features of applied behavior analysis is social significance, it is critical that caregivers validate the goals, procedures and effects of an intervention (wolf, 1978). fawcett (1991) suggested a model of applied research that facilitates a genuine collaboration between the researcher and the participant and recommended that, “interventions are designed, adapted, and implemented in collaboration with participants.” this model ensures that caregivers’ preferences are incorporated at each stage of the treatment process. therefore, the primary purpose of the current evaluation was to create an individualized toileting program for families with young children with autism. we based our toilet training protocol upon the leblanc et al. (2005) rapid toilet training program which focused on caregiver participation. however, we individualized the protocol by allowing caregivers to select the inclusion of toilet-training components, the urine alarm and positive practice component, based upon individual preference. the second purpose of our evaluation was to determine if any problem behaviors occurred with the use of a rapid toilet training model. finally, we assessed the social validity of the toileting program that had been individualized based on caregiver preference. method participants three children diagnosed with autism spectrum disorder who were currently receiving approximately 30 hours a week of early intervention services at a university-affiliated autism center participated in this study. participants were previously diagnosed by clinicians independent of the autism center and researchers in this evaluation. all participants’ treatment plan included toilet training as a goal. peter was 2 years and 6 months old and communicated using 3 to 4 word sentences. he independently requested preferred items and activities. peter did not have a history of engaging in challenging behavior. he wore diapers throughout the day and occasionally requested to use the bathroom. clinic staff changed his diapers every 2 hours, but they did not sit him on the toilet during diaper changes. his mother reported attempting to toilet train him, but she did not follow any specialized protocols. peter’s mother and clinic staff served as therapists in the evaluation. john was 5 years and 10 months old. he primarily used two-word phrases to request preferred items and activities, and also engaged in echolalia. john did not engage in international electronic journal of elementary education vol.9, issue 2, 473-486, december, 2016 476 challenging behavior. john wore diapers throughout the day and never independently requested to use the bathroom. sits on the toilet were scheduled every 2 hours, but he never voided in the toilet successfully. john’s parents reported that they previously attempted to potty train him using procedures based upon azrin-foxx’s methodology (foxx & azrin, 1973); however, they report discontinuing the training and described it as “too intense.” john’s mother and father, as well as his regular clinic staff, served as therapists. elijah was 4 years and 8 months old. he primarily communicated through the proloquo2go application on an ipad, but he also vocally communicated using either single words or single-word approximations. elijah had a history of engaging in aggression, specifically hair-pulling, during instructional activities. elijah wore diapers and clinic staff changed his diapers every 2 hours without any sits on the toilet; however, elijah’s parents sat him on the toilet at home but not on any time-based schedule. elijah’s parents also reported that they previously attempted to potty train him using a set of procedures based upon the azrin-foxx methodology (foxx & azrin, 1973); however, elijah began having accidents as the sitting schedule was thinned so they discontinued the toileting procedures. elijah’s mother and clinic staff served as therapists in the evaluation. settings and materials clinic. clinic sessions were conducted in a 7’x7’ bathroom which contained a regular-size toilet, a child-size toilet seat adaptor, and a sink. during the time off the toilet, participants were allowed access to any clinic rooms (e.g. classroom or gym) that were within 15 feet of the bathroom. home. home sessions were conducted in participants’ home bathrooms. bathrooms contained at least one regular-size toilet, a child-size toilet seat adaptor, and a sink. during the time off the toilet, participants were free to go anywhere in the house. materials. both settings contained a bag that included participants’ change of clothes, wipes and toilet paper, preferred leisure items, and preferred edible items and beverages. response measurement and interobserver agreement dependent variables. researchers, clinic staff, and/or parents collected data using a pen and paper data collection system. we collected data each day for all waking hours, except during car rides when parents would place a diaper on the child. we collected frequency data on successful voids (or urinations), toileting accidents, selfinitiations, and challenging behavior. a successful urination was defined as any amount urine produced into the toilet; any amount of urination outside of the toilet was scored as an accident. thus, it was possible for a child to have both an accident and success in close succession if the child began to have an accident but continued to then void in the toilet (or vice-versa). a successful self-initiation was defined as participants’ communicative request to use the bathroom, or if the child entered the toileting area, followed by a successful void. we did not score successful self-initiations if the participant requested to use the toilet, but did not have a successful void. to assess toileting success, we evaluated the frequency of accidents and percentage of selfinitiated success, which we calculated by dividing the total number of self-initiated successes by all successes and then multiplying by 100 to convert into a percentage. to evaluate any potential negative side effects of the program, we collected data on the frequency of challenging behavior which we defined as any instances of crying, screaming, flopping to the ground, throwing items, self-injurious behavior or aggression. a parent-oriented approach to rapid toilet training / doan & toussaint 477 interobserver agreement and treatment integrity. independent observers collected interobserver agreement (ioa) and treatment integrity measures in home and clinic settings. clinic staff and parents were trained to collect data approximately one week prior to the initiation of baseline and therefore served as primary data collectors. ioa was collected for at least 30% of days across baseline, toilet training, and follow-up phases. we calculated total ioa each day by dividing the number of agreements by the total number of agreements and disagreements and multiplying by 100. total agreement was 100% for all three participants across all phases. total ioa for challenging behavior was 93.7% for peter, 98.2% for john and 100% for elijah. we collected data on treatment integrity to ensure correct implementation of toilet training procedures during the six clinical hours on the first day of intervention and for a minimum of two hours at home on the second day of training. we measured five components: (a) prompting the child to engage in communicative response before each sit or bringing child to the bathroom if he or she self-initiates, (b) prompting child to sit for the duration of the sit or until a successful void occurs, (c) providing moderately preferred items during the sit period, (d) immediately delivering highly-preferred items contingent upon successful void, (e) and implementing positive practice procedures contingent upon an accident, if applicable. treatment integrity was 100% across both settings for all three participants. experimental design and procedures we used a nonconcurrent multiple baseline across participants to evaluate the effects of the toileting intervention. we collected baseline data for three days for peter, 6 days for john, and almost 2 months for elijah. elijah had an unplanned, extended baseline due to illness and other family factors. parent interview. the second author met with parents to create a toileting plan using the brief interview form (see appendix). we first asked parents to nominate preferred items that may function as reinforcers during the intervention. these items were evaluated in a subsequent preference assessment. next, we described the toileting intervention and parents decided if they would include the urine alarm and positive practice components or not. none of the parents elected to use the urine alarm. parents stated that they did not want to include the alarm because: (a) the cost of the alarm was prohibitive, (b) there was concern that the alarm sounds would frighten the child, or (c) the alarm would be unbeneficial as the child was already aware of when they had a void. all parents agreed to implement the positive practice procedures as long as they could elect to discontinue the practice if the child seemed distressed. after the interview, the second author provided data sheets and written instructions on the toilet program. preference assessment. we conducted a multiple-stimulus without replacement with edible and leisure items to identify highly-preferred items to deliver contingent upon successful voids during intervention (deleon & iwata, 1996). we conducted three sessions. we began each session by presenting an array of six, equidistant items and delivering the instruction, “pick one.” once the participant selected an item, we removed that item and represented the stimulus array with the remaining items. we continued representing the array until all items were selected. we identified gummy worms as peter’s highlypreferred item and m & m’s® for elijah. john’s parents requested we include a choice of items, so john was presented with his top three items and allowed to select one (i.e. fruit snacks, jelly beans, and m & m’s®). baseline. consistent with ongoing toileting practices, all children wore diapers during baseline and were changed at least every two hours. in addition, john was prompted to sit at each diaper change. if any participants requested to toilet during baseline, we international electronic journal of elementary education vol.9, issue 2, 473-486, december, 2016 478 immediately escorted them to the bathroom. in addition, we delivered brief praise for successful voids. toilet training program. the current toileting program was based on the procedure described by leblanc et al. (2005) and consisted of: (a) scheduled sits, (b) reinforcement for successful voids, (c) communication training, (d) increased fluids, and (e) positive practice for accidents. no parents elected to use the urine alarm in this evaluation. all children discontinued wearing diapers except when sleeping and during car rides in the early days of training. scheduled toilet sits. all participants experienced a schedule of prompted toileting sits that progressed over 12 levels. during intersit intervals, participants were allowed to play in a nearby classroom or gym. participants progressed through the first six levels, one level each hour, on the first day of the program. the exception to this time-based contingency is that participants needed at least one successful void before progressing beyond level 3; if not, the schedule remained at level 3 until a successful void was achieved. on days 2 and 3, participants progressed one level each half-day as long as there were at least two successful voids. progression switched to a performance-based contingency at level 8; participants had to achieve 80% success for two consecutive days before progressing to the next level in the schedule (see leblanc et al. 2005 for a full description of the sitting schedule). we modified the sitting protocol for peter and changed his sit durations from 5 minutes to 3 minutes at level 8. we made this modification because peter consistently had a short latency to a successful void or he did not void at all. in other words, a void was unlikely to occur if it had not occurred within 3 minutes. the first six levels of the program were implemented in the clinic by parents and clinic staff with researchers present. clinic staff helped parents to implement the procedure for the first 2 hours, and parents implemented all procedures in the clinic by the third hour of the program. all procedures were implemented by parents in the home environment on days 2 and 3, and researchers visited the home on at least one of these days to assess treatment integrity. all participants returned to the clinic on the fourth day at level 8. reinforcement for successful void. parents or staff provided praise and access to each participant’s highly-preferred item contingent upon a successful void (or urination). after a successful void, participants were allowed to get off the toilet and the remainder time of the sit interval was added to the intersit (or recess) interval. if the participant selfinitiated to use the toilet, the adult immediately escorted the participant to the bathroom. however, participants only received access to the high-preference item contingent upon a successful void. increased fluids. children were provided with free access to a beverage that was nominated as preferred by caregivers. if participants did not independently consume the beverage, we encouraged drinking by offering the cup at least once every 30 minutes. communication training. parents and/or clinicians provided a model prompt for the child to request the toilet before walking to the bathroom and again right before they sat on the toilet. we emphasized to parents that prompts should not be given at any other times. the communication response for peter and john consisted of a vocal request, “go potty”. elijah’s communication response was pressing a symbol of a toilet on his ipad, which produced a voice output of “go potty”. if elijah did not press the symbol after the model prompt, the adult physically guided elijah to press the symbol positive practice. a positive practice procedure was implemented contingent upon an accident. on the first indication of an accident, the adult immediately stated “no wet a parent-oriented approach to rapid toilet training / doan & toussaint 479 pants” in a firm tone and quickly guided the child to the bathroom. children undressed themselves with adults’ assistance and sat on the toilet for about 5 seconds. after the brief sit, the adult provided assistance for the participant to redress and then returned participants back to the site of the accident. this procedure was repeated up to four times. the recess time was not reset when the accidents happened. however, the adults immediately stopped implementing positive practice if participants successfully urinated in the toilet and provided access to the highly-preferred item. peter’s mother discontinued the practice technique after the first implementation of all five trials and stated that peter was too distressed by the practice to continue. maintenance. we assessed maintenance of toileting behavior after participants completed the toilet training program. we collected follow-up data for peter at approximately four months post-training, at three weeks post-training for john, and at one week post-training for elijah. results figure 1 displays the percentage of successful self-initiated voids and the number of toileting accidents for all three participants through baseline, intensive training, and follow-up phases. the results of peter’s training are displayed in the top panel of figure 1. during baseline, peter would successfully request to use the toilet when undressed before taking a bath in the evening; however, his success was limited to this situation as he still had accidents throughout the day. we initiated toilet training on day 4. although peter’s mom elected to use positive practice, she decided to discontinue implementing the procedure after the first implementation and stated that peter was too distressed by the procedure. the percentage of successful self-initiations increased to 100% by the third day of toilet training and remained at 100% over the course of the toileting training program. peter completed the toileting program in 13 days. maintenance probes indicate that peter continued to self-initiate and had zero accidents up to four months post training. the middle panel of figure 1 depicts john’s toileting performance. during baseline, john did not successfully self-initiate and had an average of 3.3 accidents per day (sd = 2.2). we implemented toilet training on day 11 and john’s accidents initially increased above baseline levels, likely due to increased fluid consumption. accidents decreased below baseline levels by day 18 (the eighth day of the program), and john began to self-initiate on day 22 (twelfth day of the program). we examined toileting performance over the last three days of the program, and john had an average of 0.3 accidents (sd = .0.5) and 78.3% (sd = 7.9) self-initiated toileting successes. he completed the program in 26 days. a maintenance probe conducted at three weeks post-training indicated that john had 100% self-initiations and zero accidents. elijah’s results are displayed in the bottom panel of figure 1. before intervention, elijah did not successfully self-initiate and had an average of 3.3 accidents per day (sd = 1.5). similar to john, elijah’s accidents increased with the initiation of the toileting program; however, accidents decreased below baseline levels within nine days of the program. elijah successfully self-initiated on the first day of the program and began to consistently self-initiate by the tenth day of the program. elijah completed the toileting program within 29 days. on the last three days of the program, accidents decreased to an average of 0.3 (sd = 0.5) and successful self-initiations increased to 88.6% (sd = 8.4). a follow-up assessment revealed that elijah had 100% self-initiations and zero accidents at one week post-program completion. international electronic journal of elementary education vol.9, issue 2, 473-486, december, 2016 480 figure 1. the frequency of toileting accidents and percentage of self-initiated successes for peter, john, and elijah. the asterisk denotes the modification of the sitting schedule for peter. 0 20 40 60 80 0 5 10 15 20 0 20 40 60 80 100 accidents self initated success peter 0 20 40 60 80 # a c c id e n ts 0 5 10 15 20 p e rc e n ta g e o f s e lf-in ita te d s u c c e ss 0 20 40 60 80 100 days 0 20 40 60 80 0 5 10 15 20 0 20 40 60 80 100 john elijah bl toilet training maintenance 120100 140 * a parent-oriented approach to rapid toilet training / doan & toussaint 481 figure 2. frequency of challenging behaviors for peter, john, and elijah. the asterisk denotes the modification of the sitting schedule for peter. figure 2 shows participants’ challenging behaviors during toileting procedures. peter’s challenging behaviors are depicted in the top panel. peter had zero occurrences of challenging behavior (i.e. crying) during baseline. challenging behavior was observed with the initiation of toilet training and occurred primarily during toilet sits. on day 4, we decreased the sitting criterion from 5 to 3 minutes, and rates of challenging behavior decreased. challenging behavior remained at zero during maintenance. similar to peter, john did not engage in challenging behavior (i.e.) during baseline, but engaged in challenging behavior with the initiation of toilet training (middle panel of 0 20 40 60 80 0 5 10 15 20 25 30 0 20 40 60 80 # o f o c c u re n c e s o f c h a ll e n g in g b e h a v io r 0 5 10 15 20 25 30 days 0 20 40 60 80 0 5 10 15 20 25 30 peter john elijah bl toilet training maintenance 100 120 140 * international electronic journal of elementary education vol.9, issue 2, 473-486, december, 2016 482 figure 2). john had 21 occurrences of challenging behaviors on the first day of training and an average of 2.7 (sd = 1.6) occurrences of challenging behavior on the second through seventh day of the program. thereafter, his challenging behavior decreased over the course of the program, and there were zero occurrences during the maintenance probe. elijah’s challenging behavior is depicted in the bottom panel of figure 2. he did not engage in challenging behavior during baseline or the initiation of toilet training. elijah engaged in an average of 1.8 occurrences (sd = 1.2) of challenging behavior on the fourth through eighth day of toilet training, but challenging behavior returned to zero levels and maintained over the remaining course of the program and during the maintenance probe. social validity assessment. we assessed the social validity of the toileting program by obtaining participants’ parents evaluation of the toileting program using the treatment evaluation inventory-short form (tei-sf; kelley, heffer, gresham, & elliott, 1989). all three respondents rated the program favorably and the results are summarized in table 1. table 1. parents’ responses to treatment evaluation inventory – short form statement responses peter’s mom john’s mom elijah’s mom i find this treatment to be an acceptable way of dealing with the child’s problem behavior. strongly agree strongly agree strongly agree i would be willing to use this procedure if i had to change the child’s problem behavior. strongly agree somewhat agree strongly agree i believe that it would be acceptable to use this treatment without children’s consent. strongly agree somewhat agree neither agree nor disagree i like the procedures used in this treatment. strongly agree somewhat agree strongly agree i believe this treatment is likely to be effective. strongly agree strongly agree strongly agree i believe the child will experience discomfort during the treatment somewhat agree strongly agree somewhat disagree i believe this treatment is likely to result in permanent improvement. strongly agree strongly agree strongly agree i believe it would be acceptable to use this treatment with individuals who cannot choose treatments for themselves. strongly agree strongly agree somewhat agree overall, i have a positive reaction to this treatment. strongly agree strongly agree strongly agree a parent-oriented approach to rapid toilet training / doan & toussaint 483 discussion it is critical that practitioners and caregivers jointly develop and implement interventions that are provided for individuals with autism. this coordination affords families with an individually relevant intervention. in addition, professional guidelines for behavior analysts require that behavior analysts include caregivers throughout the development and implementation of interventions process (professional and ethical compliance code for behavior analysts 4.02, behavior analyst certification board, 2014). the current evaluation extends the research on toilet training individuals with autism by collaborating with caregivers’ to develop an individualized intervention protocol. we first identified components of an established toilet training program (leblanc et al., 2005) that the literature suggested are nonessential to toilet training success (cicero & pfadt, 2002; post & kirkpatrick, 2004). parents decided if they would include those components, specifically, a urine alarm and positive practice techniques, prior to implementing the toilet training intervention. no parents elected to use the urine alarm while all parents elected to use positive practice, although one parent discontinued the use of positive practice after one implementation. despite these modifications, our results are similar to those reported by leblanc in that all participants engaged in successful selfinitiations and decreased toileting accidents to near zero levels. these results maintained over extended periods of time. measures of social validity indicate that parents were satisfied with the training procedures as well as the outcomes. there were several components of the rapid toilet training protocol that we did not modify as previous research suggests that certain components are essential for toilet training success. greer, neidert, and dozier (2016) conducted a component analysis of toilet-training procedures and identified that combining (a) underwear, (b) a dense sit schedule and (c) differential reinforcement produced the greatest success; implementation of individual components was largely unsuccessful. however, greer did not include communication training in their evaluation, and several participants notably did not engage in self-initiations to toilet. thus, we advise that the combined components, in addition to communication training, should be retained in any individualized toilet training program. it should be noted that the sit schedule evaluated in greer et al. required shorter sit durations and longer intersit intervals (i.e. 3 min sit every 30 min) relative to the durations evaluated in the initial levels of the current study. additional research is needed to identify the relative effectiveness of different sit schedules on both toileting success and child challenging behavior. future evaluations should also assess parent preference amongst dense sitting schedules. the current evaluation extends the literature by assessing the effects of a rapid toilet training program on child challenging behavior. all three child participants engaged in challenging behavior within the beginning days of the toileting program. this may be due to the potential aversive nature of (a) long sits, (b) positive practice, or (c) some other component of the program, such as frequent transitions. we hypothesized that the sit duration was aversive to peter, so we modified the sit schedule and decreased the sit duration from 5 to 3 min. although his challenging behavior decreased, we did not systematically evaluate the effects of sit durations on challenging behavior; therefore we cannot conclude that longer sit durations were associated with higher levels of challenging behavior for peter. these results reveal that caregivers should anticipate some negative side effects with the implementation of rapid toilet training procedures. future research should assess procedures to mitigate these side effects, such as having the child sit on the toilet for brief durations prior to implementing a rapid toilet training procedure. international electronic journal of elementary education vol.9, issue 2, 473-486, december, 2016 484 one limitation of the current evaluation is that treatment integrity data was only collected in the early stages of the program rather than throughout the intervention (gresham, 1996). we wanted to ensure that the treatment was implemented accurately and consistently when the implementation was likely to be most difficult. although integrity was at 100% during these days, we cannot conclude that staff and parents continued to implement procedures with high integrity throughout the program. this study serves as a model of soliciting parents’ participation into a toilet training program for their children. the program was successful and parents were satisfied with the procedures and outcomes. we encourage providers and researchers to create familycentered interventions that allow parents to actively participate in the individualization and implementation of evidence-based programs for their children. • • • references azrin, n. h., & foxx, r. m. (1971). a rapid method of toilet training the institutionalized retarded. journal of applied behavior analysis, 4, 89-99. doi:10.1901/jaba.1971.4-89 bakker, e., van gool, j. d., van sprundel, m., van der auwera, c., & wyndaele, j. j. (2002). results of a questionnaire evaluating the effects of different methods of toilet training on achieving bladder control. bju international, 90, 456-461. barone, j. g., jasutkar, n., & schneider, d. (2009). later toilet training is associated with urge incontinence in children. journal of pediatric urology, 5, 458-461. doi:10.1016/j.jpurol.2009.05.012 bates, j. e. (1975). effects of a child's imitation versus nonimitation on adults' verbal and nonverbal positivity. journal of personality and social psychology, 31, 840-851. doi:10.1037/h0076677 behavior analyst certification board. (2014). professional and ethical compliance code for behavior analysts. retrieved from http://www. bacb.com/downloadfiles//bacb_compliance_code.pdf. carey, r. g., & bucher, b. (1983). positive practice overcorrection: the effects of duration of positive practice on acquisition and response reduction. journal of applied behavior analysis, 16, 101109. doi:10.1901/jaba.1983.16-101 cicero, f. r., & pfadt, a. (2002). investigation of a reinforcement-based toilet training procedure for children with autism. research in developmental disabilities, 23, 319-331. doi:10.1016/s08914222(02)00136-1 deleon, i. g., & iwata, b. a. (1996). evaluation of a multiple-stimulus presentation format for assessing reinforcer preferences. journal of applied behavior analysis, 29, 519–533. doi:10.1901/jaba.1996.29-519 didden, r., sikkema, s. p. e., bosman, i. t. m., duker, p. c., & curfs, l. m. g. (2001). use of a modified azrin-foxx toilet training procedure with individuals with angelman syndrome. journal of applied research in intellectual disabilities, 14, 64-70. doi:10.1046/j.14683148.2001.00047.x fawcett, s. b. (1991). some values guiding community research and action. journal of applied behavior analysis, 24, 621–636 a parent-oriented approach to rapid toilet training / doan & toussaint 485 foxx, r. m., & azrin, n. h. (1973). toilet training the retarded: a rapid program for day and nighttime independent toileting. champaign, il, us: research press. greer, b. d., neidert, p. l., & dozier, c. l. (2015). a component analysis of toilet‐training procedures recommended for young children. journal of applied behavior analysis, 49, 69-84. doi: 10.1002/jaba.275 gresham, f. m. (1996). treatment integrity in single-subject research. kelly, m. l., heffer, r. w., gresham, f. m., & elliott, s. n. (1989). development of a modified treatment evaluation inventory. journal of psychopathology and behavioral assessment, 11, 235-247. 235. doi:10.1007/bf00960495 kircaali-i̇ftar, g., ülke-kϋrkҫϋoğlu, b., ҫetin, ӧ, & ünlϋ, e. (2009). intensive daytime toilet training of two children with autism: implementing and monitoring systematically guarantees success! international journal of early childhood special education, 1, 117-126. retrieved from http://www.int-jecse.net/ kroeger, k. a., & sorensen-burnworth, r. (2009). toilet training individuals with autism and other developmental disabilities: a critical review. research in autism spectrum disorders, 3, 607618. http://dx.doi.org/10.1016/j.rasd.2009.01.005 leblanc, l. a., carr, j. e., crossett, s. e., bennett, c. m., & detweiler, d. d. (2005). intensive outpatient behavioral treatment of primary urinary incontinence of children with autism. focus on autism and other developmental disabilities, 20, 98-105. doi: 10.1177/10883576050200020601 matson, j. l., & ollendick, t. h. (1977). issues in toilet training normal children. behavior therapy, 8, 549-553. doi:10.1016/s0005-7894(77)80184-6 post, a. r., & kirkpatrick, m. d. (2004). toilet training for a young boy with pervasive developmental disorder. behavioral interventions, 19, 45-50. doi: 10.1002/bin.149 schum, t. r., kolb, t. m., mcauliffe, t. l., simms, m. d., underhill, r. l., & lewis, m. (2002). sequential acquisition of toilet-training skills: a descriptive study of gender and age differences in normal children. pediatrics, 109, e48-e48. doi: 10.1542/peds.109.3.e48 simon, j. l., thompson, r. h. (2006). the effects of undergarment type on the urinary continence of toddlers. journal of applied behavior analysis, 39, 362-368. doi: 10.1901/jaba.2006.12405taylor tarbox, r. s., williams, w. l., & friman, p. c. (2004). extended diaper wearing: effects on continence in and out of the diaper. journal of applied behavior analysis, 37, 97–100. http://doi.org/10.1901/jaba.2004.37-97 william, g., oliver, j. m., allard, a., & sears, l. (2003). autism and associated medical and familial factors: a case control study. journal of developmental and physical disabilities, 15, 335-349. doi:10.1023/a:1026310216069 wolf, m. m. (1978). social validity: the case for subjective measurement or how applied behavior analysis is finding its heart. journal of applied behavior analysis, 11(2), 203–214. international electronic journal of elementary education vol.9, issue 2, 473-486, december, 2016 486 appendix toilet training parent interview form 1) what are some of your child’s favorite items that we can use during toilet training to reward successes? please identify both leisure items (toys or activities) and edible items. 2) is your child aware of being wet or dry? if yes, what does he/she normally do after wetting the diaper? 3) urine alarms are often included during toilet training procedures. the alarms provide the benefit of helping the child to notice when s/he has an accident by producing the alarm sound. in addition, once the alarm sounds, the child may stop the flow of urine and this provides an opportunity to take the child to the toilet for him/her to have a successful void. however, some children may find the alarm unpleasant and/or startling. we recommend the use of urine alarms for children who do not seem to notice being wet or dry. would you like to include this component? 4) positive practice techniques are often included during toilet training procedures. this involves having the child repeatedly walk to the bathroom, undress, and briefly sit several times (up to five) if s/he has an accident. the benefit is that this may also stop the flow of urine and provides an opportunity to take the child to the toilet for him/her to have a successful void. however, some children may find these procedures unpleasant. you can elect to include the component or not include the component. if you elect to include the component and think that the child is too uncomfortable with the procedures, we can eliminate it immediately. would you like to include this component? 173 © 2022 published by kura education & publishing. this is an open access article under the cc byncnd license. (https://creativecommons. org/licenses/by/4.0/) copyright © www.iejee.com issn: 1307-9298 international electronic journal of elementary education january 2023, volume 15, issue 3, 173-185 one worldview to rule them all thomas zanea,*, mary jane weissb, joseph h. cihonc, ron leafd abstract introduction behavior analysis is the scientific study of behavior. radical behaviorism is the worldview of behavior analysts and this perspective drives how we think about behavior, assess the influences on behavior, and develop strategies to change behavior. there is evidence that some behavior analysts are using interventions that are not conceptually consistent with foundational principles, which results in using treatments not based on strong scientific evidence of effectiveness, and thus, are less effective and potentially harming the client. the reasons for this drift (from radical behaviorism) could be due to inadequate training in our philosophy, poor supervision, persuasion, and financial contingencies, among others. to maximize positive impact on the consumer, and to stay true to scientific roots, behavior analysts must adhere to the worldview of behavior analysis and radical behaviorism. by doing so, one’s work will be based on science, and thus will protect our science and, more importantly, the consumer. behavior analysis is the scientific study of behavior, as espoused by jones (1924), pavlov (1927), skinner (1938), thorndike (1898), watson (1913), and many others. due to the strict adherence to a worldview built on the philosophy of radical behaviorism which influences how research and practice are conducted, behavior analysis has made incredible gains in understanding human behavior. for example, behavior analysis has caused a paradigm shift in the treatment and prognosis of autism spectrum disorder (asd), where behavior analysis is recognized as the most effective treatment for this disorder (united states department of health and human services, 1999). not to be forgotten, though, is the extent to which the methodologies born from radical behaviorism has had a major impact on other areas, such as sports (e.g., luiselli & reed, 2011; tai & miltenberger, 2017), addiction (e.g., silverman et al., 2008), human safety (e.g., dickson & vargo, 2017; geller, 2005), space (e.g., brady, 2007), gerontology (e.g., dwyer-moore & dixon, 2007), juvenile delinquency (e.g., serna et al., 1986), education (e.g., keller, 1968), healthcare (e.g., friman et al., 1986), and sustainability (e.g., bekker et al., 2010). in whichever area that this philosophy (and the methodologies connected to it) has been applied, significant improvement has been accomplished. the power of behavior analysis comes from its foundation and philosophy of science keywords: behaviorism, radical behaviorism, worldview, conceptual consistency, evidenced-based practice received : 19 october 2022 revised : 23 january 2023 accepted : 20 march 2023 doi : 10.26822/iejee.2023.290 a,* corresponding author: thomas zane, department of applied behavioral science university of kansas, usa. e-mail: tzane@ku.edu orcid: https://orcid.org/0000-0001-5854-5837 b mary jane weiss, endicott college, usa. e-mail: mweiss@endicott.edu orcid: https://orcid.org/0000-0002-2836-3861 c joseph h. cihon, endicott college, autism partnership foundation, usa. e-mail: jcihon@apfmail.org orcid: https://orcid.org/0000-0001-9272-7749 d ron leaf, autism partnership, usa. e-mail: rlautpar@aol.com 174 january 2023, volume 15, issue 3, 173-185 (skinner, 1950; 1953). however, within the practice wing of behavior analysis, there are signs of drifting from the scientific core foundational worldview (e.g., schreck, et al., 2008; 2016; zane & ellis, 2014). put bluntly, individuals claiming to be behavior analysts seem to be using treatments and interventions that are considered ‘fad’ treatments (e.g., sensory integration therapy, hypnosis, facilitated communication; zane et al., 2016). by doing so, these individuals are discarding the adherence to quality evidence and research supported by the worldview that should have been developed when learning about behaviorism and behavior analysis. evidence of this drift is found from many sources. for instance, schreck and mazur (2008) surveyed board certified behavior analysts (bcbas) about their use of different types of autism treatment (some of those treatments were evidenced-based, and some were not). a total of 469 bcbas completed a series of questions about their use and belief in various interventions. the results showed that bcbas reported using all sorts of interventions, including those without evidence of effectiveness. more specifically, even though the majority of the respondents reported using applied behavior analysis (aba), discrete trial teaching (dtt), and the picture exchange communication system (pecs), many also reported using floortime, auditory integration training, facilitated communication, and gentle teaching, which is particularly concerning since they are based on a different conceptual perspective (worldview) of behavior and have little to no empirical evidence for their effectiveness. these respondents were also asked about their “beliefs” in whether or not the treatments were effective and to what extent that belief influenced use. at least some bcbas admitted using treatments even though they believed those treatments (e.g., sensory integration, floor time, facilitated communication) were not supported by scientific evidence. zane and ellis (2014) reached similar conclusions through an internet search for bcbas who practiced fad treatments such as sensory integration, relationshipbased therapies, and facilitated communication. in searching for various combinations of “bcba and [name of fad treatment),” zane and ellis found many bcbas who advertised themselves as using or advocating for these types of treatments for which there are no supportive research, such as holistic therapy, sensory integration training, cranio-sacral therapy, and relationship development intervention. more recently, schreck et al. (2016) pursued the reasons for drifting from a worldview built on the philosophy of radical behaviorism. a total of 848 certified behavior analysts including board certified associate behavior analysts (bcaba), bcbas, and doctoral-level bcbas (bcba-d) were surveyed about the extent to which they used a total of 22 various treatments (evidencedbased or not). schreck and colleagues found that respondents at each level of training (i.e., bcaba, bcba, and bcba-d) reported using some treatments that were evidenced-based, and ones for which there was no evidence. for example, some bcabas reported using aba, dtt, pecs, floortime, and sensory integration. some bcbas reported using aba, dtt, pecs, floortime, son-rise, facilitated communication, and rapid prompting method. some bcba-ds reported using aba, sensory integration, music therapy, facilitated communication, and rapid prompting. it was clear that behaviorally-trained interventionists were using treatment methodologies that were inconsistent with behavior analytic training and a radical behaviorism worldview. schreck and colleagues (2016) also asked why such treatments were used, analyzing various potential antecedent and consequent factors that might have been influential in leading certified behavior analysts to use and recommend various treatments. many respondents reported that training in aba and science, research methodology, and evidencedbased decision making factors influenced their use of aba and other scientifically-supported methodologies. however, when responding to the same questions regarding the use of treatments that had no or little empirical support, the respondents mentioned a number of influential factors. for example, between 61% and 68% of the surveyed certified behavior analysts said they were influenced to use floortime and music therapy by their supervisor in their supervised fieldwork experience. another 55% of bcbas admitted using floortime due to its popularity, and 70% of the respondents noted that persuasion from colleagues, clients, and/or parents influenced their use of ineffective treatments. a total of 59% of surveyed bcbas reported using the son-rise program due to an a priori belief that it was an easy intervention to implement. lastly, obtaining financial reimbursement was often noted as a factor related to the use of a treatment. schreck and colleagues (2016) emphasized their findings as a “wakeup call” for the field of behavior analysis. notwithstanding the training focus of science (e.g., determinism, empiricism, parsimony) and research design, behavior analysts are influenced by a myriad of other factors such as, but not limited to, persuasion, effort involved in implementation, and financial contingencies, even when these conflict with the evidence (or lack thereof) of the considered treatment. schreck and colleagues strongly advocated for improved training in behavior analysis, including research design, supervision, and to “thoroughly indoctrinate … students into the criteria for aba and what constitutes behavior analytic practice” (i.e., worldview; schreck et al., 2016, p. 374). 175 one worldview to rule them all / zane, weiss, cihon & leaf the behavior analyst certification board (2020) also provides evidence of the existence of a drift from a radical behaviorism worldview within practice by permitting bcbas to use treatments not supported by scientific research. first, the bacb ethics code for behavior analysts (2020) section 2.01 states: “behavior analysts implement nonbehavioral services with clients only if they have the required education, formal training, and professional credentials to deliver such services” (p. 10). additionally, certified behavior analysts can advocate for and implement nonbehavioral strategies if they use a disclaimer in their marketing materials. specifically, the behavior analyst certification board code of ethics states, behavior analysts do not advertise nonbehavioral services as behavioral services. if behavior analysts provide nonbehavioral services, those services must be clearly distinguished from their behavioral services and bacb certification with the following disclaimer: “these interventions are not behavioral in nature and are not covered by my bacb certification.” this disclaimer is placed alongside the names and descriptions of all nonbehavioral interventions. if a behavior analyst is employed by an organization that violates this code standard, the behavior analyst makes reasonable efforts to remediate the situation, documenting all actions taken and the eventual outcomes. (behavior analyst certification board, 2020, section 5.06, p. 16) therefore, a board certified behavior analyst can provide any therapy/intervention (e.g., astronaut therapy, floortime, social thinking) as long as they state they are not doing so under their board certification. this loophole (schreck et al., 2016) may preclude the certified behavior analyst from behaving skeptically by analyzing an intervention based on a radical behaviorism worldview and result in the selection of interventions based on other factors (e.g., persuasion, monetary). it should be noted that observation of drift is not new within the field of behavior analysis. for instance, branch and malagodi’s (1980) paper entitled, “where have all the behaviorists gone?” noted, “it wasn't so long ago that the spark of commitment to behaviorism glowed brightly. that spark is barely visible these days as repeated mentalistic micturitions have dampened it. mentalistic psychologists, against whom we were once so squarely pitted, have outwitted us” (p. 36). pierce and epling (1980) discussed the influx of other professionals in the field and that “these people bring with them many non-behavioral practices and concepts, and because of their large numbers become influential in redefining the field” (p. 4). finally, in his presidential address to aba in 1980, jack michael noted, “the bad news is that many people working in the applied field no longer have a strong background or much interest in the science of behavior, nor have an understanding or commitment to behaviorism” (p. 11). in this paper we suggest that behavior analysts’ use of nonbehavioral treatments is related to maintaining worldviews other than a radical behavioral one, and other worldviews can become problematic to behavior analysts who are trained in the worldview of science and radical behaviorism. we further assert that the scientific worldview is the only worldview behavior analysts should possess and under which they should operate professionally. the purpose of this paper is four-fold. first, we define worldview, describe what control a worldview exerts over how one views the world in which we live, and how a worldview dictates how we interpret the world and, in the case of behavior analysis, behavior. second, we outline a radical behaviorist worldview, and explain how that worldview allows us to conceptualize behavior, as well as its assessment and treatment, in a particular way. third, we provide examples of worldviews that may be incompatible or conflict with one of radical behaviorism, and discuss how those worldviews dictate methodological practices for assessing, explaining, and influencing behavior which are opposed to radical behaviorism and less effective due to a lack of the scientific foundation of radical behaviorism. fourth, we make the case that behavior analysts – who, by definition, have studied behaviorism should be ruled by the worldview of radical behaviorism in their work. in this paper, we contend that to be maximally effective, behavior analysts who are trained in behavior analysis should subscribe to a radical behaviorist worldview and all that that means. our intent is not to criticize other worldviews or their fundamental principles. indeed, some worldviews can positively inform and influence the practice of behavior analysis. in a notable example, malagodi (1986) discussed 10 ways in which cultural materialism is compatible with a behavior analytic worldview (e.g., “…it [cultural materialism] views selection by consequences as the principal mechanism for social organization and change…” p. 12). developmental psychology provides another example. don baer argued that the two fields, behavior analysis and developmental psychology, can be similar and, thus, benefit one another (morris et al., 1982). also consider the field of neuroscience. this body of knowledge is directly useful to behavior analysis, and there is strong evidence that information from both fields might very well result in increased understanding of behavior and innovative ways in which behavior can be modified not considered even a decade ago (see schneider, 2012 for a review). indeed, thompson (2007) powerfully asserted that behavior analysis must end the ‘biological-behavioral’ distinction and begin to incorporate systems inside the skin (e.g., nervous, cardiovascular, immunological) into analysis of behavior. these, and other, worldviews are compatible and may augment understanding and 176 january 2023, volume 15, issue 3, 173-185 the discussion within this paper is not in conflict with those circumstances. instead, this paper is concerned with how clarity and efficacy may be compromised by competing worldviews. what is a “worldview?” the word “worldview” comes from the german word weltanschauung, which combines welt, meaning world, and anschauung, meaning view or outlook. phrases such as, fundamental cognitive orientation, point of view, and how to look at the world and understand it, provide other nomenclature to the definition. a worldview is a particular stance or perspective that one has about a topic or concept, which then dictates how one views, talks about, and acts towards that topic or concept. rachlin (1980) described a worldview as an outlook on life which explains how and why we think and behave. morris (1988) wrote that worldviews give us “…criteria for evaluating meaningful research questions, appropriate research strategies, acceptable explanations for empirical findings, and adequate theories of development in general” (p. 290). essentially, a worldview provides a lens through which we interpret the things that go on around us. the worldview dictates certain assumptions about explanatory causes for the phenomena being studied, and then how to impact or influence those phenomena. behaviorally, a worldview may be conceptualized as a set of contingency-specifying stimuli that govern the behavior of an individual. for example, research documenting experimental control may function as a reinforcer for behavior analysts and thus behavior analysts may engage in behavior that is more likely to produce access to studies and other information that produce such reinforcers. as such, one’s worldview helps to determine the methods and procedures one uses to study a phenomenon as well as the interpretation of the results of a study. for example, medical physicians view behavior problems as essentially medical ones; the medical model is the worldview, the lens, through which they view a problem; this worldview then also dictates assessment and treatment. in summary, a worldview is a frame of reference that sets parameters on how to view a phenomena, approach the study of that phenomena, and how to impact that phenomena. strategies and tactics flow from the respective worldview. what is the worldview of behavior analysts?1 malagodi (1986) stated, “many of skinner's major theoretical works (skinner, 1948, 1953, 1954, 1957, 1968, 1969, 1971, 1972, 1974, 1978), taken together, may be conceptualized as comprising a ‘worldview’ (cf. michael, 1980) that integrates scientific philosophy and behavior principles into an epistemologically consistent general theory of human behavior” (p. 1). the worldview to which malagodi was referring is often termed behaviorism or radical behaviorism, and is the worldview of behavior analysts. in skinner’s (1963) own words, “behaviorism...is not the scientific study of behavior but a philosophy of science concerned with the subject matter and methods of psychology” (p. 951). behaviorism, then, is a philosophical position in which science is the foundational influence of how behavior is to be studied. note the strong influence of a scientific perspective. neuringer (1991) described behaviorism as “…associated with the philosophical position of determinism. behaviors are hypothesized to be functionally related to events, with those events external to the behaving organism most helpful in predicting and controlling behavior.” (p. 9). lastrucci (1967) was careful to point out that the word science connotes content and methodology; his definition of science was “…an objective, logical, and systematic method of analysis of phenomena, devised to permit the accumulation of reliable knowledge.” (p. 6). sagan (1996) referred to “… a ‘way of thinking’” (p. 25). so, our worldview of behaviorism is a philosophical and technological one. heward and cooper (1992) stressed the guiding assumptions of science (and of behaviorism) to be determinism and empiricism. cooper et al. (2020) offered this definition of science: a... systematic approach to the understanding of natural phenomena – as evidenced by description, prediction, and control – that relies on determinism as it fundamental assumption, empiricism as its prime directive, experimentation as its basic strategy, replication as its necessary requirement for believability, parsimony as its conservative value, and philosophic doubt as its guiding conscious (p. 7) when practicing science, one adheres to the attitudes and characteristics of science. common attitudes and characteristics are determinism, empiricism, experimentation, replication, parsimony, and philosophic doubt (cooper et al., 2020). these principles translate into the practices that have come to be known as a behavioral approach toward the study of behavior – an adherence to operationally defining the subject matter, precisely measuring the behavior of interest, relying on experimentation to determine causal relationships between variables and behavior, and practicing philosophic doubt, which essentially means to believe in the data, even if it means changing one’s position on beliefs, if new data challenges those beliefs. to what extent, then, is science and the scientific worldview related to aba? skinner (1938) laid out the basic principles and practices of his new science and how it would be applied to the study of behavior. baer et al. (1968, 1987) captured, for all time, the meshing of the two (i.e., science and aba) in the dimension of conceptual consistency. vargas (2004) went further, asserting that behavior analysis is a science in and of itself. cooper et al. (2020) supported this view describing aba as “…a science devoted to the 177 one worldview to rule them all / zane, weiss, cihon & leaf understanding and improvement of human behavior” (p. 2) and noted that other fields of study also have the goal of improving behavior. however, cooper and colleagues described behavior analysis as different due to its reliance and adherence to a scientific approach toward the study of behavior. any behavior analyst who has studied the field should have learned about the scientific approach. skinner (1963) was clear – behaviorism is the philosophy of science concerned with the study of behavior. behaviorism is the connecting empirical epistemology (skinner, 1963); it dictates what we study and how we study it. in other words, the philosophy of science that is behaviorism dictates the dimensions of what is studied (i.e., the properties of behavior) and the methods used to study them. this worldview has two major impacts. first, it concentrates behavior analysts’ focus on studying behavior qua behavior (i.e., for its own sake). this contrasts to studying behavior as a symptom indicator of some internal event that is, purportedly, the higher priority of study. the second impact comes in the form of informing behavior analysts where to look for the influences on behavior. this worldview dictates an assumption that behavior is a direct function of environmental variables occurring in temporal contiguity with the behavior. this worldview, then, dictates assessment and treatment. behaviorists focus on environmental variables (preceding and following the behavior of interest) and their functional relationship with the occurrence or nonoccurrence of a targeted behavior. thus, the radical-behavioristic worldview informs and influences the methods used when assessing and attempting to change behavior. there are many examples of this within the broad field of behavior analysis. for example, heward and cooper (1992) discussed innovative approaches in education stemming from our philosophy. they noted that there have evolved several behaviorally based educational systems, such as precision teaching (lindsey, 1991), programmed instruction (keller, 1968), and comprehensive application of behavior analysis to schooling (cabas; greer, 1991). these approaches toward educational practice stem from the worldview of behaviorism and science. similarly, assessment and treatment procedures for use with individuals diagnosed with intellectual and developmental disabilities, including autism spectrum disorder (asd), have evolved from a scientific worldview. for example, this unique behavioral worldview formed the foundation and practice of functional analysis of problem behavior (e.g., carr & durand, 1985; iwata et al., 1982/1994; lovaas & simmons, 1969) as well as descriptions of strategies to prevent the development of problem behavior altogether (e.g., ala’i-rosales et al., 2019). this worldview has led to many behaviorallybased treatments for individuals diagnosed with asd, including, but not limited to, discrete trial teaching (e.g., lovaas 1987; smith, 2001), noncontingent reinforcement (e.g., vollmer et al., 1993), and peer tutoring (e.g., kamps et al., 1994). skinner’s (1957) conceptualization of language is part of this as well, leading to effective procedures in that area of human development. lastly, a vast number of behavior analysts are working in business and industry, implementing behavioral safety’ approaches to managing occupational hazards and risks of injury in factories and industry. this approach is built upon the fundamental worldview of behaviorism and science, and has proven extremely effective (e.g., austin et al., 1996; geller, 2005; greene et al., 1987), similar to most other areas of application of interventions emerging from our worldview. in sum, behavior analysts view behavioral phenomenon through the radical behaviorist lens when analyzing and changing behavior. the radical-behavioral worldview governs the actions of behaviorists in ways that are consistent with the assumption that behavior is a function of observable and measurable environmental variables. once one adopts this worldview, it excludes explanations that are incompatible with that assumption and influences the assessment (i.e., searching for a functional relationship between antecedent/postcedent variables and behavior) and treatment/intervention practices (i.e., changing the relationship between behavior and environmental variables). thus, behavioral practices are informed by, and are consistent with, this worldview. traditionally, behavior analysts have come from a large number of disciplines including education, special education, psychology, social work, speech and language, counseling, business, and basic experimental analysis of behavior (foxx, 1996). nevertheless, behavior analysts are all bound together by this worldview consisting of foundational principles, conceptual underpinnings, and clinical practices. different worldviews of behavior throughout the history of humankind, there have been many attempts to explain the world and human behavior. many religions offer explanations for behavioral and other phenomena. for example, people who practice the religion of christian science (christian science, 2018) believe in god’s word in the form of the bible. the content outlined in the bible, as well as other materials, comprise a set of beliefs (i.e., worldview) that directly impacts how christian scientists interpret and lives in the world. they believe in the one christ, jesus, being the son of god. proponents of this religion have faith in the power of the holy spirit (christian science committees on publication, 1959). a fundamental belief is that everything originates with god and since god is perfect, humans cannot really and truly be injured, have mental health challenges, 178 january 2023, volume 15, issue 3, 173-185 or be ill (squires, 2018). their claim is that since god is all powerful and can directly impact our lives on a daily basis, simply putting a person with an illness or injury into “god’s hands” will result in god healing of that person (michell, 2014). specifically, proponents of this worldview avoid medical treatment for illnesses and injury because disease can be healed spiritually (wardell, 1965) without any supplementary implementation of medicine based on science; however, a recent contemporary view of some is that practitioners may make their own personal decisions about whether or not to consult medical professionals (paulson, 2014). another worldview can be found in the field of psychology. consider the area of psychodynamic (psychoanalytic) psychology or mentalism. this worldview is conceptualized as the existence of internal phenomena that either completely or partially explain behavior (sober, 1983). to put it another way, a mentalistic worldview assumes that internal events, unavailable for detecting, observing, or measuring, are not only causally related to behavior, but that failing to incorporate these inner constructs provides a woefully inadequate account of behavior-environment relations (e.g., flanagan, 1984). these internal (mental) states influence one’s view of why behavior happens (smithies, 2012). thus, the subject matter in a mentalistic/psychodynamic psychology consists of hypothetical constructs (see maccorquodale & meehl, 1948 for a discussion) that are unable to be observed (e.g., ainsworth, 1969), but are assumed to exist based upon the verbal reports (i.e., introspection) of the individual being treated. the concept of intrinsic motivation (e.g., ryan & deci, 2000), as opposed to extrinsic motivation (commonly associated with positive reinforcement in the behavioral sense), is associated with a mentalistic approach. assessment frequently takes the form of verbal dialog between patient and care provider, or by observing parent-child interactional patterns (e.g., greenspan & porges, 1984). another worldview that provides a conceptualization of behavior, and subsequent assessment and treatment based upon that conceptualization, is sensory integration (si) theory (ayres, 1972; 2005). the basic assumption of this theoretical model emphasizes the importance of the sensory system and how it processes incoming environmental stimuli (e.g., tactile, vestibular, proprioceptive). if the sensory system is normal and functioning properly, an individual reacts adaptively. however, if there is dysfunction in the processing of stimuli, the results can manifest in many ways, such as learning, behavior, or speech disorders (e.g., blanche et al., 2016; schaaf & miller, 2005). to improve the sensory functioning, there must be an abundance of the right type of sensory activity to improve the nervous system, to allow it to process stimulation appropriately, with the result of a reduction or elimination of behavior or learning problems (lang et al., 2012). this conceptualization of behavior leads to very specific assessment and treatment protocols. if the worldview dictates that behavior is a function of sensory processing, then assessment of a behavioral situation must focus on the sensory capabilities of the individual (e.g., ayres, 1972; dunn, 2002). for example, ayres developed the sensory integration and praxis tests to assess an individual’s ability in performing a variety of visual, tactile, kinesthetic, and motor tasks (kimball, 1990). in addition, very specific treatment strategies emerge from this unique conceptualization. specifically, sensory-rich activities must be provided, such as swinging, brushing, wearing weighted vests, and adaptive seating (e.g., bagatell et al., 2010; ferteldaly et al., 2001). because of the different conceptualizations of behavior across different worldviews, we assert that worldviews that include an alternative conceptualizations of behavior are incompatible with a radical behaviorist worldview and, thus, problematic as it relates to the practice wing of the field. for example, as previously noted, a mentalistic worldview assumes the existence of inner constructs or variables that produce overt behavior. skinner (1954) referred to these constructs as explanatory fictions, an apt term because, due to their undetectability, one must assume that these exist and then assume that they are somehow accountable for behavior. that particular belief is not part of the scientific enterprise that behavior analysts learn, should learn, or use to influence practice. instead, the behavioral training to which all behavior analysts should have been exposed should lead to the acceptance of the conceptualization of direct environment-behavior relations, which is consistent with the application of science to any field of study. since mentalism injects into the analysis of behavior entities such as schemata, cognitions, and the spiritual, the mentalistic worldview explaining and approach toward the study of behavior is contradictory to the scientific behavioral worldview. the sensory integration worldview of behavior is also incompatible with the fundamental beliefs and tenets of behaviorism. as noted previously, a sensory worldview is predicated on the assumption that behavior is a function of an intact biological organism that integrates external stimulation and internal processing, resulting in adaptive functioning. when there is a dysfunctional nervous system, the processing of sensory input is disrupted, resulting in behavior and learning disorders (ayres, 1972). the conceptualization of behavior from a sensory perspective does not acknowledge the influence of environmental variables on behavior. the sensory worldview does not adhere to basic attitudes and characteristics of science in studying phenomena that are observable and measurable. the core beliefs 179 one worldview to rule them all / zane, weiss, cihon & leaf of a sensory approach toward behavior does not rest on behavioral or scientifically proven principles. how competing worldviews affect practice the question is not, “does maintaining a competing worldview affect practice?” rather, the question is, “how does maintaining a competing worldview affect practice?” that is, it goes without question that attempting to hold a competing worldview comes with some compromises. what is likely to be of most interest to the practice wing of our field is how these compromises may affect decisions related to the selection and application of interventions. in our view, the effects are systemic and detrimental to the practice of behavior analysis. endorsing evidence-based practices as previously stated, a behavior analytic worldview is rooted in the tenets of science (e.g., determinism, empiricism, experimentation, replication, parsimony, philosophic doubt). with behaviorism rooted in science as the sole worldview, the practicing behavior analyst stays true to philosophic doubt and empiricism, decreasing the likelihood of recommending, endorsing, advocating, and/or implementing procedures/interventions without empirical support and/or with limited to no evidence. simultaneously maintaining a competing worldview (e.g., astrology) can result in the endorsement and/ or implementation of procedures with little to no evidence to their effectiveness (e.g., social storiestm), that are pseudoscientific (e.g., social thinking or floortime), or have the hallmarks of anti-science (e.g., facilitated communication, rapid prompting method). unfortunately, some behavior analysts are currently recommending, endorsing, and/or implementing these types of interventions (shreck et al., 2016). some of the rationales provided for doing so has included the need for behavior analysts to work collaboratively with other professionals (brodhead, 2015; kirby et al., 2021, that it does no harm, or to appease the requests of parents and other caregivers. these rationales are indicative of the conflict between worldviews. despite the rationale, selecting interventions with little or no evidence for effectiveness is likely to result in a less effective course of action by the behavior analyst. more effective, behaviorally based interventions could be delayed, or, worse, avoided altogether. furthermore, providing a disclaimer that it does not fall under the scope of certification (i.e., bcba) is a certification and practice solution, but it does not solve the core issue of distancing oneself from the fundamental behavioral worldview. however, it is important to note that the bacb does not certify worldviews; one is a behavior analyst or one is not. a disclaimer cannot negate a worldview – it simply ignores it. causal relationships within a behavior analytic worldview, an individual’s behavior is a product of/influenced by environmentalbehavior relations. in the case of respondent, or reflexive, behavior, responses are elicited by an antecedent event (commonly referred to as an unconditioned or conditioned stimulus). in the case of operant behavior, responses are evoked (i.e., an increase or decrease in probability) by antecedent events and strengthened (i.e., through reinforcement) or weakened (i.e., through punishment or extinction) by consequent events. all of these behaviorenvironment relations occur in the environment and observable, objectively measured events and behavior are used to determine functional relations and inform interventions. therefore, the practicing behavior analyst with this worldview identifies causes of behavior in the environment and not in other nonobservable events or stimuli (e.g., alleged sensory systems or processing centers in the brain, absence of social connectedness with a parent). attempting to maintain a worldview with a competing conceptualization of behavior (e.g., sensory integration theory), can result in the behavior analyst placing cause in other places. identifying causes of behavior antithetical to a behavior analytic worldview will inevitably affect the selection of an intervention. for instance, if one presumes a behavior is a result of a dysfunction in the processing of stimuli, then a processing-oriented intervention may be selected to address this dysfunction. this intervention would be in direct conflict with a behavior analytic conceptualization of behavior outlined by a behavior analytic worldview. worse still, and perhaps most importantly, treatments based on these incompatible worldviews are likely to be less effective than the treatments developed as a result of a radical behaviorism worldview that have proliferated and have been vetted to be effective. one worldview across one’s professional practice foxx (1996) asserted that once a behavior analyst commits “…intellectually to behavior analysis…” (p. 147), then that person has a responsibility to behave in adherence to the worldview espoused by the science, philosophy of radical behaviorism, and all that that entails. foxx argued that behavior analysts have a responsibility to behave in a way that will promote that science and philosophy, and act in ways to maximize its survival in the culture. palmer (personal communication, may 27, 2018) put it succinctly – “…once you buy into the assumption, it excludes explanations that are incompatible with that assumption.” skinner asserted that neither science or a philosophy of behavior can or should include or reference the existence of hypothesized internal constructs or processes inside the organism that would be used to account for behavior (harnad, 1988). this influences 180 january 2023, volume 15, issue 3, 173-185 how a behavior analyst would observe, study, and attempt to account for a particular behavior emitted by a specific individual. for example, a behavior analyst observes a person in a casino, an account of why such a person behaves as they do would not consider compulsions, drive states, or needs. instead, one would look to the immediate environment and also learn about the person’s history of reinforcement and punishment to explain current behavior patterns. behavior analysts adhering to a radical behaviorist worldview behave according to the latter and not the former. stoneman et al. (2013) addressed incompatible worldviews in medicine, specifically that of conventional medicine and complementary and alternative medicine (cam). conventional medical approaches are based upon strong science, in terms of its conceptualization of human health, as well as the assessment of health problems and the treatment of them. in contrast, cam is a catch-all term meaning medical treatments that have not been fully vetted empirically. stoneman and colleagues noted that although some cam therapies have been shown to be effective (e.g., massage), most of the treatments in this group are considered ineffective, faddish, and sometimes dangerous. stoneman and colleagues asserted that these two approaches are “fundamentally incompatible” (p. 5) with no unifying conceptualization of health and illness, and no compatibility among the methods used to approach healing. as they note, “one either follows the strictures of science and of evidenced based medicine, or one does not” (stoneman et al., 2013, p. 5). a behavioral worldview is based upon the pillars of science and scientific practice. other worldviews providing conceptualizations of behavior may not have such a strong orientation to science. one cannot easily embrace a scientific worldview on one day while practicing aba and then a nonscientific one on another day while practicing an alternative approach. such behavior must be explained. one possibility might be that this behavior analyst never obtained adequate training during their coursework to develop a thorough understanding of a radical behaviorist worldview, which may result in a lack of appreciation of its potential. if this is the case, then efforts to improve graduate training programs and working with those who approve those training programs and those who accredit those training programs may be fruitful. another possibility might be that the reinforcers and punishers associated with the contingencies for maintaining a radical behaviorist worldview have not been conditioned. for example, if experimental control, objective and observable dependent variables, and studying behavior for behavior’s sake have not been conditioned as a reinforcer it is likely that the practicing behavior analyst will be less likely to behave in ways that increase the likelihood of accessing those outcomes. a third explanation might be that competing contingencies are resulting in jumping from one worldview to another (and changing treatment approaches to coincide) based upon maximizing reinforcers. for example, there may be financial incentives for providing treatment that involves an intervention that does not align with a behavior analytic conception of human conduct. although maintaining a behavior analytic certification, a practicing behavior analyst may, in these cases, behave in ways to access monetary reinforcers while sacrificing effectiveness and science-based decisions making. a behavior analyst who disregards a behavioral worldview for a different, nonscientific one, probably does not accept that behavioral principles are, in fact, true principles. you either believe that cheese comes from milk, that miami is in florida, that the earth is sort of round, or you do not; there is no middle ground (d. palmer, personal communication, may 27, 2018). i have more than one worldview, now what? at this point, readers considering themselves behavior analysts may be reflecting on their own worldview(s) and whether their professional practice is impacted by incompatible worldviews. what are readers to do if they are attempting to maintain one or more worldviews in addition to that of radical behaviorism? while the adherence to more than one worldview might be possible under some conditions, there are some inherently incompatible perspectives that cannot be comingled. worldviews that consist of fundamentally opposed perspectives on the nature of behavior, and, as a result, about the selection of intervention cannot co-exist. for example, in the realm of autism intervention, it is not possible to support both a behavior analytic conceptualization of intervention and a sensory integration approach to treatment. perhaps the most important question readers should ask themselves is, “does the worldview provide an alternate conceptualization of behavior?” if the answer to this question is “no,” although it is not ideal, it may be possible to maintain this worldview in addition to radical behaviorism. if the answer is “yes,” that worldview may need to be abandoned completely to ensure effectiveness. that is, treatment approaches based on a radical-behavioral philosophy have been shown to be more effective than treatments based upon other worldviews. readers must ask how thoroughgoing of a behavior analyst does one want to be and will a less-than-thoroughgoing behavior analyst result in less effective practice and cause problems for the field at large? conclusion 181 one worldview to rule them all / zane, weiss, cihon & leaf the purpose of this paper was to provide rationales for a recommendation that practicing behavior analysts should adopt one, and only one, worldview to influence and guide their conceptualization of behavior and practice. that worldview, of course, is rooted in natural science and the application of science to the study of behavior. there are multiple worldviews that provide alternate conceptualizations of behavior. some may be attractive. the strategies and tactics suggested by other worldviews may be compelling and appealing in an intellectually-stimulating way. some behavior analysts may choose to implement interventions that stem from different worldviews due to financial or other contingencies (such as persuasion or a misunderstanding of research design and evidencedbased decision making). however, to do so would require drifting from the conceptualization of behavior espoused by skinner, watson, baer, wolf, risley, and many others. the behavior analytic worldview has led to incredible advances in the assessment and treatment of a vast array of behavior problems across all areas of human endeavor. the practical strategies stemming from this worldview work, and work well. adhering to a worldview that attempts to explain the origins of behavior differently from the scientific one is intellectually dishonest and is likely to be less clinically effective. one cannot truly believe that incompatible worldviews can both be correct. if we do not hold to that perspective and worldview, then we never really believed in it in the first place. footnotes 1it is not our intent to provide an exhaustive review of the definition of behavioral philosophy or worldview. there are many publications that do this in a very thorough and scholarly way (see morris, smith, & altus, 2005; moore, 2008; skinner, 1938; 1974, to name a few). instead, our aim is to provide a sufficiently detailed definition that orients the readers to the basic philosophy and parameters of the worldview of applied behavior analysis. conflict of interest: this paper was submitted as part of a special issue that the third author was asked to develop, solicit papers, and serve as an editor. the paper was sent to a different editor and was sent for blind review. compliance with ethical standards: no funding was used in the writing of this paper. ethical approval: this article does not contain any studies with human participants or animals performed by any of the authors. references ainsworth, m.d.s. (1969). object relations, dependency, and attachment: a theoretical review of the infant–mother relationship. child development, 969–1025. ala'i-rosales, s., cihon, j. h., currier, t. d. r., ferguson, j. l., leaf, j. b., leaf, r., mceachin, j., & weinkauf, s. m. (2019). the big four: functional assessment research informs preventative behavior analysis. behavior analysis in practice, 12(1), 222 234. https://doi.org/10.1007/s40617-018-00291-9 austin, j., kessler, m. l., riccobono, j. e., & bailey, j. s. (1996). using feedback and reinforcement to improve the performance and safety of a roofing crew. journal of organizational behavior management, 16, 49-75. ayres, a. j. (1972). sensory integration and learning disorders. los angeles: western psychological services. ayres, a. j. (2005). sensory integration and the child. los angeles: western psychological services. baer, d. m., wolf, m. m., & risley, t. r. (1968) some current dimensions of applied behavior analysis. journal of applied behavior analysis, 1(1), 91-97. baer, d. m., wolf, m. m., & risley, t. r. (1987) some stillcurrent dimensions of applied behavior analysis. journal of applied behavior analysis, 20, 313-327. bagatell, n., mirigliani, g., patterson, c, reyes, y., & test, l. (2010). effectiveness of therapy ball chairs on classroom participation in children with autism spectrum disorders. american journal of occupational therapy, 64, 895-903. behavior analyst certification board (2020). ethics code for behavior analysts. retrieved november 5, 2022 at https://www.bacb.com/wp-content/ uploads/2022/01/ethics-code-for-behavioranalysts-220316-2.pdf bekker, m. j., cumming, t. d, osborne, n. k. p., bruining, a. m., mcclean, j. i., & leland, l. s. (2010). encouraging electricity savings in a university residential hall through a combination of feedback, visual prompts, and incentives. journal of applied behavior analysis, 43(2), 327331. blanche, e. i., chang, m. c., gutierrez, j., & gunter, j. s. (2016). effectiveness of a sensory-enriched early intervention group program for children with developmental disabilities. american journal of occupational therapy, 70(5): 7005220010p17005220010p8. doi: 10.5014/ajot.2016.018481. 182 january 2023, volume 15, issue 3, 173-185 brady, j. v. (2007). behavior analysis in the space age. the behavior analyst today, 8(4), 398-412. branch, m. n., & malagodi, e. f. (1980). where have all the behaviorists gone? the behavior analyst, 3(1), 31-38. brodhead, m. t. (2015). maintaining professional relations in an interdisciplinary setting: strategies for navigating nonbehavioral treatment recommendations for individuals with autism. behavior analysis in practice, 8, 7078. dot 10.1007/s40617-015-0042-7. carr, e. g., & durand, v. m. (1985). reducing behavior problems through functional communicaction training. journal of applied behavior analysis, 18, 111-126. christian science. (2018). retrieved january 31, 2019 at https://www.christianscience.com/christianhealing-today christian science committees on publication (1959). facts about christian science. boston, ma: the christian science publishing society. cooper, j. o., heron, t. e., & heward, w. l. (2020). applied behavior analysis. third edition. upper saddle river, nj: pearson. dickson, m. j., & vargo, k. k. (2017). training kindergarten students lockdown drill procedures using behavioral skills training. journal of applied behavior analysis, 50(2), 407-412. dunn, w. (2002). infant/toddler sensory profile, manual. new york: psychological corporation. dwyer-moore, k. j., & dixon, m. r. (2007). functional analysis and treatment of problem behavior of elderly adults in long-term care. journal of applied behavior analysis, 40(4), 679-683. fertel-daly, d., bedell, g., & hinojosa, j. (2001). effects of a weighted vest on attention to task and self-stimulatory behaviors in preschoolers with pervasive developmental disorders. american journal of occupational therapy, 55, 629-640. flanagan, o. j. (1984). the science of mind. cambridge, massachusetts: mit press. foxx, r. m. (1996). translating the covenant: the behavior analyst as ambassador and translator. the behavior analyst, 19, 147-161. friman, p. c., finney, j. w., glasscock, s. g., weigel, j. w., & christophersen, e. r. (1986). testicular selfexamination: validation of a training strategy for early cancer detection. journal of applied behavior analysis, 19(1), 87-92. geller, e. s. (2005). behavior-based safety and occupational risk management. behavior modification, 29(3), 539-561. greene, b. f., winett, r. a., van houten, r., geller, e. s, & iwata, b. a (eds.). (1987). behavior analysis in the community: readings from the journal of applied behavior analysis. lawrence, ks: university of kansas. greenspan, s. i., & porges, s. w. (1984). psychopathology in infancy and early childhood: clinical perspectives on the organization of sensory and affective-thematic experience. child development, 55(1), 49-70. greer, r. d. (1991). teaching practices to save america’s schools: the legacy of b. f. skinner. journal of behavioral education, 1, 159-164. harnad, s. (1988). what are the score and limits of radical behaviorist theory? in a.c. catania and s. harnad (eds.), the selection of behavior: the operant behaviorism of b.f. skinner: comments and consequences. cambridge, ma: cambridge university press. heward, w. l., & cooper, j. (1992). radical behaviorism: a productive and needed philosophy for education. journal of behavioral education, 2(4), 345-365. iwata, b. a., dorsey, m. f., slifer, k. j., bauman, k. e., & richman, g. s. (1982/1994). toward a functional analysis of self-injury. journal of applied behavior analysis, 27, 197-209. jones, m. c. (1924). the elimination of children’s fears. journal of experimental psychology, 7, 382-390. kamps, d. m., barbetta, p. m., leonard, b. r., & delquadri, j. (1994). classwide peer tutoring: an integration strategy to improve reading skills and promote peer interactions among students with autism and general education peers. journal of applied behavior analysis, 27, 49-61. keller, f. s. (1968). "good-bye, teacher...". journal of applied behavior analysis, 1(1), 79–89. kimball, j. g. (1990). using the sensory integration and praxix tests to measure change: a pilot study. american journal of occupational therapy, 44, 603-608. 183 one worldview to rule them all / zane, weiss, cihon & leaf kirby, m. s., spencer, t. d., & ferron, j. (2021). how to be rad: repeated acquisition design features that enhance internal and external validity. perspectives on behavior science, 44(2-3), 389416. https://doi.org/10.1007/s40614-021-00301-2 lang, r., o’reilly, m., healy, o., rispoli, m., lydon, h., streusand, w., davis, t., kang, s., sigafoos, j., lancioni, g., didden, r., & giesbers, s. (2012). sensory integration therapy for autism spectrum disorders: a systematic review. research in autism spectrum disorders, 6, 1004-1018. lastrucci, c. l. (1967) the scientific approach: basic principles of the scientific method. cambridge, ma: schenkman publishing company. lindsey, o. r. (1991). precision teaching’s unique legacy from b. f. skinner. journal of behavioral education, 1, 253-266. lovaas, o. i. (1987). behavioral treatment and normal educational and intellectual functioning in young autistic children. journal of consulting and clinical psychology, 55(1), 3-9. lovaas, o. i., & simmons, j. q. (1969). manipulation of self-destruction in three retarded children. journal of applied behavior analysis, 2, 143-157. luiselli, j. k., & reed, d. d. (2011). behavioral sport psychology: evidenced-based approaches to performance enhancement. new york: springer. maccorquodale, k., & meehl, p. e. (1948). on a distinction between hypothetical constructs and intervening variables. psychological review, 55(2), 95–107. malagodi, e. f. (1986). on radicalizing behaviorism: a call for cultural analysis. the behavior analyst today, 9(1), 1–17. michael, j. (1980). flight from behavior analysis. the behavior analyst today, 3(2), 1–22. michell, d. (2014). healing and happiness in the christian science tradition. feminist theology, 22(2), 203-212). moore, j. (2008). conceptual foundations of radical behaviorism. cornwall-on-hudson, ny: sloan publishing morris, e. k. (1988). contextualism: the world view of behavior analysis. journal of experimental child psychology, 46, 289-323. morris, e. k., hursh, d. e., winston, a. s., gelfand, d. m., hartmann, d. p., reese, h. w., & baer, d. m., (1982) behavior analysis and developmental psychology. human development, 25, 340-364. morris, e. k., smith, n. g., & altus, d. e. (2005). b.f. skinner’s contributions to applied behavior analysis. the behavior analyst, 28, 99-131. neuringer, a. (1991). humble behaviorism. the behavior analyst, 14, 1-13. palmer, d. (2018). personal communication, may 27, 2018. paulson, s. (2014).the emerging face of being one: discerning the ecumenical community from the christian science church. journal of ecumenical studies, 49(2), 285-294. pavlov, i.p. (1927) conditioned reflexes. oxford university press, london pierce, w. d., & epling, w. f. (1980). what happened to analysis in applied behavior analysis? the behavior analyst today, 3(1), 1–9. rachlin, h. (1980). behaviorism in everyday life. englewood cliffs, nj: prentice-hall. ryan, r. m., & deci, e. l. (2000). intrinsic and extrinsic motivations: classic definitions and new directions. contemporary educational psychology, 25, 54-67. sagan, c. (1996). the demon-haunted world: science as a candle in the dark. new york: random house. schaaf, r. c., & miller, j. (2005). occupational therapy using a sensory integrative approach for children with developmental disabilities. mental retardation and developmental disabilities, 11, 143-148. schneider, s. m. (2012). the science of consequences. amherst, ny: prometheus books. schreck, k. a., & mazur, a. (2008). behavior analyst use of and beliefs in treatments for people with autism. behavioral interventions, 23, 201–212. doi:10.1002/bin.264 schreck, k. a., karunaratne, y., zane, t., & wilford, h. (2016). behavior analysts’ use of and beliefs in treatments for people with autism: a 5-year follow-up. behavioral interventions, 31, 355-276. doi: 1-.1002/bin.1461 184 january 2023, volume 15, issue 3, 173-185 serna, l. a., schumaker, j. b., hazel, j. s. & sheldon, j. b. (1986). teaching reciprocal social skills to parents and their delinquent adolescents. journal of clinical child psychology, 15(1), 6477, doi: 10.1207/s15374424jccp1501_8 silverman, k., roll, j. m, & higgins, s. t. (2008). introduction to the special issue on the behavior analysis and treatment of drug addiction. journal of applied behavior analysis, 41(4), 471-480. skinner, b. f. (1938). the behavior of organisms: an experimental analysis. new york: appletoncentury. skinner, b. f. (1948). superstition in the pigeon. journal of experimental psychology, 38, 168-172. skinner, b. f. (1950). are theories of learning necessary? psychological review, 57, 193216. skinner, b. f. (1953). science and human behavior. new york: macmillan. skinner, b. f. (1954). critique of psychoanalytic concepts and theories. science, 79(5), 300-305. skinner, b. f. (1957). verbal behavior. new york: appleton-century crofts. skinner, b.f. (1963). behaviorism at 50. science, 140, 951958. skinner, b. f. (1968). the technology of teaching. new york: appleton-century-crofts. skinner, b. f. (1969). contingency management in the classroom. education, 90, 93-100. skinner, b. f. (1971). beyond freedom and dignity. new york: knopf. skinner, b. f. (1972, july/august). humanism and behaviorism. the humanist, 32, 18-20. skinner, b. f. (1974). about behaviorism. new york: vintage books. skinner, b. f. (1978). reflections on behaviorism and society. englewood cliffs, nj: prentice-hall. smith, t. (2001). discrete trial training in the treatment of autism. focus on autism and other developmental disabilities, 16(2), 86-92. smithies, d. (2012). mentalism and epistemic transparency. australian journal of philosophy, 90(4), 723-741. sober, e. (1983). mentalism and behaviorism in comparative psychology. in d.w. rajecki (ed.), comparing behavior: studying man studying animals. (pp. 113-142). hillsdale, new jersey: erlbaum. squires, l. a. (2018).humble humbugs and good frauds: christian science and the anglo-american professions. nineteenth-century literature, 73(3), 353-378. stoneman, p., sturgis, p., allum, n., & sibley, e. (2013). incommensurable worldviews? is public use of complementary and alternative medicines incompatible with support for science and conventional medicine? plos one 8(1): e53174. tai, s. s. m., & miltenberger, r. g. (2017). evaluating behavioral skills training to teach safe tackling skills to youth football players. journal of applied behavior analysis, 50(4), 849-855. thompson, t. (2007). relations among functional systems in behavior analysis. journal of applied behavior analysis, 87, 423-440. thorndike, e. (1898). some experiments on animal intelligence. science, 7(181), 818-824. u.s. department of health and human services (1999). mental health: a report of the surgeon general. rockville, md: u.s. department of health and human services, substance abuse and mental health services administration, center for mental health services, national institutes of health, national institute of mental health. vargas, j. s. (2004). contingencies over b.f. skinner’s discovery of contingencies. european journal of behavior analysis, 5, 137-142. vollmer, t. r., zarcone, j. r., smith, r. g., & mazaleski, j. l. (1993). the role of attention in the treatment of attention-maintained self-injurious behavior: noncontingent reinforcement and differential reinforcement of other behavior. journal of applied behavior analysis, 26, 9-21. wardell, w. i. (1965). christian science healing. journal for the scientific study of religion,4(2),175-181. watson, j. (1913). psychology as the behaviorist views it. psychological review, 20, 158-177 zane, t., & ellis, n. (2014). they should know better: a description of the drift from our ethical code. presented at the annual meeting of the international association for behavior analysis, may, chicago. 185 one worldview to rule them all / zane, weiss, cihon & leaf zane, t., weiss, m. j., blanco, s., otte, l., & southwick, j. (2016). fads in special education. in controversial therapies for autism and intellectual disabilities: fads, fashion, and science in professional practice. by r.m. foxx and j. a. mulick (eds.), new york: london. 233 copyright © www.iejee.com issn: 1307-9298 © 2022 published by kura education & publishing. this is an open access article under the cc by ncnd license. (https://creativecommons.org/ licenses/by/4.0/) international electronic journal of elementary education january 2023, volume 15, issue 3, 233-245 a summary of the recommendations to increase cultural responsiveness in the field of applied behavior analysis sarah v. kristiansen* abstract introduction cultural contingencies help shape the development of an individual’s behavioral repertoire. researchers within field of behavior analysis have, within the past decade, reiterated the importance of connecting behavioral principles to a client's cultural practices. the incorporation of cultural preferences within behavioral services is paramount to the effectiveness and sustainability of those services. the present paper summarizes the guidance of researchers for practitioners beginning to develop this repertoire and apply it to their clinical practice. this paper begins by acknowledging that the foundation of the field of applied behavior analysis set the stage for understating how cultural considerations make the field applied. different themes related to increasing the cultural responsiveness of the field are summarized by reviewing the supporting literature. the research is further summarized into guiding principles for practitioners as they begin to sharpen this skill. the paper ends with a call for scientist practitioners and researchers to capture data related to culturally responsive service provision to inspire efforts in empirical evaluation of the recommendations provided. culture is the backbone of the science of behavior analysis. differentiation in cultures forms as the result of interactions between individuals which result in specific specific repertoire being shaped by social consequences (glenn, 2004). these interactions are responsible for a large portion of human learning and the combination of consequences over time within a specific context is what makes a culture unique (glenn, 2004). the cultural identity of specific group of individuals will determine behavioral nuances that are considered appropriate or inappropriate (e.g., shaking hands as a greeting, avoiding direct eye contact, hugging when welcoming a grandparent). understanding the existing cultural repertoires of a family, community, or social group gives a behavior analyst a starting point to develop a client-centered intervention plan. an emphasis on culture is a necessary component of behavior analytic intervention because cultural responsive keywords: cultural responsiveness, cultural humility, cultural awareness, applied behavior analysis received : 27 janaury 2023 revised : 15 february 2023 accepted : 15 march 2023 doi : 10.26822/iejee.2023.296 * correspondance details: sarah v. kristiansen, institute for applied behavioral science, endicott college, usa. e-mail: skristia@endicott.edu orcid: https://orcid.org/0000-0002-8857-6316 234 january 2023, volume 15, issue 3, 233-245 intervention practices can more readily support the naturalistic reinforcement of beneficial behaviors (neely et al., 2020). researchers have given the field a foundation upon which to start developing culturally responsive repertoires to serve diverse populations (arango & lustig, 2022; beaulieu et al., 2019; conners & capell, 2021; fong et al., 2016; fong et al., 2017; fong & tanaka, 2013; gatzunis et al., 2022; jimenez-gomez & beaulieu, 2022; kirby et al., 2022; mathur & rodriguez, 2021; najdowski et al., 2021; wright, 2019). the recent adoption of the new ethics code for behavior analysts (referred to as “the ethics code” here after) outlines that incorporating cultural considerations into clinical practice is not only recommended, but required if a certified behavior analyst is in compliance with this code (behavior analyst certification board {bacb}, 2020). arango and lustig (2022) identified specific areas within the ethics code that pertain to cultural responsiveness. within the core principles section, the code requires the equitable treatment of clients and stakeholders, “regardless of factors such as age, disability, ethnicity, gender expression/identity, immigration status, marital/ relationship status, national origin, race, religion, sexual orientation, socioeconomic status, or any other basis proscribed by law” (bacb, 2020, p. 4). also, within the core principles the ethics code outlines that certified behavior analysts should continuously make efforts to increase their own scope of competence related to serving clients of diverse backgrounds (bacb, 2020). the same themes are reflected in the ethical guidelines for members of the american psychological association (apa) and the american occupational therapy association (aota; aota, 2020; apa, 2017). the ethical guidance from both organizations places an emphasis on making efforts to understand and preserve an individual’s cultural identity when providing services. the focus on cultural components across fields that often interact with one another affirms the importance of these considerations. as arango and lustig (2022) remind us, an ethical approach to serving multi-cultural clientele is one that takes into consideration the relevant contextual variables within a family’s environment as well as the family’s treatment preferences. while practitioners await further recommendations within this area, they also anticipate the empirical evaluation of these suggestions. specifically, identifying the suggestions that are most valuable for supervisors, practitioners, researchers, and organizations to invest resources into. this paper will reflect on the foundations of applied behavior analysis (aba) and how those who were present during the field’s inception set the precedent for exploring this important issue. main themes and recommendations surrounding cultural responsiveness will be outlined with supporting articles identified for each theme. a review of the few empirical demonstrations of developing a culturally responsive repertoire will be discussed and recommendations for future researchers will be given. finally, the recommendations of some of the most cited articles which focus on cultural considerations will be combined to give practitioners guiding principles as they begin developing this repertoire. it is the intention of this paper to serve as a starting point for practitioners who recognize the importance of developing a culturally responsive practice but are unsure where to start within the literature. an additional goal is to encourage scientist practitioners and researchers to conduct empirical investigations on this topic to further guide the field of applied behavior analysis in these efforts. the terms used throughout this paper as well as other terms related to cultural responsiveness are outlined in table 1 to assist readers in discriminating between terms. articles that describe these terms are cited within the table. differentiation between terms is found throughout the published literature on the topic of cultural responsiveness (gatzunis et al., 2022; mathur & rodriguez, 2022); therefore, for the sake of clarity, the terms used frequently within this paper are clearly outlined. cultural responsiveness will be used when discussing overall incorporation of family preferences, values, and cultures into clinical work (mathur & rodriguez, 2022; miller et al., 2019). when referring to verbal behavior surrounding one’s own biases and how this might affect practice, the term cultural awareness will be used (arango & lustig, 2022; fong et al., 2016; gatzunis et al., 2022; martinez & mahoney, 2022). cultural humility will refer to the ongoing development and improvement of an individual’s cultural awareness (gatzunis et al., 2022; jimenez-gomez & beaulieu, 2022; kirby et al., 2022; martinez & mahoney, 2022; national institute of health, 2020). connection to the origins of aba the science of aba has long been one that values client input and incorporates client preferences (baer et al., 1968; wolf, 1978). an emphasis on embracing the preferences of those we work with is not new for the field of behavior analysis. baer and colleagues’ (1968) seminal article describes that the behaviors chosen for intervention should be important to the individual affected by intervention. baer and colleagues (1987) updated and expanded this original call for socially significant behavior change by reminding readers that social significance can be determined by both the individual experiencing intervention as well as those who support them throughout the experience (i.e., those who have likely helped shape the individual’s cultural identity). several current publications have linked the foundational dimensions described by baer et al. (1968, 1987) to a culturally responsive approach 235 ?? / kristiansen table 1 common terms used in discussions of cultural responsiveness in human services, behavior analysis and the medical field term definition references culture “the extent to which a group of individuals engage in overt and verbal behavior reflecting shared behavioral learning histories, serving to differentiate the group from other groups, and predicting how individuals within the group act in specific setting conditions” (sugai et al., 2012, p. 200). sugai et al., 2012 cultural sensitivity recognizing and embracing differences between the practitioner’s cultural background and that of their clients. understanding how this background can create biases and working these realizations into clinical practice and interactions. this process takes place both at an individual and organization-wide level. this term is often used to reflect verbal behavior that tacts cultural differences as well as overt behaviors that incorporate accommodations for those differences in service provision. arango & lustig, 2022 gatzunis et al., 2022 martinez & mahoney, 2022 cultural competence the repertoire of behaviors, policies, and values that are combined within an organization to allow professionals to work effectively across culturally diverse situations. this term is often used to describe the group of actions an individual or organization has taken to create systematic change. when discussed, authors describe culturally competent organizations as those that impact large, community systems by creating policy change, systems changes, and advocating for disadvantages populations. this term has not been used recently to reflect an individual’s behavior as it conveys an idea of reaching a level of “mastery”. arango & lustig, 2022 fong & tanaka, 2013 miller et al., 2019 cultural awareness often describes as verbal behavior. an individual can tact the differences between their cultural backgrounds and that of their clients and can also name the ways that these two backgrounds may be different and how those differences could affect services. arango & lustig, 2022 fong et al., 2016 gatzunis et al., 2022 martinez & mahoney, 2022 cultural and linguistic diversity “(t)he social (cultural) differences between groups that may control verbal behavior” (brodhead et al., 2014, p. 76). brodhead et al., 2014, p. 76 cultural humility describes ongoing process of evaluating and modifying one's own behavior following interactions with individuals of backgrounds different than one's own. continuously increasing openness to different viewpoints. the process of continuously improving upon one’s own cultural awareness. gatzunis et al., 2022 national institute of health, 2020 jimenez-gomez & beaulieu, 2022 kirby et al., 2022 martinez & mahoney, 2022 culturally responsive teaching incorporating the diverse experiences of both a student and faculty population into a program’s curriculum to ensure equitable educational experience for all students, regardless of background. gatzunis et al., 2022 jimenez-gomez & beaulieu, 2022, miller et al., 2019 cultural reciprocity the process of evaluating ongoing interactions between individuals of differing cultures. making changes and compromises in line with each party’s values as needed. kirby et al., 2022 cultural responsiveness the process of collaboration with clients and stakeholders to ensure intervention is reflective of the client’s values and goals. involves combining a cultural awareness repertoire with actively listening to the values and preferences of clients. mathur & rodriguez, 2022 miller et al., 2019 culturally and contextually relevant used when discussing educational curricula and materials and describes designing these supports to ensure that each student can experience the maximum amount of benefit from what is developed and taught sugai et al., 2012 note: the terms above have been defined based upon either direct quotes from authors studying a specific topic or from a combination of authors describing a specific term. the terms used most often throughout the current paper will include cultural responsiveness, cultural humility, and cultural awareness. 236 january 2023, volume 15, issue 3, 233-245 to services (e.g., beaulieu et al., 2019; brodhead, 2019; fong et al., 2016; jimenez-gomez & beaulieu, 2022; mathur & rodriguez, 2021; neely et al., 2020; sugai et al., 2012; syed et al., 2023; wright, 2019). baer and colleagues described that what sets the science of behavior analysis apart from similar ones is its applied nature which aims to make behavioral changes that are significant for the learner in their natural environment. an efficient and collaborative way of determining the importance of goals is to discuss them in collaboration with the client and take into consideration the client’s specific context and the ongoing contingencies that will support whether the goals are likely to maintain. baer and colleagues (1968, 1987) also identified that behavior analysts are analytic in their work. culturally aware practitioners demonstrate this dimension by understanding the variables within a client’s environment that sustain behaviors and might support behavior change (baer et al., 1968; brodhead, 2019). if client context is not taken into consideration, a behavior analyst might miss the influence of these variables. not including a family’s cultural background into behavior analytic decisions would be, as brodhead (2019) describes, “incongruent with our core value of being analytic” (p. 828). the importance baer and colleagues (1968, 1987) placed on social significance in aba has not recently been reflected in the field’s published work. ferguson and colleagues (2018) conducted a review of the journal of applied behavior analysis that assessed whether studies evaluated social significance in research by collecting measures of social validity. the results indicated that, from 1999-2004 only between 3-7% of articles assessed for social validity in some form. this number increased only slightly from 20072016 with an average 13% of articles reporting social validity measures. one dimension of social validity that closely aligns with culturally responsive intervention is the selection of goals based on their importance for the learner. ferguson and colleagues reported that this dimension was the least likely evaluated throughout the articles reviewed. this lack of focus on input from clients and stakeholders could lead the table 2 content areas related to cultural responsiveness and relevant citations area of interest publications that offer recommendations developing one’s own culturally responsive repertoire arango & lustig, 2022 dennison et al., 2019 fong et al., 2016 fong et al., 2017 gatzunis et al., 2022 machalicek et al., 2021 neely et al., 2019 wright, 2019 developing culturally responsive educational and organizational systems fong & tanaka, 2013 mathur and rodriguez, 2021 nadjowski et al., 2021 sugai et al., 2012 cultural considerations in supervisory relationships gatzunis et al., 2022 guidance for cultural sensitivity in family interactions, assessment, and protocol development beaulieu & jimenez-gomez, 2022 dennison et al., 2019 jimenez-gomez & beaulieu, 2022 martinez & mahoney, 2022 rohrer et al., 2021 neely et al., 2020 tereshko et al., 2022 cultural considerations in research procedures ruzycki & amhed, 2022 syed et al., 2023 commentaries on culture in behavior analysis arango & lustig, 2022 brodhead et al., 2019 conners & cappell, 2021 fong et al., 2017 hayes & toarmino, 1995 miller et al., 2019 lechago, 2022 wright, 2019 note: a sampling of literature across different areas related to cultural responsiveness in behavior analysis. this table is meant to serve as a guide for practitioners/ researchers beginning to develop repertoires related to cultural responsiveness. 237 ?? / kristiansen non-behavior analytic community to believe that aba has strayed from its original intention of putting clients at the center of services. this potential stray from incorporating client preferences and cultural differences into practice may stem from an over-generalization of the understanding that behavioral principles are universally applicable (hayes & toarmino, 1995). this might mean that practitioners create goals based on responses they see as socially concerning, but do not follow through by discussing these goals with the individual or their family. the goal is targeted because the analyst is confident that the behavior can be changed given the universal principles of behavior. however, hayes and toarmino (1995) explain that the most thorough analysis of a specific behavior will include analysis of the relevant components of the individual’s cultural identity. an understanding of a client’s culture can inform practitioners of a myriad of details important to programming (e.g., existing environmental arrangements, cultural norms, existing sources of reinforcement). goldiamond (1974) described a constructional approach to the field of behavior analysis as one in which the learner’s current repertoire is always the starting point in goal development. a culturally responsive program would take this same approach in incorporating a family’s priorities and existing repertoires into intervention and building skills from there (goldiamond, 1974). the importance of considering the diverse background of families is amplified by the immense changes occurring in the cultural makeup of countries. the most recent united states census reflects the rapidly changing demographics of the country. the 2020 census estimates that non-white individuals make up 38.4% of the country’s population (united states census bureau, 2022). this is a nearly 9% increase from the previous decade’s data. it is an enormous increase from the census conducted in 1970, soon after the release of baer and colleagues’ (1968) seminal publication which put an emphasis on goal relevance for clients. the 1970 census reported that individuals of color represented only 12.6% of the total population (singer, 2022). this is especially important given the mismatch between the racial makeup of the united states with the racial makeup of certified behavior analysts with only 21.5% of bcba or bcba-d certificants identifying as black, indigenous, or people of color (syed et al., 2023). the national conversation is going through a long-overdue shift on listening to the preferences people of color who represent almost half of the country's population. in response to this cultural shift, other fields have start to prioritize cultural responsiveness training for early professionals (jernigan et al., 2016). the field of behavior analysis is just beginning to accomplish the same training for professionals implementing the science. current recommendations start with self the recommendation from numerous publications on building a culturally responsive repertoire is to begin by evaluating one’s own cultural background and biases (arango & lustig, 2022; dennison et al., 2019; fong et al., 2016; fong et al., 2017; kirby et al., 2022; lechago, 2022; mathur & rodriguez, 2021; neely et al., 2020; wright, 2019). the new ethics code calls for practitioners to carefully evaluate their own cultural awareness before beginning a professional relationship by identifying areas of strength and opportunities for growth (bacb, 2020). practitioners can evaluate their own competence by reflecting on interactions with families, determining the appropriateness of those interactions, and making decisions about whether the relationship is a good cultural fit or if a colleague might better serve the client (bacb, 2020). this might be done by taking notes on an interaction with a family following an intake meeting that describe language used during the interaction, reactions observed, or comments made by the family. these notes could be reviewed before the next interaction and modifications made based upon the family’s responses. a practitioner could review these notes with a mentor they rely on for developing a culturally responsive repertoire. determining whether interactions are improving in terms of cultural responsiveness should be assessed by checking in with families and asking whether language used, family education strategies, and recommendations are in line with their preferences. this should be done frequently throughout a therapeutic relationship. recognizing and acknowledging one’s own biases can aid in a practitioner’s understanding of how their own environment may affect their approach towards a clinical case. reflecting on these biases can also help a practitioner see how their own circumstances differ from the circumstance of the family or client they are working with. identifying biases could also include a practitioner covertly labeling any assumptions that are made before and after interactions with potential clients. for example, when first arriving to a family’s home, the practitioner can ask themselves how the neighborhood the client lives in, the vehicle they see in the driveway, or even the report written by the diagnosing physician is influencing their assumptions about a family. tacting these assumptions and recognizing them as biases might lead to differences in overt behaviors displayed during interactions with the family. an evaluation of this type of bias might lead practitioners to create interventions that are more in line with a family’s cultural identification. in a brief example, a clinician who grew up eating dinner in a dining room, at a table with at least one other family 238 january 2023, volume 15, issue 3, 233-245 member, with sparse back and forth conversation might assume that a family’s goal of “eating a meal together” takes place in the family home in a similar manner. however, the family they are working with may have a full schedule and the ideal exemplar of eating a meal together for this family commonly resembles going to a busy restaurant, engaging in loud conversation, and laughing together at the end of the day. mathur and rodriguez (2021) warn that families might be hesitant to offer goals counter to the behavior analyst’s recommendations because the behavior analyst is seen as the “expert” and the family would not want to risk losing clinical support. a self-examination of bias could help the clinician create a protocol that, from its beginning, takes family preferences into consideration. fong and colleagues (2016) eloquently outline ideas on how professionals can embark upon this process. a sampling of these ideas includes discussing diverse client interactions with mentors or in small professional groups to gain feedback. fong and colleagues remind us that selfreflection and talking about one’s own behavior can help bring awareness to interactions that may have been inappropriate. fong and colleagues also emphasize the importance of being an active listener and recommend that, “practitioners hone their ability to attend closely to clients and self, in context” (p. 86). gatzunis et al. (2022) provided similar recommendations to those supervising mentees of diverse cultural backgrounds. gatzunis and colleagues offer a comprehensive checklist that supervisors can use for self-evaluation. the checklist, called the culturally responsive supervision self-assessment (crss) tool, reminds supervisors to self-reflect on how individual biases may influence interactions. the tool combines the cultural background of the supervisor, the supervisee, and the clients being served to make a respectful interaction for all parties more likely. the ubiquitous recommendation of self-reflection supports the idea that to truly learn about how other cultures might impact programming, one must first understand how their own background creates bias. creating culturally responsive systems several papers have emphasized the importance of beginning to teach cultural awareness at the inception of behavior analytic careers (e.g., beaulieu et al., 2019; fong et al., 2016; fong et al., 2017; mathur & rodriguez, 2021; najdowski et al., 2021). training cultural responsiveness from the beginning of a behavior analyst’s familiarity with the field would support simultaneously learning the importance of cultural identity and that behavioral principles are universally applicable. as an example, an early career behavior analyst learning about functionalassessment procedures while simultaneously being taught the importance of cultural considerations in assessment procedures (dennison et al., 2019) would be better situated to choose an assessment the client is comfortable with. initially teaching behavior principles through the lens of cultural humility could prepare practitioners to incorporate these considerations more readily into intervention decisions. mathur and rodriguez (2021) provide specific components that could be involved in the implementation of a culturally responsive curriculum. the curriculum outlines competencies that are specific to client interactions such as creating treatment goals as a result of collaboration with a family. also included are repertoires that reach beyond the impact that a clinician can have on a single family. mathur and rodriguez (2021) suggest behavior analysts engage in educational events specifically focused on the cultures within their community. further, actions to help develop equitable supports for individuals with disabilities within their community are outlined as competencies. these actions include outreach to local pediatricians, collecting data on referral demographics, and actively advocating for those experiencing oppression (mathur & rodriguez, 2021). the curriculum created by mathur and rodriguez was inspired through a review of similar curricula from other disciplines. the medical field, for example, has begun the process of evaluating the results of a culturally responsive curriculum specific to training of medical residents (jernigan et al., 2016). although there remains no empirically evaluated training on cultural responsiveness for behavior analytic practitioners (lechago, 2022), it is encouraging to see that thoughtful consideration of cirricula components is beginning. cultural considerations in supervisory relationships once a behavior analyst enters the workforce, the responsibility of continuing to develop a culturally aware skill set lies with the individual, their supervisors, mentors, and the organizations that employs them. additionally, it is the supervisor’s responsibility to understand the differences between their own as well as their supervisee’s cultural backgrounds. gatzunis and colleagues (2022) outlined the importance of culturally responsive supervision and developed the previously mentioned tool that could be used during a supervisory relationship to evaluate whether the guidance of the mentor is sensitive to the background of both the supervisee as well as the individuals receiving services. the tool incorporates ongoing evaluation throughout the supervisory relationship to detect changes in bias and culturally responsive behaviors (gatzunis et al., 2022). as with the first recommendation, this tool begins with the supervisor’s self-evaluation of their own cultural background and how this evaluation could potentially affect services. the ethics code (bacb, 2020) also acknowledges the importance of cultural responsiveness in supervision 239 ?? / kristiansen by outlining that supervisors evaluate their own scope of competence when supervising it is also required that supervisors incorporate topics related to diversity, equity, and inclusion into supervision experiences. further, it is the supervisor’s responsibility to ensure that their mentees are familiar with the current ethical code (bacb, 2020). this holds the supervisor responsible for contributing to the development of their mentees own cultural awareness (bacb, 2020; gatzunis et al., 2022). guidance for protocol and assessment development guidance has also been given to practitioners in developing specific interventions that incorporate the cultural preferences of clients. several commentaries begin by suggesting an open conversation and listening to individuals and the communities who support them throughout services (beaulieu & jimenez-gomez, 2022; kirby et al., 2022; lechago et al., 2022; mathur & rodriguez, 2021). hayes and toarmino (1995) describe the importance of understanding the functionality of behavior both from an individual as well as a community (i.e., cultural) perspective. following information gathering, clinicians should ensure that the goals selected for intervention are aligned with the values of the client as well as their stakeholders and are appropriate given the client’s context (e.g., workplace, community, family dynamic; fawcett, 1991). a clinician’s ability to collect this information will vary across the different skill sets of individuals being served. if a clinician is working with an individual with limited communication skills, specific assent procedures could be discussed with the family to ensure that client preferences are being prioritized during intervention. if a client or family disagrees with a clinician regarding the implementation of protocol that would be detrimental to the client’s ultimate goal, practitioners should take the time to discuss these contingencies collaboratively. in an example, if a client is hesitant to begin an intervention to decrease facial skin picking in public settings but has an ultimate goal of going on a few successful dates, the clinician could discuss the interaction between the proposed goal and the ultimate goal to give the client more context regarding the clinician’s recommendation. beaulieu and colleagues (2022) reiterate the importance of considering the feasibility of the maintenance and generalization of skills within the client’s natural environment. determining whether the program would sustain should be a key factor in considering whether it should be started. implementing culturally responsive strategies at an organizational level fong and tanaka (2013) provide focused recommendations for how behavior analytic organizations can support the effort to increase the cultural responsiveness of the field. these standards include practices to support the diversification of the work force by mentoring and supporting candidates from diverse backgrounds. recommendations for workplace activities such as ongoing training and clearly stated goals surrounding culturally responsive efforts were suggested. one recommendation fong and tanaka provide that has been echoed through clinical and research publications (e.g., brodhead et al., 2014; fong et al., 2016; jimenez-gomez & beaulieu, 2022) is that demographic data surrounding the cultural and ethnic makeup of the individuals an organization serves are imperative to capture. organizations should be thoughtful regarding the allocation of time and funds towards these initiatives, given their importance and the amount of time and effort that will go into creating systems-wide changes. fong and tanaka’s suggestion of creating and adhering to organizationwide goals surrounding diversification and cultural education can help ensure that the time taken in the process of developing these strategies is not placed solely on the individual clinician. fong and tanaka’s recommendation to, “ensure that staff at all levels and across all disciplines receive ongoing education and training in culturally and linguistically appropriate service delivery” (p. 18) would mean a giant organizational shift for those who do not already have similar systems in place. however, organizations can follow goldiamond’s (1974) advice, once again, and start with the current organizational repertoire to build towards an overall goal in incremental, meaningful steps. this could mean that an organization begins by adding culture-specific content/trainings to already scheduled meetings. after feedback is collected, the organizations could plan for future meeting time to increase the training opportunities for their staff. providing staff with resources and support in this effort should be considered part of an organization’s longterm continuing education commitment. a call for empirical investigation getting started a decade has now passed since fong and tanaka’s (2013) call for building stronger cultural alliances through behavior analytic services. an empirical question that remains to be determined is how these recommendations are impacting the repertoire of today’s behavior analysts and the clients they serve. research in this area will be difficult as the process of identifying and measuring relevant behavior change associated with the development of a culturally responsive repertoire can be a difficult undertaking (mathur & rodriguez, 2021). is it possible to measure and capture the shifts in a bias? how can we ensure that the actions taken by practitioners are seen from the family’s perspectives as genuine, rather than just a box to be checked (rohrer et al., 2021)? we might start with evaluating the suggestions provided in 240 january 2023, volume 15, issue 3, 233-245 the articles mentioned throughout this paper and determining those that make the strongest impact. we can also examine the example of another current issue within behavior analysis that is interdependent to cultural humility, that of compassionate care (lechago et al., 2022; rohrer et al., 2021). rohrer et al. (2021) summarized the status of the field of behavior analysis as it relates to compassionate care and suggested a checklist, called the compassion and collaboration tool, that could be used in guiding practitioner behavior in this area. the compassion and collaboration tool was informed by publications that taught compassionate care skills to those within health care and other human services fields. the checklist’s publication was followed by rohrer and weiss (2022) who evaluated whether these skills could be taught to pre-certification masters students. the researchers delivered behavioral skills training to teach different skill sets outlined by the compassion and collaboration tool. these skill sets included basic skills (e.g., nodding while listening, providing a positive introduction at the beginning of the interaction), showing interest in the family (e.g., asking about preferences, incorporating the caregiver’s feedback into the intervention), and joining in with the family (e.g., providing empathy statements and statements that demonstrate the practitioner will join in the family on therapeutic efforts; rohrer & weiss, 2022). rohrer and weiss had only four participants, but the results showed excellent progress within a multiple baseline design across skill sets. this study set the stage for continued research within this and similar areas. the same type of work has just recently begun around cultural responsiveness, providing hope to potentially inspire researchers and encourage this line of work to gain momentum (neely et al., 2020). neely and colleagues (2020) conducted one of the first assessments of a culturally responsive curriculum implemented by educators in a classroom setting. the curriculum described observable actions a teacher could make to support a classroom’s inclusion of cultural differences. these components included environmental details such as displaying classroom rules in the primary languages of the students and using diverse representations of authors and individuals included in pictorial prompts. additionally, teacher responses to students were evaluated as well as whether these responses were in the student’s primary language. a multiple-probe design evaluated the effects of training on the specific components of the curriculum. all teachers increased the percentage of curriculum components implemented in their classrooms following the training and satisfaction with curriculum implementation was generally high. neely and colleagues' study serves as a good starting point for researchers to begin empirical evaluations using similar methods. continued conversations and updated research methodology regarding culture will always be needed as cultures continually shift (lechago, 2022). commentaries and advice from experts in this domain should keep coming, but as it is generated, researchers should purposefully evaluate recommendations to guide and improve future efforts. operationalizing and measuring success specific behavior change would be the goal of the aforementioned research, but how can it be determined that the time committed to this effort has made a difference? the ultimate indicator should be intervention sustainability. families often have a variety of obligations within their daily lives and practitioners should make sure that an intervention is worth the response effort committed. if a family sees the value in the results of an intervention, it is aligned with their values, and able to be implemented with the resources they have available, the intervention is more likely to be sustained. additionally, evaluation should frequently be conducted as to the acceptability of the procedures. alai-rosales et al. (2022) suggest that clinicians frequently ask themselves, “do the people the clinician is working with feel happy about the progress that was made? are the families involved and did they participate throughout the process?” (p. 161). these questions asked throughout the duration of intervention can help guide the practitioner as to when reevaluation and adjustments are needed. to ensure that clinician and organization success in cultural responsiveness is not siloed to one demographic, several publications recommend categorizing social validity data collected according to the demographic data of clients (e.g., beaulieu et al., 2019; dennison et al., 2019; lechago, 2022). doing so would allow for satisfaction to be evaluated across client cultural identities to identify areas in which additional training may be needed. data could also be collected on referrals from other providers or families. this data may be helpful in determining if families or providers of certain cultural backgrounds are recommending an organization’s services to others more or less often. an additional suggestion when conducting social validity surveys is to ensure that the questions asked are those that will inform clinicians with respect to cultural responsiveness. for example, asking questions such as, “does this program reflect a goal that you and your family prioritizes?” or “do you feel comfortable implementing all aspects of this program?” would be more beneficial in the evaluation of cultural responsiveness when compared to, “are you satisfied with the level of behavior decrease/ increase?” these surveys should also be prioritized for translation to the family’s primary language. families should be given opportunities within these surveys to anonymously reflect upon the clinician’s demonstration of respect for their preferences and culture. as previously mentioned, specific behavior change to support cultural responsiveness is a good 241 ?? / kristiansen start, but not the end goal. the goal of all services should be to make a socially valid change. families who feel a practitioner has taken the time to understand their preferences and values are more likely to continue with implementation of recommendations (beach et al., 2006). services that incorporate culturally responsive practices give the best chance for family satisfaction and sustained behavior change (jimenez-gomez & beaulieu, 2022). guidance for practitioners the articles mentioned throughout this paper offer a variety of methods that could be used as guidance in developing a culturally responsive repertoire. when combining ideas from the foundation of the field of behavior analysis with the current theme of cultural consideration, five themes emerge that practitioners should consider as they embark on the process. be a behaviorist the first recommendation is that the guidance from a behavior analyst must follow the principles of the science. above all, it the duty of practitioners to adhere to the principles of behavior analysis and to be effective (baer et al., 1968; dennison et al., 2019; kirby et al., 2022). this does not mean that benign nonbehavior analytic recommendations are not tolerated, but that the practitioner is a behavior scientist with respect to their careful evaluation (bowman et al., 2021; newhouse-oisten et al., 2017). pseudoscientific methods are commonly found among suggested interventions designed to support individuals diagnosed with autism spectrum disorder (normand, 2008; travers et al., 2016). normand (2008) describes the importance of being skeptical in the evaluations of interventions suggested or preferred by a family. skepticism, in this sense, means carefully considering the research supporting suggested interventions and evaluating those interventions in a systematic way (normand, 2008). travers and colleagues (2016) describe skepticism as, “a repertoire of behavior that is developed gradually and in lockstep with scientific literacy” (p. 268). behavior analysts should listen to the preferences and suggestions of families from varying cultures, but also be skeptical in their acceptance and application of these recommendations. there are several publications that offer guidance in evaluating non-behavioral treatment preferences expressed by families. bowman and colleagues (2021) provide standards for transdisciplinary collaboration that include actions such acknowledging and avoiding pseudoscientific suggestions. newhouseoisten et al. (2017) guides practitioners through an ethical decision-making matrix that evaluates the use of evidence-based and non-evidence-based treatments. brodhead and colleagues (2015) highlight the importance of translating recommendations from parents and colleagues to the behavioral principles that might make up the intervention. a translation to behavioral principles can allow the behavior analyst to better understand the variables that could affect behavior. using this approach provides a behavioral lens for evaluating recommendations suggested by families and others. this is the lens through which all additional recommendations should filter. be flexible thanks to almost decades of research, there are a multitude of options for the selection of procedures that utilize behavior analytic principles. sticking to only a prescribed way to conduct a specific intervention is not a practice that reflects cultural responsiveness. families will rarely be familiar with the behavior analytic literature, and their confidence in the field’s effectiveness will likely not be as strong as their practitioners' (rosenberg & schwartz, 2019). competent professionals should also be comfortable in evaluating the work of another discipline through a behavioral lens as previously mentioned. this is, no doubt, a skill set that will need to be honed. the translation and evaluation of procedures that may not be well established within the behavioral literature could go a long way in demonstrating to a family that their recommendations are taken seriously (brodhead et al., 2015; jimenez-gomez & beaulieu, 2022). bowman and colleagues (2021) provide general guidance in this area and outline that overall, treatments to be implemented, “do not put the client in any danger and will not cause harm”, “are empirically supported,”, and “are effective” (p. 1201). developing a repertoire for evaluating and translating the research of other disciplines could help increase the flexibility of practitioners in collaborative intervention efforts with families. the suggestion of flexibility might seem antithesis to what the research supports. after all, some of the most well-regarded behavior analytic publications are those that suggest specific decision-making models that guide practitioners to select interventions given specific contexts (geiger et al., 2010). this does not mean that these tools are obsolete when working with families with different cultural backgrounds. rather, when using them, practitioners should overlay a family’s values and preferences onto the decision making matrices. preferably, a practitioner could conduct this decision-making process with the family to make a collaborative decision regarding protocol implementation (geiger et al., 2010). this will help ensure the intervention not only reflects the clinician’s observations and recommendations but also those variables that are important in sustaining the intervention in the client’s natural environment. 242 january 2023, volume 15, issue 3, 233-245 be humble in the effort of increasing cultural awareness, mistakes will be made and each of them is a learning opportunity. neuringer (1991) gave the first guidance to the field of behavior analysis in the importance of professional humility. neuringer reminds the field that behavior analysts should be comfortable admitting and correcting mistakes. a humble behaviorist is one that admits ignorance, and this should be applied within the context of cultural humility as well (neuringer, 1991). practitioners that adopt a position of humility will help to advance the field and can aid in the dissemination of the science to more diverse communities (kirby et al., 2022). when working with a family who offers a suggestion a practitioner is unfamiliar with, they should admit the limitation in knowledge (kirby et al., 2022). an appropriate response to the family might be “i’m not well-versed in that. let me do some research and get back to you.” taking a truly scientific stance embraces the concept that knowledge is provisional, which is also the case with cultural knowledge (neurenger, 1991). as lechago (2022) describes, “culturally responsive behaviors must be trained such that they are sensitive to the constantly changing contingencies and interactions within and across the multiple consumers that we serve” (p. 15). a culture that a practitioner thought they were familiar with might present differently across different families, leading an action to cause offense. this means that the ultimate goal is not to fully understand a culture, but to be good at graciously admitting mistakes when they occur and acting to correct them. be brave conversations surrounding cultural preferences might initially be awkward for a clinician as they first start to develop their own repertoires. it is important that the clinician weighs the importance of these conversations with the temporary discomfort of having them. dennison and colleagues (2019) make a wonderful suggestion to take time and talk about past experiences and hardships that the family has faced as intervention begins. if this type of conversation seems awkward or disingenuous at first, role play with a colleague and solicit feedback on questions or comments that came across inappropriately. above all, behavior cannot be shaped if it is not emitted. clinicians cannot learn from mistakes that they were too worried to make. it may be helpful to solicit the support of a supervisor or colleague when these types of conversation are upcoming and to let them know the conversation is happening. this would allow the supervisor to hold the practitioner accountable for having the conversation and reporting back regarding how it went. when beginning these conversations, take the advice of alai-rosales et al. (2022) and “do the best you can with great care” (p. 159). commit time to the effort many have suggested making efforts to understand the diverse cultural backgrounds of families that will be receiving services (e.g., arango and lustig, 2022; bacb, 2020; dennison et al., 2019; fong et al., 2016), which has now been formally incorporated in the updated ethics code (bacb, 2020). this type of activity might not come naturally as our environments generally support the perpetuation of the cultures of which we are a member (miller et al., 2019). machalicek et al. (2021) describes the importance of putting in time and completing measurable activities to begin engaging with different cultures. this could seem like a daunting task for practitioners with full caseloads and little personal time, but machalicek et al. give recommendations for small tasks that can be built upon to make meaningful changes. incorporating culturally relevant topics into supervision meetings, listening to podcasts, reading books by authors of a culture that differs from one’s own, or following social media accounts that focus on issues in diversity are just a few suggestions that could be incorporated into a practitioner’s busy schedule (machalicek et al., 2021). to say that the lives of practitioners are busy is an understatement. the idea of finding time for this type of activity might seem difficult, but it is necessary. additionally, organizations that employ behavior analysts should take time to prioritize this type of training and self-reflection. trainings could be offered that educate interventionists and supervisors regarding the unique cultural landscape of a community where a clinic is located. a standing agenda item in supervision meetings could be to discuss the biases that a clinician identified during sessions that week. larger companies could identify experts or members of specific cultures who are available to review protocol development. time during parent training activities could be dedicated to talking with families about family background, previous therapy experiences, or how they have used the procedures that week. these are just a few ideas that might help practitioners incorporate the development of this repertoire within already existing professional activities. conclusion those who practice behavior analysis compromise culture that is uniquely its own (glenn, 1993). just as an alumni group, members of the same church, or members of a specific community would conduct themselves when around their own members, so too do behavior analysts. different cultures are not always siloed and must work together towards common goals. in this way, the culture of behavior analysis interacts with the cultures of the clients that they serve. it is the behaviorist’s responsibility, not the families, to make changes to allow for behavior analysis to fit into each 243 ?? / kristiansen family’s culture. it is understood that the best way to establish lasting change is to build from an already existing repertoire, and a family’s culture is the place to start and learn from (goldiamond, 1974). for a young field, behavior analysis has come far but also has much more work ahead of it. many of the research ideas presented previously are already in progress by colleagues, meaning that the empirical support for culturally responsive recommendations is well on its way. vargas (2022), summarizing skinner, suggests that “a culture increases its survival when it is sensitive to collateral and deferred effects” (p. 8). the satisfaction of those who experience intervention are the field’s collateral effects. it is the incorporation of the cultural preferences of others that will help bolster the acceptance of applied behavior analysis and solidify its survival. acknowledgement sarah would like to thank dr. noor syed, meaghan kantrowitz, and gina cross for their thoughtful comments on early versions of this manuscript. given the topic of this paper, she would like to disclose the identities that have influenced her behavior as a practitioner. sarah was born and raised in the southern united states and currently resides in texas. she identifies as christian and white. she has been in the field of behavior analysis for 14 years and practiced as a practitioner for 9. references alai-rosales, s., pritchett, m., linden, a., cunningham, i., syed, n. (2022). be humble, learn, and care: culturally responsive evidence-based practice. in j. leaf, j. cihon, j. ferguson, & m. j. weiss. handbook of applied behavior analysis interventions for autism: integrating research into practice (pp. 151-179). springer aota 2020 occupational therapy code of ethics. (2020). the american journal of occupational therapy, 74(supplement_3), 7413410005p17413410005p13. https://doi.org/10.5014/ ajot.2020.74s3006 arango, a., & lustig, n. (2022). ignorance and cultural diversity: the ethical obligations of the behavior analyst. behavior analysis in practice. https:// doi.org/10.1007/s40617-022-00701-z american speech-language-hearing association. (2023). code of ethics [ethics]. available from www.asha.org/policy/ baer, d. m., wolf, m. m., & risley, t. r. (1968). some current dimensions of applied behavior analysis. journal of applied behavior analysis, 1(1), 91. https://doi. org/10.1901/jaba.1968.1-91 baer, d., wolf, m., & risley, t. (1987). some still-current dimensions of applied behavior analysis. journal of applied behavior analysis, 20(4), 313– 327. beaulieu, l., addington, j., & almeida, d. (2019). behavior analysts’ training and practices regarding cultural diversity: the case for culturally competent care. behavior analysis in practice, 12(3), 557–575. https://doi.org/10.1007/s40617018-00313-6 beaulieu, l., & jimenez-gomez, c. (2022). cultural responsiveness in applied behavior analysis: self-assessment. journal of applied behavior analysis, 55(2), 337-356. https://doi.org/10.1002/ jaba.907 behavior analyst certification board (2020). ethics code for behavior analysts. behavior analyst certification board. https://www.bacb.com/ wp-content/ uploads/bacb-compliancecode-english_190318.pdf. bowman, k. s., suarez, v. d., & weiss, m. j. (2021). standards for interprofessional collaboration in the treatment of individuals with autism. behavior analysis in practice, 14(4), 1191–1208. https://doi.org/10.1007/s40617-021-00560-0 brodhead, m. t. (2015). maintaining professional relationships in an interdisciplinary setting: strategies for navigating nonbehavioral treatment recommendations for individuals with autism. behavior analysis in practice, 8(1), 70–78. https://doi.org/10.1007/s40617-015-0042-7 brodhead, m. t. (2019). culture always matters: some thoughts on rosenberg and schwartz. behavior analysis in practice, 12(4), 826–830. https://doi. org/10.1007/s40617-019-00351-8 brodhead, m. t., durán, l., & bloom, s. e. (2014). cultural and linguistic diversity in recent verbal behavior research on individuals with disabilities: a review and implications for research and practice. the analysis of verbal behavior, 30(1), 75–86. https://doi.org/10.1007/ s40616-014-0009-8 conners, b. m., & capell, s. t. (eds.). (2021). multiculturalism and diversity in applied behavior analysis: bridging theory and application. routledge, taylor & francis group. dennison, a., lund, e. m., brodhead, m. t., mejia, l., armenta, a., & leal, j. (2019). delivering homesupported applied behavior analysis therapies to culturally and linguistically diverse families. behavior analysis in practice, 12(4), 887–898. https://doi.org/10.1007/s40617-019-00374-1 244 january 2023, volume 15, issue 3, 233-245 fawcett, s. b. (1991). some values guiding community research and action. journal of applied behavior analysis, 24(4), 621–636. https://doi. org/10.1901/jaba.1991.24-621 fong, e. h., catagnus, r. m., brodhead, m. t., quigley, s., & field, s. (2016). developing the cultural awareness skills of behavior analysts. behavior analysis in practice, 9(1), 84–94. https://doi. org/10.1007/s40617-016-0111-6 fong, e. h., ficklin, s., & lee, h. y. (2017). increasing cultural understanding and diversity in applied behavior analysis. behavior analysis: research and practice, 17(2), 103–113. https://doi. org/10.1037/bar0000076 fong, e. h., & tanaka, s. (2013). multicultural alliance of behavior analysis standards for cultural competence in behavior analysis. international journal of behavioral consultation and therapy, 8(2), 17–19. https://doi.org/10.1037/h0100970 gatzunis, k. s., edwards, k. y., rodriguez diaz, a., conners, b. m., & weiss, m. j. (2022). cultural responsiveness framework in bcba® supervision. behavior analysis in practice. https://doi.org/10.1007/s40617-022-00688-7 geiger, k. b., carr, j. e., & leblanc, l. a. (2010). functionbased treatments for escape-maintained problem behavior: a treatment-selection model for practicing behavior analysts. behavior analysis in practice, 3(1), 22–32. https:// doi.org/10.1007/bf03391755 glenn, s. s. (1993). windows on the 21st century. the behavior analyst, 16(2), 133–151. https://doi. org/10.1007/bf03392619 glenn, s. s. (2004). individual behavior, culture, and social change. the behavior analyst, 27(2), 133– 151. https://doi.org/10.1007/bf03393175 goldiamond, i. (1974). toward a constructional approach to social problems: ethical and constitutional issues raised by applied behavior analysis. behavior and social issues, 11(2), 108– 197. https://doi.org/10.5210/bsi.v11i2.92 hayes, s., & toarmino, d. (1995). if behavioral principles are generally applicable, why is it necessary to understand cultural diversity? the behavior therapist, 21–23. jernigan, v. b. b., hearod, j. b., tran, k., norris, k. c., & buchwald, d. (2016). an examination of cultural competence training in us medical education guided by the tool for assessing cultural competence training. jimenez‐gomez, c., & beaulieu, l. (2022). cultural responsiveness in applied behavior analysis: research and practice. journal of applied behavior analysis, 55(3), 650–673. https://doi. org/10.1002/jaba.920 kirby, m. s., spencer, t. d., & spiker, s. t. (2022). humble behaviorism redux. behavior and social issues, 31(1), 133–158. https://doi.org/10.1007/s42822-02200092-4 lechago, s. a. (2022). cultural responsiveness is a critical feature of compassionate care. operants (3), 3-15.https://bfskinner.my.salesforce.com/ sfc/p/#f2000000dixk/a/8w000001wamk/ yqrqii1vd gxa0lrds95aqsf8eydnphrnrw. f2c1su5w machalicek, w., strickland-cohen, k., drew, c., & cohen-lissman, d. (2021). sustaining personal activism: behavior analysts as antiracist accomplices. behavior analysis in practice. https://doi.org/10.1007/s40617-021-00580-w martinez, s., & mahoney, a. (2022). culturally sensitive behavior intervention materials: a tutorial for practicing behavior analysts. behavior analysis in practice, 15(2), 516–540. https://doi. org/10.1007/s40617-022-00703-x mathur, s. k., & rodriguez, k. a. (2021). cultural responsiveness curriculum for behavior analysts: a meaningful step toward social justice. behavior analysis in practice. https:// doi.org/10.1007/s40617-021-00579-3 miller, k. l., re cruz, a., & ala’i-rosales, s. (2019). inherent tensions and possibilities: behavior analysis and cultural responsiveness. behavior and social issues, 28(1), 16–36. https://doi.org/10.1007/ s42822-019-00010-1 najdowski, a. c., gharapetian, l., & jewett, v. (2021). toward the development of antiracist and multicultural graduate training programs in behavior analysis. behavior analysis in practice, 14(2), 462–477. https://doi.org/10.1007/s40617020-00504-0 national institute of health. (2020). cultural awareness: glossary of key terms. https://www.niehs. nih.gov/news/events/pastmtg/ hazmat/ assets/2007/wtp_2007ntec_wruc_latino_tips_ glossary_ 508.pdf. neely, l., gann, c., castro-villarreal, f., & villarreal, v. (2020). preliminary findings of culturally responsive consultation with educators. behavior analysis in practice, 13(1), 270–281. https://doi.org/10.1007/s40617-019-00393-y 245 ?? / kristiansen neuringer, a. (1991). humble behaviorism. the behavior analyst, 14(1), 1–13. https://doi.org/10.1007/ bf03392543 newhouse-oisten, m. k., peck, k. m., conway, a. a., & frieder, j. e. (2017). ethical considerations for interdisciplinary collaboration with prescribing professionals. behavior analysis in practice, 10(2), 145–153. https://doi.org/10.1007/s40617-0170184-x normand, m. p. (2008). science, skepticism, and applied behavior analysis. behavior analysis in practice, 1(2), 42–49. https://doi.org/10.1007/bf03391727 rohrer, j. l., marshall, k. b., suzio, c., & weiss, m. j. (2021). soft skills: the case for compassionate approaches or how behavior analysis keeps finding its heart. behavior analysis in practice, 14(4), 1135–1143. https://doi.org/10.1007/s40617021-00563-x rohrer, j. l., & weiss, m. j. (2022). teaching compassion skills to students of behavior analysis: a preliminary investigation. behavior analysis in practice. https://doi.org/10.1007/s40617-02200748-y rosenberg, n. e., & schwartz, i. s. (2019). guidance or compliance: what makes an ethical behavior analyst? behavior analysis in practice, 12(2), 473– 482. https://doi.org/10.1007/s40617-018-00287-5 ruzycki, s. m., & ahmed, s. b. (2022). equity, diversity and inclusion are foundational research skills. nature human behaviour, 6(7), 910–912. https:// doi.org/10.1038/s41562-022-01406-7 singer, a. (2022). averica’s diversity at the beginning of the 21st century: reflections from census 2000. the brookings institution. https:// c i t e s e e r x . i s t . p s u . e d u / d o c u m e n t ? r e p id=rep1&type=pdf& doi=935476cda132780e4 5628c6f86c8827584a6612b sugai, g., o’keeffe, b. v., & fallon, l. m. (2012). a contextual consideration of culture and school-wide positive behavior support. journal of positive behavior interventions, 14(4), 197–208. https://doi.org/10.1177/1098300711426334 syed, n., mellon, l., kristiansen, s. (2023). equity, diversity, inclusion, and accessibility in research. in d. cox, n. syed, m. brodhead, s. quigley (eds.), research ethics in behavior analysis (pp. 6386). elseiver. https://doi.org/10.1016/b978-0-32390969-3.00012-8 tereshko, l., marya, v., orland, n. (2022). cultural considerations for mealtimes. in j.h., cihon, l. tereshko, k.b. marshall., & m. j. weiss (eds.). behavior analytic approaches to promote enjoyable mealtimes for autistics/individuals diagnosed with autism and their families. vernon press. travers, j. c., ayers, k., simpson, r. l., & crutchfield, s. (2016). fad, pseudoscientific, and controversial interventions. in r. lang, t. b. hancock, & n. n. singh (eds.), early intervention for young children with autism spectrum disorder (pp. 257–293). springer international publishing. https://doi.org/10.1007/978-3-319-30925-5_9 united states census bureau. (2022, august 18). race and ethnicity in the united states: 2010 census and 2020 census. retrieved april 26, 2023, from https://www.census.gov/library/visualizations/ interactive/race-and-ethnicity-in-the-unitedstate-2010-and-2020-census.html vargas, e. a. (2022) b. f. skinner on cultural design: contingencies of reinforcement in the design of a culture and the design of cultures. operants, (3), 7-8. https://bfskinner.my.salesforce.com/ sfc/p/#f2000000dixk/a/8w000001wamk/ yqrqii1vd gxa0lrds95aqsf8eydnphrnrw. f2c1su5w wolf, m. m. (1978). social validity: the case for subjective measurement or how applied behavior analysis is finding its heart. journal of applied behavior analysis, 11(2), 203-214. https://doi.org/10.1901/ jaba.1978.11-203 wright, p. i. (2019). cultural humility in the practice of applied behavior analysis. behavior analysis in practice, 12(4), 805–809. https://doi.org/10.1007/ s40617-019-00343-8 microsoft word 3iejee_7_1_young_nageldinger international electronic journal of elementary education, 2014, 7(1), 47-56. issn:1307-9298 copyright © iejee www.iejee.com considering the context and texts for fluency: performance, readers theater, and poetry chase young ∗∗∗∗ texas a&m university, corpus christi, tx, usa james nageldinger elmira college, elmira, ny, usa received: 23 october 2014 / revised: 29 october 2014 / accepted: 29 october 2014 abstract this article describes the importance of teaching reading fluency and all of its components, including automaticity and prosody. the authors explain how teachers can create a context for reading fluency instruction by engaging students in reading performance activities. to support the instructional contexts, the authors suggest particular text-types that are well-suited for reading fluency activities. keywords: reading fluency, text selection, performance reading introduction manny (pseudonym) jokes with some of the cast members backstage. in a few minutes, he and some of his classmates will be performing a readers theatre piece for their fifth grade class. he’s nervous but not overly so. less than a year ago reading aloud in front of people would have been the last thing he wanted to do. the way he plodded word for word through text was as painful for others to listen to as it was for him to undertake. today, he looks forward to it. what’s happened in the meantime could be telling for schools across the country. manny was a struggling reader. specifically he struggled reading in making sense of what he read. measures of oral reading fluency and comprehension put him nearly a two years behind his peers. interventions included practicing word lists, vocabulary worksheets, and drilling in phonics. then at the urging of a friend, he joined the drama club because, as he put it, “you got to like act like different people and stuff. it’s really cool!” as acting exercises progressed and morphed from physical expression to oral interpretation to full integration, scripts were added into the mix. initially, he would be given a scene to read and prepare. when it came time for the group to decide on a play to perform, their coach offered up several possibilities and gave them ∗ chase young, 6300 ocean dr. corpus christi, tx, 78418, usa phone:361-852-3661 e-mail: chase.young@tamucce.edu international electronic journal of elementary education vol.7, issue 1, 47-56,2014 48 opportunities to take the scripts home and decide whether or not they saw a character for themselves to portray. rehearsing the scene at home would entail reading the scene numerous times to understand the situation and the character. the result? more reading. importantly, more reading that is akin to what is known as “close reading”. decades of research have shown us that reading the same text several times, or repeated reading, improves reading comprehension (dowhower, 1987; faver, 2008; herman, 1985; levasseur, macaruso, & shankweiler, 2008; musti-rao, hawkins, & barkley, 2009; rasinski, 1990; samuels, 1979) . close reading is a more focused form of repeated reading that involves “an intensive analysis of a text in order to come to terms with what it says, how it says it, and what it means” (shanahan, 2012, para. 5). in close reading the reader is given purposes for reading, text dependent questions to guide their thinking, and opportunities to interact with others about the content (fisher & frey, 2012). in order to understand a character he is considering portraying, manny has to read close for deeper meaning. his purposes for reading and guiding questions from the text are inherent in his efforts to develop a character. all scripts, including monologues are dialogic in nature and meaning is shaped by interaction with other actors and the audience. for most kids, especially those who struggle with reading, getting them to read anything more than once can be difficult, especially when they have essentially given up on reading as a source of enjoyment. although manny’s involvement in a school theatre program was not intended to bolster his reading achievement, there is reason to believe it did. he not only found a source of compelling literature, and an avenue of expression, but he was developing important higher level thinking skills in the process (kuhn & stahl, 2003; mcmaster, 1998) . in addition, the confidence he gained in the theatre program led him to volunteer for the readers theatre performance his class was presenting. manny is becoming a fluent reader. in order to appreciate that important role that reading fluency plays in overall reading proficiency, we urge you to consider the broad definition and its constituents. reading researchers typically define reading fluency as smooth, effortless reading (kuhn & stahl, 2003) . more specifically, fluent readers recognize words automatically, read at an adequate pace, and with appropriate expression. when we consider all of components in the definition, reading fluency serves as foundational skill that promotes reading comprehension, the goal of reading. according to automaticity theory, when students begin to recognize words automatically, cognition is freed to focus on higher order process such as reading comprehension (laberge & samuels, 1974) . whitaker (1983) described the development of automaticity on a continuum. in the beginning, learners engaged in an effortful and laborious process to complete a given task. though, as the processes became more automatic, the act eventually occurred on an unconscious level. as readers became more automatic and accurate in word recognition, reading rate generally increased (samuels, 1979). rasinski (2000) asserted that it was important for students to read at an appropriate pace. however, reading at an “appropriate rate” has several implications. reading is not a race, but speed does matter if for no reason other than efficiency. a student that can read and comprehend at a faster rate will acquire more new information in less time (rasinski, 2000). but comprehension is the operative word here. rate is often dependent on the context. for example, when we read nonfiction, it is important adjust the rate for optimal learning, often achieved by slowing down. other forms of reading require rate adjustments, such as dramatic oral readings, including plays, poetry, or speeches. dramatic oral readings also require appropriate expression, or prosody (tyler & chard, 2000). considering the context and texts for fluency: performance, readers theater, and poetry / young & nageldinger 49 early research on eye tracking revealed that good readers move their eyes forward and backward over the page, jumping ahead and slowing down both within and between sentences (huey, 1908). later research confirmed that good readers make meaning of text by adjusting rate, parsing complex sentences into meaningful phases (schreiber, 1991; (rasinski, 1989) , and applying emphasis even when reading silently (kentner, 2012) . (2009) did work showing that the audio imaging experiences that occur during silent reading allow a reader to experience the prosody of an already familiar character. in a recent study, petkov & belin (2013) using neuroimaging and neuronal recording work confirmed that inner processes at play during reading involve experiencing voices, and that voice-sensitive brain regions can be activated even when the quotation is from a fictional person and the voice is unknown. research has indicated that appropriate prosodic reading is a good predictor of overall reading proficiency (daane, national assessment of educational progress, & national center for education statistics, 2005) . daane et al. revealed a strong correlation between prosodic reading an overall reading achievement. in another study, miller and schwanenflugel (2008) corroborated this finding and reported that students who read with “adult-like” prosody in first and second graders are more likely to competently comprehend text by the end of thirdgrade. because we are aware of prosody’s important role in the reading process (schrieber, 1991), we suggest that reading fluency activities should possess both a practice component that builds automaticity and a prosodic component that encourages expression. we believe that both of these components occur in performance activities. thus, we recommend teachers include a performance element in reading fluency instruction. performance activities the minute a teacher stands in front of their class, the performance begins. we know the feeling. it is a wonderful, thrilling rush that we cannot achieve anywhere else. there is nothing quite like a five-star teaching performance. which makes our request even more difficult to stomach: teachers— share the stage. i know you think we are traitors. we are not; we are reading fluency researchers carrying out our life’s mission to create a global society of fluent readers. in order to do that, you must invite and encourage student performances of text. there are many different instructional activities that call for performance. we would like to focus on a couple research-based strategies that can increase your students’ reading fluency. first, we will discuss readers theatre, an activity that is similar to a putting on a play, but simplified in order to focus on fluency instruction. the next, poetry café, is similar to readers theatre, but the students perform poetry. finally, we discuss other texts that students can perform to increase reading fluency. the following examples are from particular classrooms, which are always unique by nature. we encourage you to consider the protocols carefully; perhaps modifications are necessary to meet the needs of your students. however, when modifying, we recommend that you keep the most important element—consistency. the research supporting these strategies all came from classrooms that committed reading fluency instruction throughout the school-year. therefore, to increase the likelihood of positive results, you should consider making fluency instruction a part of your daily routine. international electronic journal of elementary education vol.7, issue 1, 47-56,2014 50 readers theatre readers theatre is an educational activity that requires students to perform a text. these texts can be from existing literature, nonfiction, poetry, parodies, or student generated. readers theater requires no props, no memorization, and no costumes. students entertain audiences with their expressive oral reading (prosody), while the prior rehearsal gives a purpose for repeated readings, a well-research method for increasing reading fluency (mercer, campbell, miller, mercer, & lane, 2000; vadasy & sanders, 2008; vaughn, chard, bryant, coleman, & kouzekanani, 2000. research (griffith & rasinski; 2004; tyler & chard, 2000; young & rasinski; 2009) suggests that readers theater, in particular, is an effective means of enhancing students’ reading fluency and is a motivational reading activity (martinez, roser, & strecker, 1998). first, the teacher selects a variety of scripts (see figure 1 for suggested resources). the scripts can be trade books, poetry, fiction, newspaper articles, or just about any other text that lends itself to performance. student generated texts can also be used as scripts (young & rasinski, 2011). you can even script excerpts from movies or television. then again, expressive oral readings can turn almost any text into an exciting performance (even standardized tests!). www.thebestclass.org/rtscripts.html www.timrasinski.com http://www.teachingheart.net/readerstheater.htm http://www.aaronshep.com/rt/rte.html http://www.storiestogrowby.com/script.html figure 1. sources for readers theater scripts because the text selection is in the teacher’s control, teachers can easily differentiate readers theatre for a variety of students. for students that struggle, you can select easier or more familiar texts. conversely, for advanced students, you can increase the text difficulty. because any text can potentially become a script, the text can be in any language. so whether you are performing the three little pigs in english, german (die drei kleinen schweinchen), french (les trois petit cochons), or spanish (lost tres cerditos), the procedure is still the same. young and rasinski (2009) described a five-day format that worked well in a second grade classroom. students in young’s classroom participated in readers theatre each week for an entire school year (approximately 8 months). overall, students word recognition accuracy and comprehension increase. in addition, students’ prosody increased by 20% and students doubled the expected growth in words read correctly per minute. the five-day format is easily implemented on a weekly schedule. on monday, the students choose their scripts after the teacher reads each of them aloud. the students then read over the script for two purposes. first, the students need to comprehend the overall meaning of the texts. second, students should decide on which parts they might like to play. the following day, students choose and highlight their parts. on tuesday, the students focus on word identification making sure they know all the words and can pronounce them correctly. wednesday’s goal is to read with appropriate expression. the students focus on matching the meaning of the text with the expressiveness of their voice. while prosody is the key component in the entertainment value of readers theatre, it also requires students to deeply analyze the text and calibrate their oral reading based on their reading considering the context and texts for fluency: performance, readers theater, and poetry / young & nageldinger 51 comprehension. students practice their performance on thursday, and prepare for friday’s big premier. by performance time, each student will have closely read their script over a dozen times. performance day serves as an opportunity for students to simultaneously entertain audience and demonstrate how their practice leads to fluent oral readings of text. thus, it is important to create a context where their hard work is appreciated. because entertaining an audience is a motivating factor and the reason for rehearsal, locating an audience is imperative. fortunately, there are many sources for an audience. you can invite parents, administrators, other classes, or other school staff. if audiences are scarce, the students can always perform for their peers. with the increased availability of technology, there are other ways to procure an audience. the internet is another context for performance. for example, teachers sometimes film performances, so the students can watch themselves. the videos also make great gifts for their families. with permission, you can upload the performances to a private blog, youtube, or other media sharing sites. in another technological variation, vasinda and mcleod (2011) described a performance method that required students read their scripts into a microphone and published the recordings as a podcast. these examples only represent two ideas for performance on the internet, and leave plenty of room for teacher creativity. we invite you to consider how web 2.0 tools, websites, social media, and other applications could serve as a venue for readers theater performances. poetry jorge luis borges once said, “truly fine poetry must be read aloud. a good poem does not allow itself to be read in a low voice or silently. if we can read it silently, it is not a valid poem: a poem demands pronunciation. poetry always remembers that it was an oral art before it was a written art.” (1972, p. 9). we whole-heartedly believe these words, and concur that poetry is meant to be performed. indeed, students today may have a different notion of “fine poetry”, so if you would allow us to generalize, we suggest that any poetry can be performed. this could include all genres of poetry (e. g. epic, narrative, prose, or humorous) from any era. there are many ways to ask students to recite poetry, but we strongly recommend time for rehearsal and a context for performance. wilfong (2008) described a reading program that she called “the poetry academy.” the teacher selects a poem with each student’s reading level in mind. a volunteer reads the poem aloud to the student. next, the volunteer and the student read the poem together— often referred to as choral reading. the student then reads the poem aloud to the volunteer independently. essentially, the volunteer gradually releases the responsibility to the student. after the gradual release, the student and volunteer engage in a conversation about the text, the purpose of the dialogue is to identify the meaning of the poem and build identify any troublesome words. the student then takes the poem home and reads the poem to as many human beings as possible, each of which provide a signature as proof of the performance. upon returning to school, the student reads the poem to the volunteer to demonstrate mastery. the poetry academy is a good example of utilizing volunteers and incorporating friends and family into students’ learning, but as always, there are several options, and you get to decide what works best for your classroom. another option, often dubbed, “poetry café”, provides a larger venue for performance. after a quick google search you will find many variations for implementing the activity. we will share a method that is similar to the five-day format used for readers theatre. international electronic journal of elementary education vol.7, issue 1, 47-56,2014 52 on monday the students select a poem. students can surf the internet, peruse the classroom, take a trip to the library, or ask the teacher for suggestions. after the students choose a poem, the teacher makes sure the students can identify all of the words. though not every student will require additional assistance, some might. we caution you to think carefully before determining if a text is too difficult for a student. often times the answer is to help the student choose an easier text, but you may want to consider providing implementing a fluency intervention as an alternative to switching poems. providing a research-based fluency interventions combined with the opportunity for practice could increase the likelihood of a successful performance. if a student is struggling with rapid word recognition, the teacher may want to employ modeled fluent reading, assisted reading, or repeated readings (national institute of child health and development, 2000). in the case that the student needs assistance with both word recognition and expression, you might try a method called reading together (mohr, dixon, & young, 2012; young & mohr, in press; young, mohr, rasinski, in press). to utilize reading together, the teacher first employs the neurological impress method (heckelman, 1969). the teacher and student read the poem aloud together. however, the teacher stays slightly ahead of the student and reads with appropriate expression while the student “chases” the teacher. next, similar to repeated readings (samuels 1979), the student then rereads the poem aloud independently. the teacher carefully listens for accuracy in word recognition and for appropriate expression. research (young, mohr, & rasinski, in review) suggests that the expression modeled by the teacher can be heard in the student’s rendering of the text. of course, not all students require such intense interventions. after the students are comfortable with the poem, the students practice every day until the performance. you can also encourage students to practice at home. some variations of poetry café recommend a week of practice, performing on fridays. other research (young, valadez, & power, in revision) suggests that practice of shorter poems can be limited to two days. in this case, students perform twice per week, once on wednesday and again on friday. thus, students select the poem at the beginning of the week and practice on monday and tuesday for the performance on wednesday. after the performance, students select another poem and practice wednesday thursday for the performance on friday. you can enhance performance days by adding some poetry café elements. ask the students to dress in black. dim the lights in the classroom and provide a stool for the young poets. to complete the context for poetry performance, instruct the students to snap instead of clap. though you may feel as if you are truly sitting in a poetry café, remember that you are still in the classroom building your students’ reading fluency, a foundational component in reading (national institute of child health and development, 2000). speeches, monologues, and presentations finally, we would like to point out a few other types of text that can be performed— speeches, monologues, and other presentations. although you can apply the following tips to any oral reading, we will use speeches as an example. speeches are written for the sole purpose of reading aloud. the speaker rehearses pronunciation, timing, pitch, pause, intonation, inflection, with the hope that their rehearsals increase the impact of the speech. text of famous speeches can be found on the internet, all the way from socrates’ “apology” to “the gettysburg address” delivered by abraham lincoln. students can also write their own speeches. when students rehearse speeches, they need to focus on several different subcategories in prosody. the multidimensional fluency scale (mfs; zutell & rasinski, 1991) captures most considering the context and texts for fluency: performance, readers theater, and poetry / young & nageldinger 53 of these. therefore, it is possible to use the mfs rubric (figure 1) to help students refine their speeches. we can direct students to consider expression and volume, phrasing, smoothness, and pace. in order to infuse expression and volume in a speech, the student must first consider the intended meaning of the speech. then, the student adjusts his or her expression to match the meaning. volume in a speech is also very powerful when employed effectively. they should think about the most important or dramatic parts of the speech, and consider adjusting his or her volume while reading aloud. sometimes speaking in a quieter voice draws the crowd in, and speaking at a higher volume may emphasize the speaker’s message. phrasing is also important when delivering a speech. sometimes, to add effect, the speaker reads with intentional phrasing, pausing to add effect. therefore, students need to consider how to phrase their speeches, and ideal occasions for pause. and with any oral reading, smoothness is also essential. speeches are typically not read in a choppy and laborious manner. it is crucial, then, that students practice enough to read smoothly and effortlessly. finally, the pace of the speech is very important. sometimes speakers slow down or speed up to increase the impact particulars in their speeches. students should rehearse their speeches, and analyze the parts for opportunities vary the pace of their reading. if students attend to these dimensions of fluency while rehearsing their speeches, the performance should be a successful one. additional questions although we know that reading fluency is a foundation for comprehension, we are still unsure of the magnitude that performance methods enhance comprehension as opposed to other fluency building activities. it would be interesting to determine what direct effect performance has on comprehension. in addition, a most of the research on performance methods is conducted in middleelementary classrooms, and there are fewer studies in first-grade and kindergarten (see garrett and o’connor, 2010). research has also been conducted in secondary classrooms (see keehn, harmon, & shoho, 2008), but if we are to promote performance methods for all elementary students, we need to increase the classroom-based research in those grade levels. conclusion the lights dim, and the small group takes a step downstage. scripts in hand, they read the name of the play in unison. it is a script the teacher adapted from will hobb’s crossing the wire about a boy from mexico crossing the border seeking work to save his family from starving. manny follows along, reading in unison, and then alone. he is playing the part of the young boy, one of the only members of the party that understands english. throughout the play, manny reads lines in both english and spanish, his native tongue and feels a pride at being able to read now in both languages. as the group takes a bow after their final line is delivered in unison, applause erupts and manny beams. he can’t wait to see the scripts from next week. • • • international electronic journal of elementary education vol.7, issue 1, 47-56,2014 54 dr. chase young joined the department of educational leadership, curriculum and instruction at texas a&m university-corpus christi in 2013. dr. young received his ph.d. in reading education from the university of north texas where he was named the 2011 outstanding doctoral student in reading education. the association of literacy educators and researchers honored him with the 2014 jerry johns promising researcher award. his primary research interests include reading fluency, supporting struggling readers, and integrating technology in elementary literacy instruction. previously, he taught in the primary grades and served as a literacy coach. dr. james nageldinger is an assistant professor of literacy education at elmira college. he received a bs in theatre arts from grand valley university, a ma in special education from the university of washington, and a phd in curriculum and instruction with an emphasis in literacy from kent state university. his scholarly interests include reading fluency, emphatic prosody, hybrid text, readers who struggle, and the impact of theatre activities on reading. his research can be found in several trade journals. james has taught in classrooms in washington state, ohio, and hawaii. prior to pursuing a career in education, he directed community theatre in berlin. references borges, j. l. (1972). seven nights. new york, ny: new directions. daane, m. c., national assessment of educational progress, & national center for education statistics. (2005). fourth-grade students reading aloud. washington, d.c.: national center for education statistics, u.s. dept. of education, institute of education sciences. dowhower, s. l. (1987). effects of repeated reading on second-grade transitional readers' fluency and comprehension. reading research quarterly, 22(4), 389-406. faver, s. (2008). repeated reading of poetry can enhance reading fluency. reading teacher, 62(4), 350352. fisher, d., & frey, n. (2012). improving adolescent literacy: content area strategies at work (3rd ed.). boston: pearson garrett, t. d., & o'connor, d. (2010). readers' theater: "hold on, let's read it again.". teaching exceptional children, 43(1), 6-13. griffith, l.w., & rasinski, t.v. (2004). a focus on fluency: how one teacher incorporated fluency with her reading curriculum. the reading teacher, 58(2), 126–137. herman, p. a. (1985). the effect of repeated readings on reading rate, speech pauses, and word recognition accuracy. reading research quarterly, 20(5), 553-565. keehn, s., harmon, j., & shoho, a. (2008). a study of readers theater in eighth grade: issues of fluency, comprehension, and vocabulary. reading & writing quarterly, 24(4), 335-362. kentner, g. (2012). linguistic rhythm guides parsing decisions in written sentence comprehension. cognition, 123(1), 1-20. doi:10.1016/j.cognition.2011.11.012 kuhn, m. r., & stahl, s. a. (2003). fluency: a review of developmental and remedial practices. journal of educational psychology, 95(1), 3-21. kurby, c. a., magliano, j. p., & rapp, d. n. (2009). those voices in your head: activation of auditory images during reading. cognition, 12, 457–461. laberge, d., & samuels, j. (1974). towards a theory of automatic information processing in reading. cognitive psychology, 6, 293-323. levasseur, v. m., macaruso, p., & shankweiler, d. (2008). promoting gains in reading fluency: a comparison of three approaches. reading & writing, 21(3), 205-230. doi:10.1007/s11145-007-90701 considering the context and texts for fluency: performance, readers theater, and poetry / young & nageldinger 55 martinez, m., roser, n.l., & strecker, s. (1998). “i never thought i could be a star”: a readers theatre ticket to fluency. the reading teacher, 52(4), 326–334. mcmaster, j. c. (1998). `doing' literature: using drama to build literacy. reading teacher, 51(7), 574. mercer, c. d., campbell, k. u., miller, m. d., mercer, k. d., & lane, h. b. (2000). effects of a reading fluency intervention for middle schoolers with specific learning disabilities. learning disability research and practice, 15(4), 179-189. miller, j., & schwanenflugel, p. j. (2008). a longitudinal study of the development of reading prosody as a dimension of oral reading fluency in early elementary school children. reading research quarterly, 43(4), 336-354. musti-rao, s., hawkins, r. o., & barkley, e. a. (2009). effects of repeated readings on the oral reading fluency of urban fourth-grade students: implications for practice. preventing school failure, 54(1), 12-23. mohr, k. a. j., dixon, k., & young, c. j. (2012) effective and efficient: maximizing literacy assessment and instruction. in ortlieb, e. t., & cheek, jr., e. h. (eds.). literacy research, practice, and evaluation. bingley, uk: emerald group national institute of child health and human development. (2000). report of the national reading panel. teaching children to read: an evidence-based assessment of the scientific research literature on reading and its implications for reading instruction (nih publication no. 00-4769). washington, dc: u.s. government printing office. petkov, c., & belin, p. (2013). silent reading: does the brain ‘hear’ both speech and voices? current biology, 23(4) doi:10.1016/j.cub.2013.01.002. rasinski, t. v. (1989). adult readers' sensitivity to phrase boundaries in texts. the journal of experimental education, 58(1), pp. 29-40. rasinski, t. v. (1990). effects of repeated reading and listening-while-reading on reading fluency. journal of educational research, 83(3) rasinski, t. (2000). speed does matter in reading. the reading teacher, 54(2), 146-151. samuels, s. j. (1979). the method of repeated readings. the reading teacher, 41, 756-760. samuels, s. j. & flor, r. f. (1997). the importance of automaticity for developing expertise in reading. reading & writing quarterly, 13, 107-121. schneider, w. & fisk, a. d. (1983). attentional theory and mechanisms for skilled performance. in r. a. magill (ed.), memory and control of action (pp.119-143). ny: north-holland publishing company. schreiber, p. (1991). understanding prosody’s role in reading acquisition. theory into practice, 30(3), pp. 158-164 shanahan, t. (2012 june 18). what is close reading? [web log comment]. retrieved from http://www.shanahanonliteracy.com/2012/06/what-is-close-reading.html tyler, b., chard, d. (2000). using readers theatre to foster fluency in struggling readers: a twist on the repeated reading strategy. reading and writing quarterly. 16(2), 163. vaughn, s., chard, d. j., bryant, d. p., coleman, m., & kouzekanani, k. (2000). fluency and comprehension interventions for third-grade students. remedial and special education, 21(6), 325335. vasinda, s. & mcleod, j. (2011). extending readers theater: a powerful and purposeful match with podcasting. the reading teacher, 64(7), 486-497. vadasy, p. f., & sanders, e. a. (2008). repeated reading intervention: outcomes and interactions with readers’ skills and classroom instruction. journal of educational psychology, 100, 272–290. international electronic journal of elementary education vol.7, issue 1, 47-56,2014 56 whitaker, h. a. (1983). towards a brain model of automatization: a short essay. in r. a. magill (ed.), memory and control of action (pp.199-214). new york: north-holland publishing company. wilfong, l.g. (2008). building fluency, word-recognition ability, and confidence in struggling readers: the poetry academy. the reading teacher, 62(1), 4–13. young, c. & rasinski, t. (2009). implementing readers theatre as an approach to classroom fluency instruction. the reading teacher, 63(1), 4-13. young, c., & mohr, k. a. j. (in press). successful literacy interventions: an rti case-study analysis. in s. garrett (ed.). ceder yearbook. texas a&m university-corpus christi: center for educational development, evaluation, and research. young, c., mohr, k. a. j., rasinski, t. (in press). reading together: a successful reading fluency intervention. literacy research and instruction. zutell, j., & rasinski, t. (1991). training teachers to attend to their students' oral reading fluency. theory into practice, 30(3), 211-217. international electronic journal of elementary education vol. 3, issue 3, july, 2011. issn:1307-9298 copyright © iejee www.iejee.com dear reader, while we are looking forward to publishing the next special issue of international electronic journal of elementary education (iejee) which will be about reading comprehension by prof. karen m. zabrucky of georgia state university, usa, as the guest editor, we are pleased to present you volume 3, issue 3 of the journal. the first article is a cross-cultural study of humour among chinese and greek preschool children in relation to cognitive development. researchers juan guo, xiangkui zhang, yong wang and aphrodite xeromeritou address an intersting topic: humour among preschool children. in their research they found that chinese and greek children’s humour recognition were significantly and positively correlated to their cognitive development, but there was a different correlation pattern between humour response levels and cognitive development. chinese children’s level of humour responses was negatively and moderately correlated to their level of cognitive development in contrast to the positive correlation between these two variables among the greek children. they conclude that cultural factors play a strong role in determining the correlation between humour response and cognitive development among young children. the second article is about another contemporary global topic in elementary education: the self-assessment processes in the elementary schools. canadian researchers thomas g. ryan and leslie telfer conducted a research review that includes an examination of the foundational components of effective schools, the purpose of school selfassessment, and the successes and challenges of school self-assessment both locally and internationally. ryan and telfer present informative and research based perspectives and practical ideas about school self-assessment that the decision makers at any level of the educational system can make use of. international electronic journal of elementary education vol.3, issue 3, july, 2011 vi the third article is addressing assessment of bilingual children with inattention, overactivity and impulsivity. it’s a well known fact that adhd is one of the widespread neurological disorders among children. researchers meral r. özerk, jan arne handorff and kamil özerk draw our attention to several critical aspects of the assessment practices and medical diagnosis of bilingual children with immigrant background based on a research project in norwegian context. in their article, the authors present some solutions as an alternative to one-sided intelligence-test based approaches. they stress the importance of multidimensional, multisourced and bilingual assessment model for identifying the knowledge-related and language-related elements of the academic learning gap that these children usually experience prior to and during the assessment period. the fourth and the last article of this issue is about the effects of using flashcards with reading racetrack to teach letter sounds, sight words, and math facts to elementary students with learning disabilities. those who work with or interested in reading and math teaching in elementary education know that there are several approaches to and arguments about how to ensure the development of these important skills among young students. at the same time it’s important to present research based ideas, programs and methods that can help elementary teachers to teach students with learning disabilities letter sounds, sight words and math facts. on the bases of their restricted research study, researchers rachel erbey, t. f. mclaughlin, k. mark derby and mary everson present informative pedagogical ideas from their practical study. with their words: “the study was practical. the racetrack was effective, and was easy to use in an elementary resource classroom. the flashcards and racetracks were simple to create, and were easy to use in two subject-matter areas. reading/math racetracks were easy to implement and could be incorporated into the general and special education classroom routine.” i hope you, as a reader, will enjoy the content of this issue. i want to express my gratitude to my closest colleagues and executive editors dr. turan temur of university of dumlupinar and dr. gökhan özsoy of aksaray university for their editorial contributions. i also want to thank to all of the peer reviewers for this issue of iejee. sincerely, dr. kamil özerk, editor-in-chief professor of education university of oslo international electronic journal of elementary education, december, 2016, 9(2) issn:1307-9298 copyright © iejee www.iejee.com dear iejee reader, treatment, training and teaching of children with autism/autism spectrum disorders (asd) is one of the important topics for psychologists and educators. this field is a challenging field. those who are involved in the field as service providers need to be informed about new ideas, concepts, methodological approaches and experiences of the others within the field. practitioners deserve research based knowledge that can help them to provide better treatment, training and teaching for children with autism/autism spectrum disorders. international electronic journal of elementary education – as an open access journal decided to devote this year’s special issues to autism/asd. the main aim is contribution to the field and creation of better treatment, training and teaching opportunities for children regardless who they are and where they live. i owe a great thankfulness to dr. shahla ala'i-rosales and dr. svein eikeseth who without any hesitation accepted my request to join me as guest editors of this special issue. they did a great job. without their academic expertise and network, it could not possible to materialize this special issue. i also want to thank all the authors and peer-reviewers for their contributions. their genuine interest in the field and collaborative attitudes will be remembered. at the back stage dr. turan temur, dr. hayriye gül kuruyer, dr. gökhan özsoy, doctoral candidate hasan tabak, and doctoral candidate mustafa bakir, doctoral candidate emel bayrak özmutlu, saniye nur gündüz did a great job as usual with their close monitoring us during the entire process. i also expressed many thanks to them as iejee’s executive editoral team. editor-in-chief kamil özerk http://www.iejee.com/ international electronic journal of elementary education january 2023, volume 15, issue 3, 171-172 171 © 2022 published by kura education & publishing. this is an open access article under the cc byncnd license. (https://creativecommons. org/licenses/by/4.0/) copyright © www.iejee.com issn: 1307-9298 editors’ note: important topics in applied behavior analysis as it relates to autism joseph h. cihona, justin b. leafb, ellie kazemic introduction the field of behavioral intervention as it relates to intervention for autistics/individuals diagnosed with autism spectrum disorder1 (asd) has grown exponentially. as with the growth of any field dedicated to the service of others, this growth has come with its successes and failures, its inspiration and trepidations. undoubtably, the lives of many diagnosed with asd and their families have been improved through the dissemination and accessibility of quality behavioral services based on the science of applied behavior analysis (aba). there is also little doubt, and emerging evidence, that the challenges related to the rapid growth of the field have also resulted in harms as a result of the misapplication or malpractice of aba-based services. answers to how we may address concerns related to this rapid growth such as maltreatment, improved quality, and increased access to progressive methods are not easy and cannot be answered in a single manuscript or issue of a journal. the purpose of this issue, instead, is to continue and expand on discussions examining some recent important topics in behavioral intervention specifically as it relates to care available for autistics/individuals diagnosed with asd. there are several broad topics included across the papers in this special issue ranging from private equity to perspectives of autistic individuals to advances in research, and credentialling to compassion. each paper takes an in-depth examination of each particular topic pushing us all to think deeper about the contingencies operating on our field and their effects. furthermore, this special issue includes a variety of perspectives within aba and autism intervention in an effort to provide the reader with pathways forward for behavior analysts, parents, and other professionals. zane and colleagues highlight the importance of reflecting on our own world views and how that impacts the quality and type of services we provide. leaf and colleagues discuss the difficult crossroads behavior analysts can find themselves in when navigating working collaboratively while remaining faithful to the principles of our science. dillenburger and keenan identify parallels in discrimination practices experienced by immigrant communities in britain received : 23 march 2023 revised : 25 march 2023 accepted : 31 march 2023 doi : 10.26822/iejee.2023.289 a,* corresponding author: joseph h. cihon, autism partnership foundation, endicott college, usa. e-mail: jcihon@apfmail.org orcid: https://orcid.org/0000-0001-9272-7749 b justin b. leaf, autism partnership foundation, endicott college, usa. e-mail: jbleaf@apfmail.org orcid: https://orcid.org/0000-0001-8315-7203 c ellie kazemi, california state university, northridge, usa. e-mail: ellie.kazemi@csun.edu orcid: https://orcid.org/0000-0001-8316-4112 january 2023, volume 15, issue 3, 171-172 172 after world war two and the current debate among behavior analysts and anti-aba advocates and discuss pathways forward. lui reminds us of the possible damaging effects of overpromoting any particular narrative related to autistic perspectives. gerhardt and colleagues highlight the importance of targeting meaningful curriculum to support independence as early as possible. melton and colleagues discuss the importance of leading with compassion and extending the current discussion around compassion to direct-line interventionists. kristiansen provides a wonderful summary and description of the recommendations within the literature to increase cultural responsiveness in aba. linnehan and colleagues begin a discussion about the contingencies leading to the development of certification and licensure within aba as well as the resulting unintended consequences. kazemi extends the discussion of certification and licensure to the challenges and benefits of accreditation of organizations. gershfeld-litvak provides a balanced discussion of the possible benefits and drawbacks of private equity entering into the field of aba-based interventions for autistics/individuals diagnosed with asd. dixon and colleagues make a strong argument for the use of group design methods and statistical analyses in an effort to expand the impact the science of behavior analysis can make on the world. saunders provides an empirical evaluation of supervision strategies that can improve treatment fidelity as well as job satisfaction for behavior technicians providing intervention for autistics/individuals diagnosed with asd. finally, ferguson and milne examine the research to highlight progress in moving the field to a more progressive approach to aba while highlighting several areas for continued research and improvement. our field has come a long way as measured by many metrics (e.g., number of individuals holding certifications, insurance providers funding aba-based interventions for autistics/individuals diagnosed with asd, behavior analytic publications). nonetheless, it has yet to live up to its fullest potential and has had many missteps along the way. our hope is that the articles included in this special issue that highlight some of our successes as well as our missteps lead to continued progress and change, especially as it relates to aba-based interventions for autistics/ individuals diagnosed with asd. progress at a large scale may not be easy or fast, but shifting our focus on the contingencies preventing and leading toward that progress may make the road toward progress a bit smoother and more welcoming to others focused on improving the field and outcomes for the clients we are fortunate to serve. footnotes 1the terms diagnosed with autism/asd, on the autism spectrum, individual with autism/asd, and autistic are used throughout this paper. the authors recognize that there are varied preferences and conventions related to personand identify-first language among the academic and autistic communities. the terminology selected for use in this paper is to be inclusive of varying preferences as well as grammar and stylistic needs and does not reflect a terminological intent. international electronic journal of elementary education, 2015, 8(1), 55-68 issn:1307-9298 copyright © iejee www.iejee.com students’ opinions on the light pollution application cengiz özyürek ordu university, turkey güliz aydin muğla sıtkı koçman university, turkey received: june, 2015 / revised: august, 2015 / accepted: august, 2015 abstract the purpose of this study is to determine the impact of computer-animated concept cartoons and outdoor science activities on creating awareness among seventh graders about light pollution. it also aims to identify the views of the students on the activities that were carried out. this study used one group pre-test/post-test experimental design model with 30 seventh graders. the data in the study were collected via open-ended questions on light pollution and semi-structured interview questions. the open-ended questions on light pollution were administered as a pre-test and a posttest. after the post-test was administered, semi-structured interviews were conducted with seven students. the data collected from the open-ended questions and semi-structured interviews were qualitatively analysed and quotes from the students’ statements were included. looking at the answers of the students to questions on light pollution, it was understood that the activities that were carried out were effective. furthermore, all of the students that were interviewed made positive statements about the activities that were carried out. keywords: light pollution, concept cartoons, students’ views. introduction humans are an indispensable part of the environment that they live in. due to the rapid increase in population, overurbanization, industrialization and, consequently, the excessive use of natural resources, today, environmental issues have become global issues. currently, light pollution is one of the global issues that are negatively affecting the whole world. light pollution is defined as the wrong use of light in a way that disturbs living things (aslan, 2001). various studies have revealed the reasons and effects of light pollution. crawford (2001) found that wrong outdoor lighting causes light pollution. osman, isobe, nawar and morcos (2001) and çetegen and batman (2005) stated that light pollution is an important problem for astronomical studies, the environment and the  cengiz özyürek, department of elementary education, faculty of education, university of ordu, turkey. phone: +90 452 2265200 e-mail: cengizozyurek@outlook.com http://www.iejee.com/ international electronic journal of elementary education vol.8, issue 1, 55-68, 2015 56 economy. in their studies, they suggested what could be done. in their study on the reasons, results and solutions to light pollution, percy (2001) and hanel (2001) found that people did not have sufficient knowledge on light pollution. sadık, çakan and artut (2011) examined the perceptions of students on environment problems via the pictures that the students drew. they found that the students drew the loss of forests, air pollution, decreasing species, spoiling ozone layer, sound pollution, soil pollution, global warming, etc., as environmental problems in a reason-result relationship. however, they saw that only one student drew a picture about light pollution. it is necessary to move beyond determining the current state and create educational activities that are aimed towards preventing light pollution and thus, creating awareness among individuals. in greece and hungary, which recognized the significance of creating awareness in society on light pollution, various activities are carried out among primary school teachers and students in order to teach the subject and create awareness (demircioğlu yıldız & yılmaz, 2005). studies that have been conducted have revealed that students do not have enough knowledge on light pollution, which is an interdisciplinary issue concerning many disciplines (sadık, çakan & artut, 2011; seçgin, yalvaç & çetin 2010). as environmental problems negatively affect all living things, environment education, which is an important part of science classes, has become a significant necessity. the aim of science teaching is to educate science-literate individuals. enabling students to find solutions to problems primarily in their environment using scientific process skills is an important step towards developing science-literacy. in the seventh grade science teaching curriculum, various environmental problems are discussed. however, light pollution, which is an important issue in ecological, economic and astronomical terms, is not included. this study discusses light pollution, which is included within the topic of environmental problems and their impact in the science classes of seventh graders. this topic aims to teach students about the light pollution topic through computer-animated concept cartoons and outdoor educational activities. the learning outcomes concerning this topic in science and technology classes are given below: the students collect information on an environmental issue in our country and in the world. they present it to the classroom and discuss its consequences. the students suggest collaborative solutions towards environmental problems in our country and in the world, and participate in activities. the research problem of the study can be stated as: “what are the students’ views on computer-animated concept cartoons and outdoor education activities?” in the current study, the scenarios that are related to light pollution are supported by computer-aided concept cartoon animations in order to attract the interest of the students. in teaching through concept cartoons, some alternative ideas concerning scientific facts are drawn on paper as cartoons. however, only one of these given ideas is scientifically correct. the concept cartoons start a discussion concerning the concepts in the characters included in the cartoon and present them (keogh, naylor & wilson, 1998; keogh & naylor, 1999). thus, the aim is to reach the scientifically correct idea. in concept cartoons, generally, the questions or views of three or more characters on a daily issue are given in speech balloons (uğurel & moralı, 2006). the ideas are then discussed by the whole classroom through the cartoon characters. the purpose of this study is to create awareness among seventh graders about light pollution through computer-animated concept cartoons and outdoor science activities. it also aims to enable the students to develop solutions and reveal their views concerning the activities and practices that are carried out. students’ opinions on the light pollution application / özyürek & aydin 57 within the scope of outdoor educational activities, sky brightness was measured with a sky quality meter during the night in selected settlements of ordu province, which is located in the black sea coast of turkey, in selected dark locations and natural living environments. in addition to numerical measures, the photographs of the sky were taken with a camera with a wide-angle lens and thus, light pollution is shown visually. in addition, the students took photographs showing the light pollution within the city and conducted studies towards removing the existing light pollution. with this study, the studying and thinking environments in which the students were active and carried out activities towards solving an everyday problem were created. in order to ensure meaningful learning on light pollution and create awareness among the students about the importance of the issue, computer-animated activities, which were enriched with concept cartoons and outdoor educational activities, were carried out. the students’ views on the activities that were carried out were taken. we found that the results of previous studies indicated the positive impact of materials that had been prepared using animations in computer-aided teaching in student achievement and in removing misconceptions (yılmaz & saka, 2005; rotbain, marbach-ad & stavy, 2008; çepni, 2009; özyılmaz akamca & hamurcu, 2009). taking such results into consideration, the current study used animations in preparing the concept cartoons concerning light pollution. one of the most efficient activities that are used to make education more fun and interesting is computer animations (arıcı & dalkılıç, 2006). with their colours and motions, animations increase retention and make it easier to materialize abstract concepts or things and to visualize them in mind (rieber, 1990; çakır, 1999). thus, it is possible to create rich learning environments for students. in this study, which aims to create awareness among students about light pollution, scenarios and concept cartoons concerning the reasons and effects of light pollution were prepared, considering the negative impacts of light pollution in ecological, economic and astronomic terms (osman et al., 2001; percy, 2001; demircioglu yıldız & yılmaz, 2005). we tried to make the concept cartoons visually interesting for the students through computer animations. in addition, work sheets that were prepared for each animation with concept cartoons were administered to the students. only one of the character’s statements on light pollution in the concept cartoons is scientifically correct. in the activities that were prepared and developed, the students were asked which concept cartoon’s idea they agreed with. thus, a discussion was started and they were made to reach the correct one. the students worked in groups and stated which cartoon character’s idea they agreed with. the scientific content of the concept characters in the computer animations were examined by two experts working in the field of science. furthermore, the draft animations were examined by experts who prepare animations in flash programme and thus, the identified gaps were filled. method participants as this was an experimental study, a study group was taken instead of choosing a sample. this study was carried out with 30 seventh grade students (18 males and 12 females) studying at a secondary school in ordu province during 2014-2015 academic year. research design one group pre-test-post-test experimental design model was used in this study. in this model, assessments both before the experiments (pre-test) and after the experiment (post-test) were made (karasar, 2014). research design is shown in table 1. international electronic journal of elementary education vol.8, issue 1, 55-68, 2015 58 table 1. experimental design before the experiment experimental process after the experiment pre-test (t1) instruction through computeranimated concept cartoons and outdoor science activities. post-test (t1, t2) t1: open-ended questions related to light pollution t2: semi-structured interview form data collection the data in this study were collected through open-ended questions and semi-structured interview questions on light pollution, which were prepared and developed by aydın (2015) and used by özyürek and aydın (2015) in their project study. the open-ended questions related to light pollution were administered both as a pre-test and post-test. the experimental study took two weeks. in the first week of the experiment, five computer animated concept cartoon activities were carried out during four class hours. the pilot study of the activity was carried out with 19 seventh grade students. the activities carried out were: “why are the roads dark?”, “where is the sea?”, “the migratory routes”, “what unconscious lighting causes” and “where should i make astronomical observations?” additionally, in the second week, outdoor education activities were carried out with the students. using a sky quality meter, the sky brightness measurements were made in the school garden, city centre and yoroz city forest during the night. thus, areas where there was light pollution were identified and the common features of these places were discussed. similarly, aydın and özyürek (2015) used a sky quality meter for night sky brightness measurements in their study. in addition, the students were made to take photos of wrong lighting, which could cause light pollution. the wrong lighting practices, as well as the solutions to them, were discussed through the photographs that the students took. after the experimental study was completed, post-tests were administered. with seven students among the 30 participating students of the study, semi-structured interviews were made to determine the students’ views on the activities conducted, and these interviews were recorded. analysis of data the students’ answers to the open-ended questions were descriptively analysed. the data were reduced, categorized and put into tables. the data collected via the open-ended questions from 30 students were analysed by two researchers. to ensure its reliability, the consistency between the two analyses was controlled. the reliability rate was found to be 93%. according to miles and huberman (1994), the analysis of the data collected from an interview follows a consecutive three-step that affect each other: reduction of data, data display and conclusion drawing and verification. in order to reduce the data, raw data are coded based on certain categories (patton, 1990). within the framework of these categories, the data are coded, the essential parts are extracted and the remaining parts are taken out. the decodings of the semi-structured interviews were recorded by a tape recorder and controlled by two researchers. this ensured that the decodings were correct. the two researchers examined the data from the interviews and the themes and codes of the interviews were formed. the reliability percent between the two was found to be 95%. the data that were collected from the descriptive analysis of the interviews were summarized and interpreted according to the pre-set themes. furthermore, in order to students’ opinions on the light pollution application / özyürek & aydin 59 reflect the opinions of the individuals that were interviewed, direct quotations were included. findings the purpose of this study is to determine the impact of computer-animated concept cartoons and outdoor science activities on creating awareness among seventh graders about light pollution. it also aims to identify the students’ views on the activities that were carried out. the findings of the study are presented within the framework of a descriptive analysis of open-ended questions and themes from the semi-structured interviews, which were carried out with seven students among 30 participating students to whom openended questions were administered. table 2 includes the frequencies and students’ statements concerning their answers to the types of pollution that could be considered as an environmental problem today. table 2. the frequencies of and students’ statements to “what do you think are the types of pollution that could be considered as an environmental problem today?” question. types of environment pollution pre-test (n= 30) post-test (n= 30) f student statements f student statements air 18 air pollution, drying up of water resources, not making forestation, light pollution (2nd student). 20 air, light, environment and sea pollution are significant environment issues (2nd student). water 12 that people throw litter after they have a picnic and this litter is not collected. that factories throw their litter into water and do not attach filters to their chimneys (7th student). 16 light pollution, air pollution, water pollution and soil pollution, etc. (7th student). light 20 throwing litter on the ground, spitting on the ground, unnecessary lighting of street lights, etc. (11th student). 30 light pollution, air pollution, water pollution, environment pollution (11th student). soil 1 throwing litter on the ground, spitting on the ground, that factories throw litter into the sea, cigars polluting the air (13th student). 15 light pollution, sea pollution, and soil pollution (13th student). sound 14 light pollution, sound pollution, air pollution, environment pollution and water pollution (23rd student). 8 light pollution, sound pollution, soil pollution, environment pollution and water pollution (23rd student). as table 2 shows, the students saw air, water, light, soil and sound pollution as types of pollution that could be considered as environmental issues today. furthermore, there was an increase in the students’ answers on all types of pollutions in the post-test and they all mentioned light pollution. table 3 shows the frequencies and students’ statements concerning their answers to what kind of negative effects light pollution has on the environment question. international electronic journal of elementary education vol.8, issue 1, 55-68, 2015 60 table 3. the frequencies of and students’ statements to “what kind of negative effects do you think light pollution has on the environment?” question. negative effects of light pollution pre-test (n= 30) post-test (n= 30) f student statements f student statements on animals 13 there will be excessive light, which disturbs both humans and animals and thus, causes light pollution (23rd student). 22 there will be excessive light and the migrating birds move towards these lights and therefore, they lose their way. sea turtles hatching move towards the light instead of the sea and they could die (23rd student). on humans 17 light pollution negatively affects the human eye: eyes prone to excessive light lose their seeing ability (12th student). 18 light pollution might affect us economically. it could cause problems in our eyes (12th student). on economy 4 excessive light causes us to lose our energy. excessive lighting in a place causes to unnecessary use of light (2nd student). 11 it causes animals, for example, caretta carettas, to move towards the coast instead of the sea due to the lighting and to die, as a result. similarly, it causes us to lose our energy (2nd student). on astronomical studies 6 we cannot see the stars during the night because there is so much light, we spend a lot of energy (3rd student). as it is seen in table 3, the students indicated that light pollution has negative impacts on animals, humans, economy and astronomical studies. there was an increase in the frequency of students’ answers to the negative effects of light pollution on the environment after the application. meanwhile, there was no student who could state that light pollution could negatively affect astronomical studies. there were six students in the post-test who mentioned this fact. in order to find out the opinions of the students on the activities and practices that were carried out, they were asked: “were there any differences between how the light pollution topic and other topics were previously taught?”. all of the students indicated that there were differences in how the light pollution topic was taught, compared to the other topics that were previously taught. table 4 shows the students’ statements concerning what kind of differences the students see between how the light pollution topic was taught, compared to other topics that they were taught before. students’ opinions on the light pollution application / özyürek & aydin 61 table 4. the frequencies of and students’ statements to “what kind of differences did you see in how the light pollution topic in science class was taught compared to other topics that were taught before?” question. in table 4, it is seen that the interviewed students stated that there were such differences as making measures, carrying out outdoor education activities and practices in how the light pollution topic was taught in science classes, when compared to other topics of the same course. table 5 shows the views of the students about what caught their attention the most when the light pollution topic was taught. table 5. the frequencies of and students’ statements to “what caught your attention the most when the light pollution topic was taught?” question it is seen in table 5 that the students stated that they were impressed with the animation of migrant birds, caretta-carettas animations, taking measurements with the sky quality meter and astronomy animations while the light pollution topic was taught. table 6 shows the views of the students on what they think about the activities and studies that were carried out while the light pollution topic was taught. differences in how the light pollution topic was taught f student statements measurements that were made 3 in other classes, it was mere teaching but when we were taught about light pollution, we conducted activities and made measurements (8th student). outdoor education activities that were made 5 in the activities that were related to light pollution, we carried out outdoor education activities. however, in other topics, we only made activities within the school; the difference is doing activities outside (10th student). activities that were carried out 4 with the visual activities that we carried out, i learnt more (23rd student). things that caught the students’ attention when the light pollution topic was taught f student statements animations of migrating birds 2 i was very impressed with the slowing down, hindering and even stopping of the migration of migrant birds and, as a result, with their death (25th student). caretta-carettas animations 3 that caretta carettas could not move towards the sea because of the artificial lighting and they died (27th student). taking measurements using a sky quality meter 2 measuring light pollution with a sky quality meter (3rd student). astronomy animations 1 i was impressed with whether or not we see the stars and the brightness of the activity of the stars (8th student). international electronic journal of elementary education vol.8, issue 1, 55-68, 2015 62 table 6. the frequencies of and students’ statements to “what do you think about the activities and studies that were carried out while the light pollution topic was taught in science classes?” table 6 shows that the students stated that the activities and studies that were carried out while the light pollution topic was taught were fun, interesting, easy to remember and didactic/informative. table 7 shows the views of the students on the activity that they liked the most while the light pollution topic was taught. table 7. the frequencies of and students’ statements to “can you please tell the activity you liked most while you were taught light pollution? why did you like that activity?” question according to table 7, it is seen that, of all the activities conducted while the light pollution was taught, what the students liked the most were making measurements using the sky quality meter, group discussions and decision-making after the animations and caretta carettas animations. table 8 shows the views of the students on whether they would like other topics to be taught in the way that light pollution was taught. opinions on the light pollution activities f student statements fun 3 it was fun to tell our personal ideas and discuss them with the group that was formed after we watched the animations on light pollution... (27th student). interesting 3 outdoor education activities were fun. it was interesting and enjoyable to have the chance to make measurements with the sky quality meter... (10th student). easy to remember 2 in science classes, it is easy to remember the things that were prepared with animations (25th student). didactic/informative 5 first of all, with these activities, i produced an idea, i mean, i did not know much about light pollution and i learnt more (23rd student). the activity that the students liked the most f student statements taking measurements using the sky quality meter 5 …the data that we measured with the sky quality meter and, in particular, the data from the city forest attracted my attention. for example, i did not know that there was so much light pollution around the school. we learnt it more easily with that device. in other words, we learnt more... (23rd student). group discussions and decision-making after the animations 1 ...the activity that i liked the most was the animations and the activities in which our opinions were asked. we expressed our ideas freely and learnt about the topic (25th student). caretta carettas animations 3 i most liked the caretta carettas animations and i was very sad that they die because of light pollution (8th student). students’ opinions on the light pollution application / özyürek & aydin 63 table 8. the frequencies of and students’ statements to “would you like to learn other topics in science classes in the way you were taught light pollution? why?” question according to table 8, the students stated that the reasons behind demanding other topics to be taught in the way that light pollution was taught are because it is fun and interesting. table 9 shows the opinions of the students on what kind of benefits learning science classes in the way that light pollution was taught would bring. table 9. the frequencies of and students’ statements to “what kind of benefits do you think learning science classes in the way you were taught light pollution will bring? it is seen in table 9 that the students stated that learning other topics in science class in the way that light pollution was taught would be beneficial. this is because it will ensure more detailed/comprehensive learning and permanent learning. it will also make learning easier and more fun, as well as creating awareness and raising consciousness. the reasons for demanding other topics to be taught in the way that light pollution was taught f students’ statements fun 3 sure i would want that. first of all, it was fun. we normally do not do such wide scale activities at school what we normally do is very limited (10th student). interesting 4 there are more visual activations and i liked them more, they were more interesting (23rd student). benefits of learning science in the way that light pollution was taught f students’ statements detailed /comprehensive learning 3 ...in other subjects, we cannot spend as much time as we did on light pollution... (27th student). … for example, i did not know that there was so much detail and information about light pollution. with such things as outdoor observations, i believe i will learn a lot more (23rd student). more permanent learning 3 …it is more permanent and we learn better (11th student). easy to learn 2 we learn more easily when we are doing enjoyable activities or experiments ... (10th student). fun 1 ...for example, while we are taught a topic, we are sometimes bored and do not want to listen to the teacher or we cannot understand because we are bored. however, the measures we take outside of school are fun and permanent (10th student). creates awareness/ raises consciousness 3 for example, before we did the activities and practices concerning light pollution, i would not realize the places where there is light pollution and would just pass by (25th student). international electronic journal of elementary education vol.8, issue 1, 55-68, 2015 64 discussion, result and suggestions computer-supported concept cartoons that enabled the students to fill work sheets, which are made up of concept cartoons, in groups and outdoor school activities increased the rate of correct answers to the open-ended questions in the post-test. while the number of students who indicated light pollution as an environmental issue was quiet small in the pre-test, this number increased in the post-test. similarly, sadık, çakan and artut (2011) examined 206 fifth grade students’ perceptions on environment problems studying at three different primary schools from different socio-economic levels via the pictures that the students drew. they saw that the students drew the loss of forests, air pollution, decreasing species, spoiling ozone layer, sound pollution, soil pollution, global warming, etc., as environmental problems in a reason-result relationship. however, they saw that only one student drew a picture about light pollution. additionally, taşlıdere (2013) indicated in his study that the work sheets that were enriched with concept cartoons had positive effects on the students’ conceptual learning. it is believed that the activities that are carried out with seventh grade students concerning light pollution will ensure that the students have awareness on the ecologic, astronomical and economic results of light pollution and that they will develop solutions towards preventing light pollution. all of the interviewed students from the classroom, in which computer-animated activities enriched with concept cartoons and outdoor education activities were carried out, stated positive views on the activities that were conducted. the students stated that they learnt the reasons and impacts of light pollution well. in addition, it was seen during the interviews that the students did not think that light pollution could harm some living things. the students also indicated that they enjoyed making computer-animated activities and outdoor education activities. the students stated that the way light pollution was taught was different from the way that other science topics were taught and that they did activities, outdoor education activities and measurements. all of the students stated that they wished science classes were taught in the way that light pollution was taught. they also stated that learning in this way is more enjoyable and interesting. the students stated that the activities were fun, interesting, informative and easy to remember. the following suggestions can be made for further studies: concept cartoons, which prevent misconceptions and support students to learn by searching and questioning, can be used to determine and remove misconceptions. it is believed that including light pollution by emphasizing its ecological, astronomical and economic significance within the scope of a science teaching programme will create awareness of the topic among students in other words, among the future generation. it is believed that using computer-aided concept cartoons in learning environments and learning science through outdoor education activities will contribute to meaningful learning among students and will make classes more interesting for them. • • • students’ opinions on the light pollution application / özyürek & aydin 65 references arıcı, n. & dalkılıç, e. (2006). the contribution of animations to computer assisted education: an application sample. kastamonu education journal, 14(2), 421-430. aslan, z. (2001). light pollution, what do other countries do? national light committee. http: www.tug.tubitak.gov.tr aydın, g. & özyürek, c. (2015). environment education through outdoor science activities: light pollution case. oxidation communications, 38, 608–621. aydın g., (2015). the effects of computer-aided concept cartoons and outdoor science activities on light pollution. international electronic journal of elementary education, 7(2), 143-156. crawford, d. l. (2001). light pollution changing the situation to everyone’s advantage. preserving the astronimical sky, iua symposia, 196, 33-38. çakır, h. (1999). use of graphics and animation techniques in computer-assisted education, master’s thesis, gazi university, institute of educational sciences, ankara. çepni, s. (2009). effects of computer supported instructional material (csim) in removing, students’ misconceptions about concepts: “light, light source and seeing”. energy education science and technology, part b: social and educational studies, 1(2), 51-83. çetegen, d. & batman, a. (2005). light pollution. journal of i̇stanbul kültür university, 2, 29-34. demircioğlu yıldız, n. & yılmaz, h. (2005). light pollution: problems and solution proposals. journal of atatürk university faculty of agriculture, 36(1), 117-123. hanel, a. (2001). the situation of light pollution in germany. preserving the astronomical sky, iua symposia, 196, 142-146. karasar, n. (2014). scientific research methods (27th edition). nobel publications, ankara. keogh, b., naylor, s. & wilson, c. (1998). concept cartoons: a new perspective on physics education. physics education, 33(4), 219224. doi: 10.1088/0031-9120/33/4/009. keogh, b. & naylor, s. (1999). concept cartoons, teaching and learning in science: an evaluation. international journal of science education, 21(4), 431446. doi: 10.1080/095006999290642. miles, m. b. & huberman, a. m. (1994). qualitative data analysis (2nd edition). thousand oaks, ca: sage publications. osman, a. i. i., isobe, s., nawar, s. & morcos a. b. (2001). light pollution and energy loss from cairo. preserving the astronomical sky, iua symposia, 196, 107-110. özyılmaz akamca, g. & hamurcu, h. (2009). science and technology education based on analogies, concept cartoons and predict-observe-explain techniques. e-journal of new world sciences academy, 4(4), 1186-1206. özyürek, c. & aydın, g. (2015). creating an awareness of light pollution: ordu province case. a research project of ordu university, ar-1371. patton, m. (1990). qualitative evaluation and research methods, 169-186, beverly hills, ca: sage. percy, j. r. (2001). light pollution: education of students, teachers, and the public. preserving the astronomical sky, iua symposia, 196, 353-358. rieber, l. p. (1990). using computer animated graphics in science instruction with children. journal of educational psychology, 82, 135-140. rotbain, y., marbach-ad, g. & stavy, r. (2008). using a computer animation to teach high school molecularbiology. journal of science education and technology, 17, 49–58. doi: 10.1007/s10956-007-9080-4. http://www.tug.tubitak.gov.tr/ international electronic journal of elementary education vol.8, issue 1, 55-68, 2015 66 sadık, f., çakan, h. & artut, k. (2011). analysis of the environmental problems pictures of children from different socio-economical level. elementary education online, 1066-1080. seçgin, f., yalvaç, g. & çetin, t. (2010). the environmental problem perceptions of primary school 8th grade students through cartoons. proceedings book of international conference on new trends in education and their implications (iconte), 11-13 november, isbn: 978 605 364 104 9, pp. 391-398. taşlıdere, e. (2013). the effect of concept cartoon worksheets on students’ conceptual understandings of geometrical optics. education and science, 38(167), 144-161. uğurel, i., ve moralı, s., (2006). cartoons and their use in mathematics education. journal of national education, 170. yılmaz, m. & saka, a. z. (2005). material development and application based on worksheets in computer-assisted physics education. the turkish online journal of educational technology, 4(3), 120-131. students’ opinions on the light pollution application / özyürek & aydin 67 appendix – a picture 1: an outdoor science education activity with students picture 2: an example for light pollution international electronic journal of elementary education vol.8, issue 1, 55-68, 2015 68 www.iejee.com this page is intentionally left blank http://www.iejee.com/ international electronic journal of elementary education, 2015, 7(2), 199-216. issn:1307-9298 copyright © iejee www.iejee.com all-day schooling: improving social and educational portuguese policies jorge martins  university of porto, portugal ana vale university of porto, portugal ana mouraz university of porto, portugal received: 28 december 2014 / revised: 8 january 2015 / accepted: 4 february 2015 abstract over the past decade, several european countries have implemented policies and programmes leading to the introduction of the concept of ‘all-day schooling’, thus acknowledging the need to guarantee the guard of all children and to enhance equal opportunities of success at school. the portuguese ministry of education created and funded the curricular enrichment activities programme as a measure to support the generalisation of all-day schooling in portuguese primary schools. our study aimed at evaluating the reach of the political measures associated with all-day schooling, as was implemented in portugal by the aec programme, by focusing on two central dimensions: the political and the curricular. two cases of different local decisions were studied. the results revealed a unanimous valorisation of the philosophy and the objectives underlying the aec programme, which gives shape to a ‘good measure’ of social and educational public policies, consistent with the nuclear project of allday schooling. they also showed, however, that the dimension of the curricular enrichment still needs to be improvedover the past decade, several european countries have implemented policies and programmes leading to the introduction of the concept of ‘all-day schooling’, thus acknowledging the need to guarantee the guard of all children and to enhance equal opportunities of success at school. the portuguese ministry of education created and funded the curricular enrichment activities programme as a measure to support the generalisation of all-day schooling in portuguese primary schools. our study aimed at evaluating the reach of the political measures associated with all-day schooling, as was implemented in portugal by the aec programme, by focusing on two central dimensions: the political and the curricular. two cases of different local decisions were studied. the results revealed a unanimous valorisation of the philosophy and the objectives underlying the aec programme, which gives shape to a ‘good measure’ of social and educational public policies, consistent with the nuclear project of allday schooling. they also showed, however, that the dimension of the curricular enrichment still needs to be improved. keywords: educational policies; educational equity; all-day schooling; curricular enrichment activities.  jorge martins, rua alfredo allen, 4200-135, porto, portugal tel: +351 226 079 700 e-mail: jorgemartins51@gmail.com. http://www.iejee.com/ international electronic journal of elementary education vol.7, issue 2, 199-216,2015 200 introduction educational policies are in line with globalization and policy borrowing movement that crossed europe in the last 20 years and aloud extensive reforms of education systems (steiner-khamsi, 2012) that accompany wider reforms of the welfare states. welfare states should evolve from a social protection to a social promotion framework, in which educational equity for all is essential (bolívar, 2012). however, the idea of universal welfare has been increasingly affected by market-oriented and globalised economy’s close connections with educational policy (põder, kerem, & lauri, 2013). therefore, given this changing educational landscape, this text, focused in a unique programme implemented in portugal is to be seen as a paradigmatic case of the controversial uses and perceptions that “all day schooling programme” could cause in educational discourses, school everyday practices, and educational outcomes, within other educational contexts. in contemporary societies, thousands of children and adolescents are left every day to their own devices after they leave school. as a result of a change of the patterns of employment, as well as of the family typology, many parents do not succeed in guaranteeing the care and supervision of their children after school (armstrong & armstrong, 2004). it is estimated that in 2008, in the usa, 7.5 million children were left without supervision after school. in europe, this situation has reached alarming dimensions, namely in germany, a country which particularly penalises the children of divorced parents, of single parents, and resourcepoor families, who have to work in order to guarantee the well-being of their children (pfeifer & holtappels, 2008). after-school hours have thus become a problematic period, on the one hand, because children are more susceptible to being neglected, and on the other hand, due to the relationship that has been established, both in germany and the united states of america, between that period of time and the academic difficulties experienced by the students. in the usa that time is still associated with the emotional and behavioural problems experienced by children and adolescents, as well as with delinquency and the use of drugs (armstrong & armstrong, 2004). in the last decade, acknowledging the need to guarantee the guard of all children, as well as to ensure equal opportunities for academic success to the children and adolescents from socio-economic disadvantaged backgrounds, several european countries have implemented policies and programmes which led to the introduction of “all-day schooling”, i.e., to the expansion of school time. amongst these countries were germany (reh, rabenstein, & fritzche, 2011; pfeifer & holtappels, 2008; den besten, 2010; schnniter & häselhorff, n.d.) and portugal. in the case of germany, as stated by the above mentioned authors, the creation of “all-day schooling” programmes was a strategy to avoid the low levels of performance of the students that had been disclosed by the programme for international student assessment (pisa) in 2000. those low levels were seen as a negative result of the fact that a remarkable number of children and adolescents were left to their own devices, i.e., were left without adult supervision in the period between the end of school and the end of their parent(s)’ day at work. in this country, the all-day schooling programme, which was implemented over a decade ago, has won increasing importance. still, it has not been extended to all the state schools yet (reh et al., 2011; pfeifer & holtappels, 2008; den besten, 2010; schnniter & häselhorff, n.d.). all-day schooling: improving social and educational portuguese policies / martins, vale & mouraz 201 accordingly with gepwert, hofmann, and hopmann (2012), the “current state of research shows that all-day schooling is seen as a necessary response to social and societal change, structural change in the family, social inequality and unequal education opportunities” (gepwert, hofmann, and hopmann, 2012, 17). furthermore, all-day schooling is seen by policies parties as an interesting measure that contributes to promote equity. in portugal, the all-day schooling programme has been generalised to the 1st cycle of all the state schools during the school year of 2006-2007, through the aec (curricular enrichment activities) programme1. this programme became a relevant political tool for the fulfilment of the peti (all-day schooling programme)2, a decentralised educational public service meant for social intervention with a double finality: 1) to provide, free of charge, a number of activities capable of enriching the curriculum of the 1st cycle of elementary education; 2) to provide social measures for family support. when compared to the above-mentioned german case, the portuguese case reveals greater concern for the social dimension of the measure. this study aimed to evaluate the scope of the political measures associated with all-day schooling as it was implemented in portugal by the aec programme and put into practice in 2009/2010. the two fundamental dimensions of its means of operating were identified: the political and the curricular. the study focuses on two cases of different local decision. political dimensions of the programme the curricular enrichment activities programme is a powerful instrument for intervention and change in several domains of the educational field and in particular in the administration and management of the 1st cycle of elementary education. this programme has asserted itself as a relevant component of a decentralised educational public service, which is contracted and open to the laws of the educational market. designed at first by the socialist government (2005-2009) as a tool for the policy of “modernization” of the country, in line with the prevailing european policies, the programme aimed to promote the early teaching of the english language in the 1st cycle of elementary education. it thus aspired to pair the portuguese educational system with the “european patterns” with respect to the “high level of training and qualification of the future generations”, as well as to the “early development of competencies, in the framework of the increasing mobility of people in the space of the european union” (ordinance no. 16 795/2005). however, the programme for the generalisation of the teaching of english in the 3rd and 4th years of the 1st cycle of elementary education was soon extended and converted, by the diverse dynamics of its decentralised development, into the aec, a programme with strong social concerns that aimed to promote relative equality of educational opportunities. many were the changes the programme introduced in the curricular offer of basic education: (i) it put forward, right from the beginning, the concept of “enrichment” in the context of an “old” curriculum which had reached a critical point, (ii) it reformulated the finalities of the 1st cycle in the framework of the pedagogical continuity desired by the schools groups, (iii) it led to the recruitment of new professional profiles meant to work with children within the same class-space, (iv) it established new relationships with respect to the way schools and groups of schools were supposed to work, (v) and it forced the sharing of educational tasks between the central administration, the municipalities and the schools. however, the programme took root mainly in the social sphere. this happened not only because it provided answers to many problems families had with the management of their daily lives – when it was reconfigured as peti –, but also because it democratised a number of educational advantages, which were within reach of certain social strata only. the programme also innovated as it provided ground for the compatibility of the promotion of those apparent international electronic journal of elementary education vol.7, issue 2, 199-216,2015 202 educational equal opportunities and the diverse curricular proposals made by different entities, having thus tried to respond to the interest evinced by local communities. in spite of some criticism on the part of the anmp – associação nacional de municípios portugueses (portuguese association of municipalities), who put back on the agenda the discussion about the circumstances pertinent to the operating of that measure, the programme relies on a strong involvement on behalf of the municipalities. it was, in fact, mainly the gradual juridical recognition of the social importance of the autonomous educational intervention of the municipalities that led the government to successively change the normative definition of the support activities offered by the municipalities to the 1st cycle of elementary education: being at first described as “extra-school activities for the occupation of leisure-time”, they were subsequently named “support and complementary educational activities”, then “complementary curricular activities”, and finally “curricular enrichment activities”. for the first time, “subject areas” chosen by the promoting entities (municipalities, parent associations and groups of schools) among a number of areas considered by the ministry of education relevant for the promotion of success (thus being funded) were integrated into the curriculum of the 1st cycle. these “subjects” have “programming guidelines”, support material, staff and schedules specially designed for them and defined by the promoting entities as if they were real curricular subjects. it is thus a form of educational decentralisation, which manifested itself in the first and most significant reconfiguration of the old relationship between the two powers that used to oversee the 1st cycle. in spite of a renewed “school-centrism” (correia & matos, 2001), the doors of a field that had so far been of the exclusive responsibility of the central administration were opened to local institutions and entities: they now define the curriculum – what ‘enriches’ –, as well as which classroom is going to be used for that purpose. the support component for families that the programme works with is mainly characterised by an expansion of the time children are now spending in educational activities in the school environment (table i). the implications of this extension of the opening hours of 1st cycle schools forced the adoption of new equipment and changes to physical and human resources. besides the characteristics mentioned in table 1, it is necessary to clarify that the planning of the activities is carried out by coordinators of the aec programme, while the pedagogical supervision is of the responsibility of the head teachers of each class. it is also worth noting that those in charge of schools may, when necessary, render the schedule of the curricular activities more flexible in order to provide the best conditions for the implementation of curricular and enrichment activities. curricular issues the aec programme aims to achieve the double objective of guaranteeing, to all students of the 1st cycle, the offer – free of charge – of a variety of learning activities that may contribute to the enrichment of the curriculum, as well as the accomplishment of the government’s priority of promoting the articulation between the operational conditions of schools and the organisation of the social response to families’ needs. the student’s length of stay at school is thus extended, and all is done to render that period pedagogically enriching and complementary to the learning associated with the acquisition of key competencies. all-day schooling: improving social and educational portuguese policies / martins, vale & mouraz 203 table 1. characterisation of the aec programme in the context of the peti aec programme optional formal curriculum compulsory family support component optional entity in charge local authority: municipalities, parent associations, school groups central administration ministry of education central administration and local administration length of stay at school (40 hours per week) 7.5 hours 25 hours variable up to 15 hours place classroom /other spaces around the school building / spaces outside the school facilities (swimming pools) classroom/school classroom / other spaces around the school building / spaces outside the school facilities compulsory offer optional offer . english . learning support . music . physical and sport activities . artistic expression . other formal curriculum meals and surveillance operational staff aec teachers of each specific area head teacher non-teaching staff attendance control regulated by schools, but without any effect on the students regulated by the ministry of education, with effect on the students not applicable assessment of student learning without any effect on academic progression with effect on academic progression this brief description of the intentionality behind the aec programme, as well as of the way it has been put into action, entails two concepts, which are worth revisiting: the concept of allday schooling and the concept of informal curriculum. the concept of all-day schooling can be defined as the !full-time educational occupation of students over the course of school time and in the physical space of the school” (pires, 2007, p. 78). as it corresponds to an important change of the learning time, as well as of the agents that are in charge of its definition, the concept defies what one usually understands as schooling. all-day schooling is a response to the challenges – never before experienced – raised by mass schooling in portugal, and has forced people to face new issues, such as the idea that social justice should be guaranteed above all. this new reality of having all children at school involves thinking about what should be taught, and how it would serve the goals (which goals?) of the education provided by the state (leite, 2006). this worry is ever more relevant at a time such as the one we are currently living, when political speeches resort to the flag of international electronic journal of elementary education vol.7, issue 2, 199-216,2015 204 qualification as a way to reach the 2015 targets for education and contribute to the europe of knowledge defined by the lisbon strategy. the concept of all-day schooling is often associated with an egalitarian intent, as it aims to ensure equal educational opportunities to all children (dobert, echard, & sroka, 2004; ocde, 2007). in this regard it is essential to guarantee that the “real and effective equality offers adequate and differentiated opportunities so that all students, no matter their starting points, their needs or circumstances, can go beyond the minimum knowledge and acquire basic school learning” (muñoz, 2005, p. 17). with the aec programme, portugal is following the political measures that have been implemented in other european countries for the past twenty years, with the same purpose of positive discrimination (demeuse, frandji, greger, & rochex, 2008). such measures were conceived in order to resolve or minimise, in one go, some social inequality, thus contributing to improve the level of success of students, during their years of formal schooling, as prescribed by the tendencies identified by the oecd with regard to the forms of promoting equity in the field of education. “to strengthen the links between school and home to help disadvantaged parents help their children to learn; to provide strong education for all, giving priority to early childhood provision and basic schooling” (oecd, 2007, p. 9) are two of the ten steps to support social equity. the concept of all-day schooling also incorporates the idea that some of the learning provided by the school, during that supplementary time, is more important and becomes socially more homogeneous over time than the activities that the families and the children themselves would be able to arrange. as argue by authors like magalhães and stoer (2002) this effort contributes also to a new commitment that middle class addresses to school system. an additional finality associated with all-day schooling concerns its capacity to prevent marginal behaviour on the part of the students that could occur if they were left alone without adult supervision. furthermore, the national policy chose to value structured educational action, to be held, in most cases, at school, or to resort to a model of schooling, as happens with the aec programme. finally, all-day schooling is associated with the idea that students, in their capacity as learners, are somehow unaccountable. in fact, although it is said that the intent is to promote the autonomy of students, it contributes instead to a more extended reliance on adults, who are taken as their reference. it is precisely within the dimension of the offer that characterises all-day schooling that it makes sense to examine the concept of informal curriculum, as well as the links that the concept has established with other concepts, which may contribute to the understanding of its theoretical density, namely the concepts of territorialisation, of curricular coherence and curricular articulation. the concept of informal curriculum is defined by the learning that the school intends to foster in its students, which results from the values that shape their educational projects or their identities in their role as educational organisations, but which are not an explicit target of a formal instruction and of an evaluation of the results evinced by the students (pacheco, 1996). the underlying conviction, shared by many authors of the critical sociology of the curriculum (forquin, 1993), is that the walls around schools are merely symbols of other social control walls, which the school conveys. therefore, when an educational system does not just give shape and determine the formal curriculum, but chooses and funds a number of choices regarding the offer for the occupation of free time instead, it is formatting the school culture all-day schooling: improving social and educational portuguese policies / martins, vale & mouraz 205 twice, thus contradicting the idea of curricular territorialisation. in other words, it formats school culture as it chooses a formal curriculum, but also because, taking over the parents’ role of conveying the values that are closer to their family experiences, it forgets the principle that all learning should be locally significant and that it should depend on local agencies, as vindicated by curricular territorialisation. as mentioned by leite (2005, p. 5), “the foundations that legitimise the curricular contextualisation, at a micro level, refer to the possibilities of local action in the process of curricular decisions, within the logic of territorialisation of education”. this concept is founded upon the possibility offered to local territories, as well as to their agents, to decide matters concerning education in accordance with principles and interests that are locally relevant. in portugal, curricular territorialisation has been commonly used to justify a curricular offer of a compensatory nature, in the context of elementary education, as it has been primarily oriented to the formative offer aimed at students who have failed regular schooling. on the contrary, the rationale behind the aecs has been to enrich the formal curriculum or to provide a number of physical or artistic activities of a more universal nature. we believe it is legitimate to read in that difference evidence of a certain return to the movement back to basics. it certainly is what the practice (also preached in contemporary political speeches) of maintaining the pressure on the areas of learning considered more noble and more basic – thus making time available for the learning of those activities – indicates. thus, as a result of the inscription of a set of findings associated with artistic expressions in an area of informal curriculum, it has made room for what is considered essential. that is why all-day schooling – and the aecs in particular – is seen today as a way of implementing the tendency of the back to basics, of the fundamental literacies (reading, writing, numeracy, communication). the dimensions of the curriculum that are more connoted with the achievement of projects or with a training of a more humanistic nature are thereby relegated to other periods of time. as a result, we would have two kinds of schools: in the morning, the school of the essential curriculum with the aim to ensure the basic literacies, and at the end of the day the school of the territorialised curriculum with a more informal and experiential nature. two objections can be made to this unfulfilled promise of articulating the formal with the informal curricula: 1) if the curricular coherence and significance is entrusted with the informal part of the curriculum, do we not risk a divide between the experiential and the official curricula? in fact, the experiential curriculum can very well transform experiences into learning, whereas the official curriculum is based on the idea of an external knowledge appropriated by a student who has in mind the final exam that will confirm that appropriation. 2) should we not beware of the results of a relative inconsequential superficiality of the educational offers as they risk the temptation of trying to ensure a high range of educational experiences, which is very close to the exaggerated experientialism that characterises contemporary society? (lipovetsky, 1989). from another point of view, the curricular coherence is not restricted in regard to the application of the knowledge which derives from the formal curriculum to the informal one, or vice-versa, to the continuity between knowing and acting, but rather to the transforming potential that the learned knowledge can have on the lives and the contexts of the learning individuals. such is its potentiality, such is its frailty. the concept of curricular articulation has acquired increasing importance in the curricular development, as it is discursively presented in the documents which shape the curricular decisions of the schools, and results both from the legal acts and the observation that the curriculum cannot just be a conglomerate of juxtaposed parts. theoretically associated with the globalising dimension of the curriculum, the concept has been split into two vectors that international electronic journal of elementary education vol.7, issue 2, 199-216,2015 206 explain how that totality has been achieved, either from a diachronic or a synchronic perspective. this is what we call the vertical or the horizontal articulation. 1. the “vertical curricular articulation is present in the continuity of the levels/cycles/years, in the hierarchy of decisions and in the balance between the formative curricular components and their extension; 2. the horizontal curricular articulation can be observed in the correlation which exists inside each unity and each organ, in the transversal relationship between areas/subjects in the same year and in the coherence between the components that make the curriculum operational”. (leite & pacheco, 2010, p. 6) the curricular articulation reaches its full sustainability in three planes that correspond to three other relevant axes of the curricular work, namely: the development of the students; the work of teachers and curricular agents; the structure of the system where the articulation is politically shaped. these three axes define what is to be understood as a good practice of curricular articulation, i.e., an articulation able to identify and intervene in each school at the level of decision or curricular implementation where it seems to be most needed. a good practice of curricular articulation also ensures a work of curricular coherence, which is centred on the curricular alignment and made available to the collective subject responsible for its implementation – the teachers who, over time, work with the same student. as an extension of the former reason, a good practice of curricular articulation values the availability of teachers to work in a collaborative way with their peers. methodology in order to evaluate the implementation of the aec programme in the district of porto, we decided for a “multi-case” qualitative study (triviños, 1987), as it is the most adequate for the study of contemporary events (yin, 2002). as it is customary in this methodology, the choice of two cases, in a total of four schools, was made in order to ensure the variability that is present in the universe of the promoting entities responsible for the aecs in that district. case a, chosen among the universe of groups of schools of which the porto town council was in charge, represents 16 cases (stake, 2005), or groups of schools. case b, on the other hand, stands for the exception to the rule: schools integrating an alternative offer promoted by a parish council. in all cases, we selected groups of schools that might contribute, on a voluntary basis, to a better understanding of the circumstances, as well as of the implications and the problems resulting from the implementation of the aec programme. the data collecting tools were: official documents, interviews, observation of the aec activities, texts and drawings by the students. the set of documents that have been examined is composed of: • legal documents by the ministry of education • programming guidelines issued by the ministry of education • educational projects and annual activities plans • aec annual planning • aec evaluation forms the interviews, individual or in group, according to an open and complex model, were made to the representatives of the local promoting entities, to the people in charge of the administration and management of the aecs, to the directors of the groups of schools and the directors of the schools, to the head teachers and to the aec schools, to the all-day schooling: improving social and educational portuguese policies / martins, vale & mouraz 207 representatives of the companies (hired as a result of the political option of the town council) and to the entities responsible for the provision of services, and also to the students’ parents and guardians. the criterion of availability was used for the selection of all the participants, as well as for the phases of observation and gathering of the students’ work. the observations were both informal and formal. the latter were based on “observation protocols”, whereas the former, made over the course of field visits, originated a number of research notes that were examined as well. a qualitative analysis of all the sources was implemented, and nvivo software was used. the referential analysis, initially oriented by the research questions, was subsequently enlarged by emerging categories. results in this section, we present the results of the research carried out in order to meet the research questions. these were as follows: 1. what distinguishes the organisational model of the programme of the submitting entities? 2. what characterises the educational offer of the submitting entities? 3. what are the effects of the different offers on students’ learning? 4. what are the effects of the different offers on personal and social development? the text follows the axes: i) characterisation of the offer and working regulations; ii) impacts on the aec programme. characterisation of the offer and work regulations with regard to the nature of the offer, we observed that, in both cases, it was composed of a pack of activities that would grant the highest funding on the part of the ministry of education (english, music, physical and sport activities and other). thus, although there were small variances regarding the selected optional activities and week workload of the activities, there were no significant differences. in the first year, except for the optional activity, the offers were fairly homogeneous: music, english, and learning support. in case b, the optional activity consisted of two weekly slots of portuguese learning support. in case a, in two of the schools the optional activity consisted of ict (information and communication technology) and in the third one of math learning support. in the 2nd year, the coincidence was even bigger: besides music, english and learning support, the optional activities consisted, in both cases, of math learning support. in the 3rd and 4th years, the offer was exactly the same. although the promoting entities were different, we came to the conclusion that the offer was not very different (1st and 2nd years) or even quite the same (3rd and 4th years). probably, this may have resulted from the situation created by the ministry of education when it defined the funding of the activities. on the other hand, in breach of what has been established by the founding text of the aec programme, the definition of the offer counted, in both cases, with a modest participation of the school groups, which – it is important to emphasise – should have been more substantial. as was happening at the national level, in the schools of case a the activities were held in the classrooms or the school facilities. in case b, because of the high number of students and of the low number of rooms available, alternative places had to be found for the extracurricular activities, and protocols with neighbouring entities were signed, as bernet (1999) recommends, in order to make the best use of the educational resources of local institutions. international electronic journal of elementary education vol.7, issue 2, 199-216,2015 208 with regard to the working hours of the aecs, as stated above, there were big differences between the school in case b and the schools in case a. in the latter, the activities were held from 4:00 pm to 5:30 pm, i.e., after the curricular activities. in case b, the activities were held in the morning when the students had their curricular activities in the afternoon, and viceversa. as a rule, aec classes integrated students of the same year, and each group had no more than 20 students, in case a, and no more than 15, in case b. in both cases, however, there were some classes where children from the 1st and 2nd years, on the one hand, and from the 3rd and 4th years, on the other, were brought together. also in both cases, the students’ parents and guardians could decide whether they wanted their children to attend the activities, and in case they enrolled their children, the latter were bound to attend all the offered activities. nonetheless, because of the situation described above, only in case b did that option involve the choice for the physical space of the institution where they wanted their children to attend the aecs. setup and human resources in their quality as promoting entities of the aec programme, both the town council and the parish council felt they should create new structures to coordinate and accompany the implementation of the activities in the schools for which they were responsible. in case a, a “technical team” was established, and they directly depended on the office for educational policies, and were trusted with the accompaniment, coordination and evaluation of the programme. in case b, a team called ‘technical device’ was brought into being, and they depended on the vice-president of the parish council. in both cases, the above-mentioned structures had forms of coordination of their own and very specific competencies. as we can conclude from table 2, part of the competencies of those teams were very similar, from the meetings with the representatives of the groups of schools to the method used for the evaluation of the activities and the aec programme, as a whole. however, in case b the competencies of the coordinating team included a pedagogical dimension. on the other hand, both the team of case a and the companies – according to what had been established by law – left the pedagogical issues to be managed by the head teachers of each class. the team in case a centred its action on the control and monitoring of the fulfilment of the contract documents signed with the companies hired for the implementation of the programme. the team in case b revealed a veritable preoccupation with the pedagogical dimension, and this resulted in the elaboration of a teaching pack – a portfolio – that aimed to provide support for the people involved in the implementation of the programme, as well as in the work of pedagogical guidance provided by the team coordinator to the aec programme teachers, which included the articulation of the aecs with the schools’ plans of activities and the curricular projects of each class. it should be further noted that the promoting entity, in case a, delegated to the companies which acted as service providers the hiring of the activities, whereas the promoting entity b took charge of that responsibility. in both cases, the teachers who were hired were graduates. all-day schooling: improving social and educational portuguese policies / martins, vale & mouraz 209 table 2. devices created by the promoting entities parish council town council human resources sport, education and youth office technical team dependency vice-president of the pc alderman in charge of education composition coordinator area coordinator school coordinator coordinator of the educational services head of office companies company coordinator area coordinator mission management, coordination and evaluation of the implementation of the programme pedagogical accompaniment of the aecs pedagogical accompaniment of family support activities management, coordination and evaluation of the implementation of the programme duties relationship with entities elaboration of a pedagogical portfolio production of pedagogical documents curricular articulation with the structures of the schools and of the groups of schools evaluation of the aec teachers update of the evaluation methodologies of the aecs relationship with other entities, namely the coordinators of the hired companies (edutec, espalha ideias, portolazer) and with the coordinators of the schools and of the groups of schools production of documents connected with the programme elaboration and management of the questionnaires to be sent to schools evaluation of the aec teachers annual meetings with representatives of the groups of schools impact of the aec programme on the schools one of the most obvious impacts of the programme was the expansion of the school offer to eight daily hours, five for the formal curriculum and three for the enrichment activities. in most portuguese schools, all the activities are held in the same physical space. this redefinition of the ‘intra-scholar temporality’ originated relevant changes in the organisation and management of the schools in case a, namely with regard to the need to monitor students during the extension periods and between the time when the curricular activities of the school end and the aecs begin, and to the need to provide lunch for the students. this extension of time in the same physical space has fostered conflicts and has originated disciplinary problems. this idea has, in fact, been mentioned by the children, who have often referred to the existence of conflicts in their texts. the fact that there was not enough staff to ensure the supervision of the children, in most schools, also contributed to the situation. in case b, for the aforementioned reasons, there was no need neither to resort to the flexibility of the timetables of the head teachers, nor to extend the period of supervision of international electronic journal of elementary education vol.7, issue 2, 199-216,2015 210 the students, so the school management was able to keep to its routine. the temporal impact was not thus felt in the same way. another sort of impact is the one connected with the changes produced by the integration of “learning support” activities into the head teachers’ set of tasks and timetables. although this activity is part of the aec programme, as the school was responsible for its implementation, the effect was particularly felt in case b: in fact, it originated an increase of the head teachers’ weekly workload and forced them to go to one or more physical spaces outside of the school facilities during the periods when they were not teaching curricular subjects. finally, it should be noted that, in order to achieve the aims that have been set for the programme, the curricular enrichment activities still call for a significant horizontal and vertical articulation with both the 1st and the 2nd cycle curricula. according to the testimonies, the conditions that would enable that articulation have not been met yet, either from the point of view of the formal conditions, or from the subjective point of view of the intervening people. in that sense, the curricula need to be redefined and the teachers’ timetables need to be changed. even the concept of having one teacher responsible for the teaching of all the curricular subjects in the 1st cycle would have to be altered in order to strengthen the co-operation. on the relationship between the students and the school, as well as on the learned knowledge in both cases, those in charge of the pedagogical and administrative managements of the schools, as well as the head teachers, acknowledged the pertinence of the social dimension of the programme and the role that the aecs can potentially play in the development of children. however, at the same time, they identified a number of adverse effects on the students, both with respect to their behaviour and attitudes, and to their relationship with the school and learned knowledge. some of the worries and reservations concerning the aec programme have been based on this idea, namely:  the exhaustion provoked by an excess of school hours, i.e., the hyper scholarisation of children. according to the head teachers, this has negative consequences on the process of teaching and learning, such as student distraction, and loss of interest and demotivation with regard to the curricular learning, which is far more complex and non-ludic;  the lack of interest conveyed by some students for most curricular activities;  the anomie (according to several head teachers, the enrichment activities can favour indiscipline and loss of the sense of “rules”, and have negative effects on the formal curricular activities);  the reduction, almost annulment, of time for individual work and study and the consequent diminishment of academic achievement;  the “pedagogisation of leisure” which significantly reduces the time children have to play and, as such, carries with it the risk of getting tired of school. until the current moment, a systematic evaluation of the effects of the aec programme on the progression of the students’ learning has not yet been carried out. in the absence of that process, the nonexistence of benefits, on the plane of the acquisition of knowledge and development of competencies, underlined by the majority of the ptts, is based on an ‘impressionistic’ evaluation which cannot, and should not, be devaluated, but which calls for a better substantiation. all-day schooling: improving social and educational portuguese policies / martins, vale & mouraz 211 on the relationship between the parents and guardians and the school in both cases, the parents and guardians valued the all-day schooling programme and the free of charge offer of activities and lunch, as well as the prolongation of the period of supervision of the children. the degree of involvement of parents in the all-day schooling and in the implementation of the programme was diverse. in school b, where the offer was locally promoted, parents played a relevant role in the establishment of cooperation protocols with partner institutions. in case a, parents did not play any sort of role in the implementation of the programme. that is perhaps why they kept themselves at a distance from the aec programme, as well as from the school, and why they somehow devaluated the activities and showed no respect neither for the way the aec programme was organised nor for the teachers’ work. it should be noted that this devaluation has been reinforced by the lack of influence on the evaluation of the students. on the other hand, and having the two cases in mind, we can say that the parents were confronted, in the process, with new fields of knowledge and new forms or methodologies of teaching and learning. although this dimension still needs to be thoroughly studied, we admit that the introduction of the aecs has changed the way parents and guardians perceive the educational institution and the people who work there, namely the teaching work of the head teachers. final reflections the unanimous valorisation of the philosophy and objectives of the aec programme – stated in the introduction – on the part of all the involved subjects, and in particular on the part of the parents and guardians, gives clear evidence that this programme was a “good measure” of public educational policy, and was coherent with the nuclear project of “all-day schooling”. the effectiveness of this socio-educational measure depends, however, on a set of factors that have not yet been consolidated, and thus call for a deep reflection. among these, we should mention the weak articulation between 1st cycle teachers (and particularly aec programme teachers) and the technical team responsible for the local implementation and evaluation of the activities. one of the conditioning aspects of that connection lies in the curricular articulation. most of the head teachers, of the aec programme teachers, as well as schools in general, discursively value the curricular articulation (one of its formal competencies) and believe that it should exist, but – as they themselves recognize – it has not been enough to change practices. this dimension is still evidencing low levels of satisfaction and is even one of the most acknowledged weaknesses. another critical factor stems from the overlapping of formal and non-formal education in the same educational space, which gave rise to some misunderstandings with regard to the aims of the activities. if we look at the terminology used by all the subjects involved in this programme (the recurrent use of words such as classes, students, teachers, textbooks, pedagogical material, evaluation…) we will come to the conclusion that for both parents and the children in general, and also for some head teachers and aec programme teachers, the specificity of the aecs has not been identified yet. in spite of that, curiously enough, for a group of children from joão de deus elementary school, the aecs do clearly belong to the field of free time occupation. the absence of a thorough clarification of this issue has, meanwhile, led to the devaluation of the activities, either on the part of the parents and guardians or on the part of the head teachers. one of the central reasons for their criticism lies on the weariness of the students as a result of their remaining in the same physical space (mainly in the classroom). this worry with international electronic journal of elementary education vol.7, issue 2, 199-216,2015 212 the consequences of physical space is present in the studies by schnitter and haselhoff (n.d.) and reh et al. (2011), who underline the need for a diversification and flexibility in terms of physical spaces as a way to foster the socio-educational development of the students. incited by a similar worry, and with the aim to avoid the repetition of the model of the school and to minimise the weariness and/or the annoyance generated by the activities on the children, the parents and their representatives put forward a set of proposals. these, however, have not been as welcome as described in other european experiences (schnitter & haselhoff, n.d.; reh et al., 2011). although they did not put it precisely in the same terms as bernet (1998), underlying the proposals presented by the parents we can clearly see the idea of citizenship education. this is evident in the recommendation to make the most of the educational resources of the parish or of neighbouring parishes, of having the activities held in local studios or workshops, of organising visits to exhibitions, planning to attend concerts or participate in other cultural local events. these suggestions aim no doubt to overcome the “poverty” of the children’s cultural world. according to den besten (2010), this should in fact be a measure of public policy and one of the aims of the aecs, especially in culturally disfavoured and resource-poor families. another nonconsolidated factor relates to the “time and its use”, as described by roldão (2008). according to this researcher, the aecs cannot originate an unbearable and inadmissible pressure on children in case the municipalities and the children decide to invest in many areas, thus taking away the effective free time the children have. the promoters, be they the town councils, the parish councils or the groups of schools, have the possibility to put into practice curricular differentiation through the selection of part of the offer that best suits the local needs and meets the problems felt by the local community. but instability was also caused by the fact that although the offer of activities was compulsory, the children’s attendance was not obligatory – especially with regard to the teaching of english. this incongruence potentiates inequalities in the process of learning this foreign language and has negative consequences in the transition to the next cycle (madureira, 2011). the solution to this problem lies in the decision to integrate – or not – this activity into the curriculum of the school. finally, the success of the aec programme is compromised by the absence of an ample and rigorous system of evaluation, which allows for the identification, in a sustainable way, of the measures needed for its improvement. the directions towards which the aec programme seem to evolve oscillate among the social dimension of the programme, the concern with the curricular articulation and the curricular differentiation. the tendency to exaggerate one of these dimensions is a risk associated with some of the interveners. it is fundamental to balance these dimensions. the uncertainty with regard to the funding of the programme is no doubt another menace to this balance. the aec programme was created with the stated aim of promoting the improvement of the quality of the schools of the 1st cycle and consequently to contribute to the progress of the students in terms of learning, as well as to provide a social response in the field of the support to families, namely in the harmonisation between the professional and the family life. if it is true that the latter aim has been reasonably met, further improvements still have to be implemented in order to meet the former. all-day schooling: improving social and educational portuguese policies / martins, vale & mouraz 213 in a broad perspective, as aec programme could be a part of educational policies conducting to equity, it could be an assessment of welfare state’s measures. educational equity for all is essential in a changing view of welfare states from a social protection to a social promotion framework. this is relevant at the present as europe is facing a financial crisis which puts this equity at risk as investment in education has been regarded as expenditure. • • • acknowledgements this work was funded (in part) by national funds through the fct – fundação para a ciência e a tecnologia (portuguese foundation for science and technology) within the strategic project of ciie, with the ref. “pest-oe/ced/ui0167/2014”. jorge martins is a phd researcher and collaborative member of the ciie and is a member of the educational cities’ observatory. he has been team member of research projects, concerning educational government and administration, that is the main focus of his research interests. ana vale is a phd researcher and collaborative member of the ciie and is member of educational cities’ observatory. she has been team member of research projects, such as educational dynamic’s cartography facing inequalities in the city of porto. her research interests concern educational inequalities and social and educational dynamics. ana mouraz is a phd researcher and effective member of the ciie research subunit training, knowledge and work and educational contexts and is a member of the board of schools life observatory. she has been team member of research projects, such as ‘peer to peer – peer observation at higher education’ and ‘argumentation is needed!’ she did publish several scientific papers in international peer reviewed journals. references armstrong, t., & armstrong, g. (2004). the organizational, community and programmatic characteristics that predict the effective implementation of after-school programs. journal of school violence, 3, 93-109. bernet, j. t. (1998). la educación fuera de la escuela. ámbitos no formales y educación social. barcelona: ariel. bernet, j. t. (1999). a educación non formal e a cidade educadora: dúas perspectivas. revista galega do ensino, 24, 199-221. bolívar, a. (2012). justicia social y equidad escolar: una revisión actual. revista internacional de educación para la justicia socia, 1, 9-45. correia, j. a. & matos, m. (2001). da crise da escola ao escolocentrismo. in s. r. stoer, l. cortesão & j. a. correia (orgs.), transnacionalização da educação: da crise da educação à educação da crise (pp. 91-117). porto: afrontamento. demeuse, m., frandji, d., greger, d., & rochex, j.y. (2008). l’éducation prioritaire en europe: conceptions, mises en œuvre, débats. paris: inrp. den besten, o. (2010). visualising social divisions in berlin: children's after-school activities in two contrasted city neighbourhoods, forum: qualitative social research, 11. [online] available: http://www.qualitative-research.net/ index.php/ fqs/ article/ download/ 1488/ 3010 (3/15/2013) international electronic journal of elementary education vol.7, issue 2, 199-216,2015 214 dobert, h., echard, k., & sroka, w. (2004). conditions of school performance in seven countries. múnster: waxmann. forquin, j. c. (1993). escola e cultura: as bases sociais e epistemológicas do conhecimento escolar. porto alegre: artes médicas. geppert, c. b., hofmann, s., & hopmann, t. (2012). policy reform efforts and equal opportunity: an evidence-based link? an analysis of current sector reforms in the austrian school system. center for educational policy studies journal, 2, 9-30. leite, c., & pacheco, j. a. (2010). para uma clarificação de conceitos que atravessam ‘a prestação do serviço educativo’, do processo de avaliação externa de escolas. [online] available: http://sigarra.up.pt/ fpceup/ pt/ publs_pesquisa.show_publ_file? pct _gdoc_id=37835 (11/2/2012) leite, c. (2005). a territorialização das políticas e das práticas educativas. in c. leite (org.), mudanças curriculares em portugal: transição para o século xxi (pp. 15-32). porto: porto editora. leite, c. (2006). políticas de currículo em portugal e (im) possibilidades da escola se assumir como uma instituição curricularmente inteligente. currículo sem fronteiras, 6, 67-81. lipovetsky, g. (1989). a era do vazio. lisboa: relógio d'água. madureira, c., rando, b., alexandre, h., & martins, m. (2011). cadernos ina n.º 48: avaliação das actividades de enriquecimento curricular nas escolas do 1º ciclo do ensino básico. estudo de caso nas áreas metropolitanas de lisboa e porto. lisboa: ina editora. magalhães, a. m., & stoer, s. r. (2002). a nova classe média e a reconfiguração do mandato endereçado ao sistema educativo. educação, sociedade & culturas, 18, 25-40. muñoz, j. m. e. (2005). valores institucionales de la escuela pública: ideales que hay que precisar y políticas a realizar. in j.m.e. munoz et al. (eds.), sistema educativo y democracia: alternativas para un sistema escolar democrático (pp. 9-40). barcelona: ediciones octaedro. oecd (2007). no more failures: ten steps to equity in education. paris: oecd publishing. ordinance no. 16 795/2005, 3 august. diário da república: ii série, no. 148. [online] available: http://dre.pt/pdf2sdip/2005/08/148000000/1110111102.pdf (3/15/2013) pacheco, j. a. (1996). currículo: teoria e práxis. porto: porto editora. pfeifer, m., & holtappels, h. g. (2008). improving learning in all-day schools: results of a new teaching time model. european educational research journal, 7, 232-242. pires, c. (2007). a construção de sentidos em politica educativa: o caso da escola a tempo inteiro. sísifo: revista de ciências da educação, 4, 77‑86. põder, k., kerem, k., & lauri, t. (2013). efficiency and equity within european education systems and school choice policy: bridging qualitative and quantitative approches. journal of school choice, 7, 1-36. reh, s., rabenstein, k., & fritzsche, b. (2011). learning spaces without boundaries? territories, power and how schools regulate learning. social & cultural geography, 12, 8398. roldão, m. c. (2008). prefácio. in m. oliveira, r. coelho, r. matos & s. milhano (eds.), actividades de enriquecimento curricular: relatório sobre a sua implementação no 1º ciclo do ensino básico do concelho das caldas da rainha (pp.1-11). leiria: centro de investigação identidades e diversidades e folheto edições e design. schnitter, m., & häselhoff, k.-s. (n.d.). all-day schooling in düsseldorf, germany: education, economy and society. [online] available: http://www.oecd.org/ edu/educationeconomyandsociety/all-dayschoolingindusseldorfgermany.htm (1/21/2013) stake, r. e. (2005). investigación con estudio de casos. madrid: ediciones morata. steiner-khamsi, g. (2012). understanding policy borrowing and lending building comparative policy studies. in g. steiner-khamsi & f. waldow (eds.), world yearbook of education 2012: policy borrowing and lending in education (pp. 3-17).london and new york: routledge. all-day schooling: improving social and educational portuguese policies / martins, vale & mouraz 215 triviños, a. n. (1987). introdução à pesquisa em ciências sociais: a pesquisa qualitativa em educação. são paulo: atlas. yin, r. k. (2002). estudo de caso: planeamento e métodos. porto alegre: artmed. notes note 1. aec – in portuguese, atividades de enriquecimento curricular. note 2. in portuguese, “programa escola a tempo inteiro”. international electronic journal of elementary education vol.7, issue 2, 199-216,2015 216 www.iejee.com this page is intentionally left blank http://www.iejee.com/ international electronic journal of elementary education, 2015, 8(1), 161-174 issn:1307-9298 copyright © iejee www.iejee.com the right to be included: homeschoolers combat the structural discrimination embodied in their lawful protection in the czech republic irena kašparová  masaryk university, brno, czech republic received: june, 2015 / revised: september, 2015 / accepted: september, 2015 abstract there is a 240-year tradition of compulsory school attendance in the czech republic. to many, compulsory school attendance is synonymous with the right to be educated. after the collapse of communism in 1989, along with the democratization of the government, the education system was slowly opened to alternatives, including the right to educate children at home, expressed in act no. 561/2004. this inclusive law has had exclusionary consequences for many families who wish to choose this mode of education. the situation reveals a clear struggle over various forms of capital in the field of education, as famously described by bourdieu (1998). the article, based on a longitudinal ethnographic study of homeschooling families, maps the structural discriminative dimension of the law and displays the strategies that the actors have adopted in order to combat them. keywords: homeschooling, structural discrimination, education, difference. introduction over the past two hundred years, nation states and their societies have become increasingly technocratic, secular, and meritocratic. one powerful tool enabling this shift is a centralized, compulsory, state-run education (lancy, 2010). in europe, education is now a fully institutionalized process. professional teachers and state-run schools are the children’s chief educators. neither of them is questioned by the majority population. as classics of identity theory have pointed out, although the process of schooling is presented as natural, it is always marked by ideological conflicts and power struggles characteristic of the time in which the schooling occurs (anderson, 1983; bourdieu & passeron, 1990; berger & luckmann, 2001).  irena kašparová, department of sociology, faculty of social studies, masaryk university joštova 10, 602 00 brno, czech republic, e-mail:kasparov@fss.muni.cz http://www.iejee.com/ international electronic journal of elementary education vol.8, issue 1, 161-174, 2015 162 modes of compulsory school education and the necessity for reforms have been the subjects of ongoing debates among parents, government officials, and school representatives for quite a while now throughout europe, including in the czech republic. alternatives to state schooling have an inconsistent tradition in the czech republic. during the communist era (1949-1989), church and private schools were closed and the public sector was sentenced to meaningless activities promoting life in a socialist country, while being closely watched by the state officials. after the velvet revolution in 1989, the situation changed. new modes of schooling were introduced, including renewed private and church schools and new styles of learning, including montessori, waldorf, dalton, and step-by-step educational methods. most current school reforms concern the content of learning and pedagogical methods without touching upon the very idea of centralized, universal, state-run education and specialized professionals as educators. however, an increasing number of parents, without questioning the right to and the need for education, strive for a different mode of transmitting knowledge and information outside the presence and intervention of a centralized system. they wish to homeschool their own children. the paper describes the quest for this option. the political vision of inclusive education in the czech republic allows for only a one-sided approach: all children, regardless of their abilities or wishes, ought to be integrated into school, the only place where education can happen (bartoňová & vítková, 2013). homeschooling in praxis questions this right, which over time turned into a requirement, bringing to light subtle forms of discrimination, disguised behind the label of inclusion, that create obstacles on the path to education. since 2005, education act no. 561/2004 § 40-41 coll. (zákon o předškolním, základním, středním, vyšším odborném a jiném vzdělávání) allows for a child to be homeschooled in the czech republic at the primary school level. seemingly inclusive, discriminatory devices are nevertheless built into the conditions that homeschooling families must fulfill in order to be able to homeschool. my main interest here is to map, describe, and analyze their struggle for a chance to homeschool a child who has no special needs or disabilities and does not otherwise fall into a category of special conditions. i propose that homeschooling ought to be seen not as a reaction to growing dissatisfaction with the present mainstream, state-run school system in the czech republic, but rather as a natural enactment of an alternative worldview or ethos, built upon different ideas about learning, teaching, education, and schooling. as such, homeschool challenges the inclusivity of the school system in the same way as any other cultural, social, or religious tradition. my particular interest is in the subjective evaluation of learning in childhood by homeschool families and in their current options for homeschooling within the legal framework of the czech republic. the process of developing the legal status of homeschooling, from a dismissed alternative to a possible legal right, provides an inseparable background to this debate. i also aim to comment on the relationships among the various forms of capital (bourdieu, 1998) owned and manipulated by various families and their decisions to opt for homeschooling. notes on methodology the data introduced below represent part of my longitudinal ethnographic research into alternative teaching and learning methods of primary and secondary school-age children, as well as the life strategies of their families, all situated in the czech republic. for two years, i interviewed families (parents and children) who opted for homeschooling as an alternative to mainstream state-run education. at the same time, i conducted interviews the right to be included: homeschoolers combat the structural discrimination / kašparová 163 with teachers and directors of the index schools1 that enroll homeschooling children and act as examining boards in their compulsory periodic examination. i also regularly participated in the daily homeschooling routines of six families, observing the learning processes of the children and the teaching methods of the parents, spending time with them and attending their extra-curricular activities. in conjunction with these observations and interviews, i was a member of two of the most frequented online homeschooling forums (svobodauceni.cz; domaciskola.cz). i actively participated in selfeducating activities for homeschooling families (conferences, workshops, and informal gatherings) and included written publications in my analysis. in keeping with the idea of ethnographic research, i pooled data from several sources in order to find as much as possible about a narrow sector of human interaction. although separate, all of these data-gathering methods are interconnected. most of the interviews with homeschooling families were conducted in their homes, where i was also able observe the learning environment and family interactions. as the homeschooling families lived scattered across the whole country, i often had to stay several days in order to conduct the interviews with all of the family members. that extended and prolonged my opportunities for observation. the information gathered through the interviews could then be cross-checked with the data gathered through observation in the families, thus capturing the dynamics of learning and teaching, and of social and family interactions, all at the same time. on several occasions, i was able to follow these families to their index schools for mid-term or final examinations, thus gaining yet another opportunity to add to the data gathered in their homes. when possible, i interviewed all the members of a homeschooling unit: the children, the parents, and the other educators involved. the written data used for my analysis here came mainly from online debates and publications either by homeschooling parents themselves (through blogging or online magazines on the topic) or by teachers/directors of the index schools. the written data provided a wider context that served as a field of reference for my own findings. it also represents a conscious image or self-presentation of the homeschoolers to the rest of the society. it is the shop window they use to inform, attract, and draw in others. data analysis is not the final chapter of the research, but a process that intersects with the whole research period, including data gathering (hammersley & atkinson, 1995). in the course of research, a scientist makes several decisions that consequently influence the future analysis, such as whom to interview, what to ask, and what to amend. such choices are an internal part of the interpretation of studied phenomenon (ezzy, 2002). the same was true for my research. interpretations that are part of this study were focused around the key question, “what does education mean to homeschooling families and how do they implement this meaning in practice in a society that does not share their views?” in order to answer this question, i concentrated on the meanings and interpretations given by the homeschoolers themselves. their emic interpretations are set against the wider sociocultural context, to mirror the generally accepted patterns of education cosmology represented by mainstream schooling. judging from the rising number of readers of online homeschooling forums, as well as the increasing number of homeschoolers nationwide, it is fair to estimate that similar cognitive, cultural, and social processes are taking place in other families across the country. 1 an index school is any state-registered school where a child is enrolled for compulsory education and which takes responsibility for that child’s periodic examination. success in passing these examinations is a prerequisite for the right to remain in the homeschooling regime. for a child at the primary education level (6-11 years), any state-registered school in the country may act as an index school. for children in the lower secondary education level (12-16), only a few schools in the country, chosen by the ministry of education, may fulfil this role. international electronic journal of elementary education vol.8, issue 1, 161-174, 2015 164 childhood learning theories and homeschooling in the czech republic from a cultural perspective, the issues of learning in childhood have been part of sociocultural anthropology since its early days (malinowski, 1929; mead, 1928, 1930). attention has been paid to culture-dependent presumptions and generalizations about psychological determinants of social interaction between adults and children. studies have demonstrated the diversity of the ways in children are brought up and taught around the world, as well as the sense they make of their experiences (lancy, 2010). ethnographies from around the world point to a model of village learning in which a child receives a guided acquisition of cultural practices. in such a setting, children learn on their own, at their own speed, anything the children presume to be important for their lives under the instruction of any adult around. this model serves pre-industrialized societies well, but to what extent is it sustainable in urban communities? formal compulsory schooling brought major changes to the real lives of people and the way that learning and education are perceived was dramatically altered. anthropological literature has provided a theoretically informed appreciation of innumerable culture-dependent adaptive paths of childhood learning and has shown how difficult, if not impossible, it is to transfer a village-learning model into a school curriculum (lancy, bock & gaskins, 2010). lancy asserted (2010) that what works well for transmitting culture does not necessarily work for transmitting school knowledge. while the village-learning model requires a child’s independence and non-interference in learning, formal schooling expects the opposite: uniformity in daily preparation for classes and standardized systematic curricula-shaped learning and knowledge, certified by uniform testing. every culture applies two main methods in order to shape people into desired forms: socialization and education (cohen, 2000; rival, 2000). socialization is carried out by kin and encompasses personal feelings. it labels the behaviours through which the basic cognitive models of society are learned. these models allow us to navigate through daily events and interactions with others. through socialization, children acquire what lancy (2008) calls sense, which is a prerequisite for successful acceptance by others, as well as for successful learning processes. education, in contrast, is about acquiring standardized knowledge, abilities, and values. it takes place in a standardized and prescribed way, administered by a person without any affection necessary (cohen, 2000). learning factual knowledge is only a small part of the goal of education. the processes of thinking and the structure of relations that are taught to the students are much more important (williamson, 1979). several authors noted the loss of illusions that schooling brought to rural communities (bledsoe, 2000; ogbu, 2000; rival, 2000; reed-danahay, 2000). the status quo of current schooling does not seem to work also for the culture of its origin, the urban industrial society (stahl, 2015). a kind of combination of the compulsory schooling model and the village-learning model has been sought (holt, 1997; ricci, 2012; kašparová, 2014). within the czech scientific and pedagogical contexts, homeschooling is a side issue, explored only by a few (kostelecká, 2003, 2005; kašparová, 2012; 2014; 2015). the law pronounces compulsory school attendance for children, rather than any compulsory form of education. education in the czech context has historically been strongly connected to schools and other educational institutions (kostelecká, 2005, 2010). rooted in the socialist era, learning at home has been traditionally associated, in the broader czech view as well as in scientific discourse, with various forms of handicaps or differences that cannot be integrated into the mainstream school system (bartoňová & vítková, 2013). the right to be included: homeschoolers combat the structural discrimination / kašparová 165 sociologist of education karen chapman (1986) identified two basic approaches to education: functionalist and conflictualist. functionalists believe the goal of the education system to be to select individuals according to their abilities and to allocate appropriate posts and positions to them. conflictualists see education as a battleground for several interest groups (economic, social, political, and other), where the winners formulate the curricula that is later taught to all, until another victorious group takes over. in other words, the winners decide what, when, how, and why everyone else is to learn and which cultural capital is vital for succeeding in the education system (bourdieu, 1998; bourdieu & passeron, 1990). among czech homeschoolers, the conflictualist line of understanding prevails. homeschoolers perceive themselves as promoters of a different cultural capital in the terms bourdieu (1998) so famously described. they clearly recognize the political and philosophical dimensions of the issue and see their activities as a challenge to the inclusive system of schooling: “of course i believe each child needs an education but i do not believe the current school system offers the best option. it is too uniform; it does not respect differences in children and because of that it manages to kill the hunger for education in many children before it has ripened and borne fruit. every child learns at their own speed with a different method. only such methods make their desire for knowledge sustainable or even stronger. i believe i can provide such an environment for them at home. i want my children to be educated this way. i don’t think just because the state decides something, that it must be the right decision. look at our communist past, it is the best example of how things can go wrong” (helena,2 commenting on reasons to homeschool). helena described the inequalities and differences, and the causal mechanisms which allow for them, which she believes exist between homeschooling and state-run schools. she used generalized and over-simplified expressions describing two opposing fields. homeschool and state education were presented as two mutually exclusive homogeneous systems. this was a frequent approach and argument that homeschoolers used in their initial self-explanation. since homeschooling is unusual in the czech republic, homeschoolers are challenged rather frequently, and they have to constantly defend themselves against a differently-minded majority. such superficial declarations often suffice for this defense. should the inquirer prove to be more deeply interested in the reasons for homeschool, the very core of the argument that recognizes education as a means of control (williamson, 1979) can be seen in this statement by daniela. “think of the logic. we are living in a free country, you can have as many children as possible and nobody asks you if you will have money to support them. you can make major decisions about their bodies – for example you can decide about their vaccinations, reject them altogether or on the contrary buy many more3. but once you wish to influence their minds through the stuff they are learning, suddenly there is a problem. why should you challenge the state authority on a universal curricula? who are you to decide what is best for 2 unless otherwise stated, all the data presented here comes from my own research, described in the notes on methodology section. names and places have been altered to ensure anonymity. 3 the respondent is referring to a dispute between a group of parents and the czech state about the possibility of abstaining from the compulsory child vaccination program. in this legal case, a group of parents raised a concern about human rights violations, in which individuals are forced by law to be vaccinated against their will. in the meantime, these parents did not have their children vaccinated. this resulted in the fact that these children were not admitted to state-run school institutions. the issue was debated in the media and occasionally the parents were fined. the case was finally resolved at the beginning of 2015 by the constitutional court of law, which confirmed the earlier decisions of the lower courts and dismissed the possibility of a human rights violation. however, this result was not known to the respondent when the data was collected. international electronic journal of elementary education vol.8, issue 1, 161-174, 2015 166 your child?” (daniela, commenting on reasons to homeschool). knowledge becomes an objective quality, verified and certified by the state. its uniformity supports the authority of those in charge, while divergence brings about danger, which may take the form of an ideological or economic revolution (illich, 1971). homeschooling constitutes an alternative platform,4 where education, in the general understanding of the word, is being tested. it represents much of bourdieu’s (1998) reconciliation of the objective (the field) and the subjective (habitus), running along the conflictualist paradigm mentioned above (chapman, 1986). the habitus is here represented by the homeschoolers, who mirror the objective field represented by mainstream state-run education. mutual negotiations between the two display the power struggle and control over the various forms of capital (bourdieu, 1998). like any other urban society, the czech state provides a ready-made framework for combining family and working life. this model includes working parents and state-run care for children. under this model, in general terms, the mother stays at home with a child during the whole period of her maternity and parental leave, lasting from the birth of the child up to three years of age. at the age of three, the child is placed in full-time day care, and the mother returns full time to her former profession, while continuing to perform her caring and homemaking activities. thus, the mother ends up having two jobs, or two shifts at least. the father continues uninterruptedly with his career. the model often results in lasting inner stress and personal unhappiness on the part of those women who have opted for it – especially when the children are of young school age and a substantial proportion of care is still necessary (kašparová, 2012). homeschooling families do not accept this model: “we do not understand the logic of family policy at all. you are encouraged to care for your children for three years during the parental leave day and night, without any help or relief from the state, and then, overnight, just as the children are getting some sense, you are expected to place them in full-time care and go another way, working nine to five, seeing them awake for two hours a day and weekends, placing their upbringing into the hands of strangers. they do not see their parents, they do not see their siblings all day long. this is not a healthy model. this is not what we wanted,” (katka, commenting on reasons for homeschooling). katka was commenting on the normative actions of the state, which definitively assigns the role of the parent with its policy incentive: no intrusion or state support up to the age of three, voluntary cooperation via kindergarten activities from the ages of three to six, and obligatory care from the age of six onwards. the different primary socialization of homeschooling children, with all its consequences, tests the validity, rigidity, and flexibility of the educational system in the czech republic. bourdieu’s concepts of habitus and cultural capital are closely connected with the process of homeschooling, since “cultural capital that is effectively transmitted within the family itself depends not only on the quantity of cultural capital, itself accumulated by spending time, that the domestic group possess, but also on the usable time (particularly in the form of the mother’s free time) available to it.” (bourdieu, 1986: 253). bourdieu’s conceptual framework is placed at the center of attention here, since it illustrates in detail how the operationalization of social differentiation in schooling is tied to individual people’s activities. 4 the paradox remains that although internally fairly different, homeschoolers are perceived by the majority as a homogeneous community. social networks and on-line images contribute a great deal to such understanding. the right to be included: homeschoolers combat the structural discrimination / kašparová 167 structural discrimination of homeschoolers: five occasions a growing parental lobby calling for the accommodation of alternative approaches to education resulted in a revised political attitude, and act no. 561/2004 § 40-41 was passed in 2005. this law ensures that each child at the primary school level has the option to be educated at home, providing several conditions are satisfied. these conditions include: 1) the educator must hold at least a high school diploma, 2) the homeschooling family must find a school willing to act as a guarantor and examiner for regular compulsory testing of the child, 3) the homeschooling family must provide a letter of explanation as to why homeschooling should occur, 4) the homeschooling family must provide a written declaration that it has sufficient material means to educate the child, and 5) the homeschooling family must provide a letter of opinion from a pedagogicalpsychological advisory bureau. although everybody has the right to apply, there is no lawful demand for successful approval, since the option to homeschool is conditioned by several subjective requirements, as illustrated below. the requirement concerning the qualification of educators brings two issues into the foreground. the first issue operates on philosophical grounds, questioning the need for this condition in the first place: “in primary school, children learn reading, writing, and basic math. in fact, this is the stuff and the level most of the population uses all their life, regardless of their future profession. unless you are an accountant or a scientist, you do not need much of the lower secondary school level math in your life. sometimes a person without high school is a much better teacher than a university graduate. a degree does not say anything about the ability to be a good teacher. i see this condition as completely unnecessary,” (headmaster of kletná school, commenting on homeschooling conditions). this issue reflects the philosophical nature of the problem. it is created by the use of symbolic power: the educated are the gatekeepers of their trade. in order to remain so, they have to maintain their difference, certified by formal education. another dimension is opened by the necessity of having a high school diploma: the structural discrimination intrinsically built into the requirement. the social and political history of the czech republic includes periods of open discrimination against selected groups of inhabitants (e.g. the educated proletariat, the bourgeoisie, and kulaks), preventing them from participating in some aspects of life, including education. as late as the 1970s, it was a common practice of the communist regime to influence educational strategies and restrict the educational options of certain families. if there were no workers in a family, the options for the children’s further education were diminished due to their “unsuitable origin”. if one child chose to study a subject considered undesirable by the regime, such as philosophy or especially theology, that child’s siblings had no choice but to enroll into an apprenticeship, learn a trade, or work in agriculture – i.e. to usefully serve the regime5. likewise, members of national minorities (especially roma) whose children were placed in special schools were, by the nature of these schools, prohibited from continuing in further education and were thus destined to perform unqualified manual work or to learn a trade without a high school diploma. members of these groups thus had no real chance for success in formal education during the communist era, which may yet have direct consequences in their current lives. “julie is taught by my mother, her grandmother. she is retired now and they spend a great deal of time together, most of it outside. my mother had many professions, she worked most of her life with children, in kindergarten, in afterschool activities, too. but after the war, she was not able to study, her family was of bourgeois 5 personal family history of the author. for a further description of the period, see: kárník, z. 2004. bolševismus, komunismus a radikální socialismus v československu. praha: ústav pro soudobé dějiny av čr, 2004 international electronic journal of elementary education vol.8, issue 1, 161-174, 2015 168 origin, so the communists did not allow her to go to school, so she had to train to be a tailor. she does not have a high school diploma. so officially on paper, i am julie’s teacher. in practice, it is my mother,” (jana, commenting on homeschooling conditions). the case of julie and her family shows a maneuvering strategy around the borders of the law in question. having lifelong experience with teaching children is insufficient if it is not supported by a legal document; this forces the family to bend the law and expose themselves to consequent punishment. to fulfil the second condition of the law in order to homeschool, the family has to find a school willing to enroll a homeschooler and to provide for regular testing. although the law says the parent can choose any primary school, indicating that any primary school ought to consider accepting a homeschooler, the conditions and rules of co-operation between the homeschooling family and the school are within the competence of the school to decide. the research shows the schools differ greatly, from support and acceptance to obstacles and difficulties: “we were the first homeschooling family in the former school. it was all new to the director, who was afraid of it all. she was very pedantic about all the paperwork that accompanied the change and did not pay attention to anything that i was saying to her. she ignored the fact that we decided to do a project-centered education and thus could not cover all the topics they did at school, while the ones we did cover we would do in much more depth. at the end of the school year, she was examining danny in front of the whole class on all the stuff they covered in the whole last semester. as if it were a punishment for his having this option to be different. it was very stressful for both of us: for him, because he thought he was stupid for not remembering the difference between a rabbit and a hare after birth, which was one of the 60 questions from biology, and for me, because i could see the whole time and the whole way through the examination how very unfair and pointless such testing was. so we searched for a different school where they would be more understanding of our methods. and we found one. the director is trusting; they are interested in our project work and make danny talk about it. in fact they are really evaluating his work, rather than testing silly encyclopedic knowledge as they were doing at the former school,” (irena, commenting upon their choice of index school). the attitude of the director is a key factor in homeschooling application success. the director is the gatekeeper who, in the end, makes the sole decision about this possibility. at the same time, each family, if they are rejected by one school, are free to turn elsewhere. this in praxis leads to the migration of homeschooling families towards welcoming directors and their schools. for this reason, specialized homeschooling schools are forming within czech educational institutions. this is a paradox, since the intention of act no. 561/2004 § 40-41 was to enable the opportunity of inclusion for homeschooling children in any primary school in the country, rather than setting them aside into specialized institutions. the third condition, the provision of a written explanation of why homeschooling should occur, reveals the puzzle of act no. 561/2004 § 40-41 in all its complexity. in order to apply, parents must state their motives for homeschooling. in order to be successful, motives have to be approved by the index school director. although parents have the right to apply for homeschooling, there is no legal right to be granted it in the czech republic. depending on the institution and the belief of its director, fulfilling this condition may require a high standard in literacy, debate, or even legal training: “we wrote this motivation letter two or three times. the director kept returning it to me with notes, like i was her student or something. i was getting desperate,” (vlasta, commenting upon the conditions for homeschooling). returning the homeschooling topic once again to the philosophical level, there is no such thing as the equity of an alternative to the state-run education and schooling. each case is evaluated individually; the authority to decide is the right to be included: homeschoolers combat the structural discrimination / kašparová 169 placed once again in the hands of the state, personified by the director of the school. in practice, therefore, homeschoolers are indirectly motivated to choose a homeschoolfriendly index school, thus deepening the exclusion of homeschooling from the mainstream system. “we have several schools in the neighborhood, but my friend who also homeschools recommended a school about 25 km away, in rodnov. we know there are several homeschooling families there and the director does not make a big deal out of it. here in the local school we would be the first family to homeschool,” (iveta, commenting upon the choice of school). this selection takes away the burden of being a homeschooling pioneer in the local school, but it raises the cost of homeschooling due to necessary commuting. most importantly, it discourages homeschoolers from forging ties with local communities, which traditionally center around schools. rather, they seek alternatives and support via various other interest groups and/or social networks. the fourth condition deals with financial matters. parents have to declare in writing that they can provide adequate space and material conditions for the education of their child. this does not entitle them to financial support from the state, reallocated from taxpayer money, for the education of their children. the allocated amount goes to the index school where the child is enrolled and tested. administratively, a homeschooler brings the same amount of money to the school as any other pupil. enrolling homeschooling pupils can thus be a survival or developmental strategy for some schools, especially geographically isolated schools, where there are few other pupils.6 it is a big financial decision for a family to homeschool. a czech school traditionally covers most of the equipment necessary for learning (textbooks, microscopes, computers, software, gymnastic equipment, etc.) from taxpayer money. in addition to material equipment, schools offer other benefits, such as subsidized meals and extra-curricular activities. the law does not specify how many of these advantages the index school must offer to homeschoolers. although the schools usually offer textbooks and consultations, the availability of other material and services vary greatly. due to the geographic distance that most homeschoolers have from their index schools, subsidized meals and after-school activities are not realistic options, and even borrowing other equipment is problematic. the system does not offer any means of compensation, and thus the financial burden of education and schooling falls almost entirely on the homeschooling family. since most homeschooling families, in order to spend time with the children, often have only a single full-time (or two part-time) breadwinner(s), the decision to homeschool naturally hinders lower-income and single-parent families. “i know we can afford to homeschool only because of my husband’s job and also because we have opted for voluntary modesty in our lifestyle. we buy most stuff second hand, we try to grow a lot of our food, so we have to buy cheap, low-quality food as little as possible. after-school activities are very costly and since we have four children, we had to limit it to two per child. we have no more money to pay for it,” (jarka on managing their homeschool financially). for all the families that managed to jump through the hoops of homeschool enrollment so far, the final condition is usually the most difficult and feared. they have to provide a document presenting the opinion of a state-run institution called the pedagogicalpsychological advisory bureau. this institution has several competencies. among others, it deals with children who are different in some way, be it in their iq, abilities, or behavior. the difference is determined by various measures and tests, labelling the child accordingly. both the parents and the state can order an evaluation of a child, should they feel the need. usually only children with learning or behavioral difficulties are required by 6 one example of this phenomenon is zš březová – see http://www.zsbrezova.eu/index.php/onas/napsali-o-nas/474-skola-pro-cely-svet, /accessed 15.9.2015/ http://www.zsbrezova.eu/index.php/o-nas/napsali-o-nas/474-skola-pro-cely-svet http://www.zsbrezova.eu/index.php/o-nas/napsali-o-nas/474-skola-pro-cely-svet international electronic journal of elementary education vol.8, issue 1, 161-174, 2015 170 the order of the state (represented by a school director) to undergo tests here. on the basis of these tests, children with special needs are then integrated into the school system in a variety of ways, ranging from full classroom integration with various on-the-spot adjustments, e.g. longer time for written tests, to seclusion at home, e.g. due to a long term illness (mcdonald & lopes, 2014; bartoňová & vítková, 2013). “we came there [to the pedagogical-psychological advisory bureau] and the lady in charge told us straightaway that she would not write the letter of recommendation for us, because she believes children at home are not socialized properly, that we were going to make his life miserable without friends and why would we want to do it, that he is healthy, without any handicap, so why would we not let him go with others. perhaps she would not be so difficult about this if he had learning difficulties or some kind of long-term illness. i thought we would go somewhere else, but then she started to ask us questions, like why we wanted to do it, why we didn’t like our former school, and so on. it was rather a long talk. she talked only to us, not to our son. it was like a political debate on tv. but she did not write a letter for us in the end. so we went to a different bureau, where mr. alvin was in charge. i knew about him from the social forums chats – he supports homeschooling – he sent us a letter via e-mail, we did not even have to go to see him,”(mira, on her bureau visit). a similar reaction to the bureau officials was given by a different respondent: “the director told us: you have a healthy child? then homeschooling is out of the question. i do not believe this exclusion is good for anybody,” (daniela, on her bureau visit). this echoes the formal political regime, where the idea of inclusion in the education system was totally dependent on the health of the child. while healthy children were prohibited from being educated anywhere but in a state institution, the home education of children with different needs was tolerated, since they were considered second-class citizens without much potential use for the regime.7 as a result of past attitudes, homeschooling continues to be seen by many officials and administrators as a symbol of discrimination – an undesirable practice that ought to be erased, using the means and tools given to them by the state. at the philosophical level, the homeschooling parents thus face their toughest opponents here, with a body whose job it is and has historically been to both define and diagnose differences. the paradox remains that although the law does not list the favorable opinion of the bureau among the necessary conditions for an application’s approval, in practice, the school directors require it, citing it in order to distribute the burden of deciding. failure to provide a positive opinion may thus serve as a justification for homeschool rejection. parents trying to formalize homeschool are thus challenged in five steps by discriminatory measures. overcoming these challenges requires a high degree of social, cultural, and economic capital, substantially limiting those interested in this model of learning. what started off as a project of connection between the immediate human and natural environment is transformed via obscure state measures into an exclusive scheme for the educated, resourceful, and wealthy. the conditions open a space where the right of parents to educate their children at home is severely hindered by institutional discrimination. 7 as an example of the approach of the communist ideology to handicapped children, see “children with hearing disabilities are deprived of impulses that develop speech …or other higher principles, including passion for work and the love of a collective of co-workers and socialistic community” (translation of the author). in další rozvoj československé výchovně vzdělávací soustavy, svazek 2, czech socialist republic (czechoslovakia). ministerstvo školství, slovak socialist republic (czechoslovakia). ministerstvo školstva spn, 1970 , str. 59 https://www.google.cz/search?hl=cs&tbo=p&tbm=bks&q=inauthor:%22czech+socialist+republic+(czechoslovakia).+ministerstvo+%c5%a1kolstv%c3%ad%22 https://www.google.cz/search?hl=cs&tbo=p&tbm=bks&q=inauthor:%22slovak+socialist+republic+(czechoslovakia).+ministerstvo+%c5%a1kolstva%22 https://www.google.cz/search?hl=cs&tbo=p&tbm=bks&q=inauthor:%22slovak+socialist+republic+(czechoslovakia).+ministerstvo+%c5%a1kolstva%22 the right to be included: homeschoolers combat the structural discrimination / kašparová 171 the law itself does not guarantee the right to homeschool. it is not enough to simply provide all the documents required by the law (such as a written application, a description of material conditions, a list of textbooks to be used, and certificates of qualification). in order for the application to be successful, all of these materials have to be approved by the director of the index school. the subjective decision of the director is hidden behind seemingly objective documents and materials, provided (or not) by the parents. since not all directors are open to the possibility of homeschooling, this rules out the possibility of enrolling the child into homeschooling in any czech primary school. rather, schools that specialize in homeschooling seem to be forming in many regions, and homeschooling families cluster around them. the disapproving attitude of the state towards homeschooling is fully revealed at the lower secondary school level (ages 12 to 16), the second and last part of compulsory education in the czech republic. it is traditionally perceived by all the stakeholders (teachers, children, parents, directors, and government officials) as the most problematic period of compulsory learning. teachers report a tremendous loss of interest on the part of the pupils, children report increasing boredom at school and a loss of sense in all the facts they have to learn, directors report growing difficulties in managing the behavior of the pupils (all three, see straková, spilková & simonová, 2013), parents express fears for their children’s future and pressure to send them to better schools,8 and government officials report falling positions in european testing surveys.9 lower secondary school level education is loaded with factual knowledge from many fields of interest: chemistry, physics, math, history, biology, home economics, music, art, literature, czech language, two foreign languages, geography, physical exercise, civic education, it, and manual training. looking at the list of subjects, it is tempting to say that williamson (1979) was wrong; it seems that acquiring factual knowledge is the goal of education at this level after all. my data show that to many czech parents, the quantity of information, rather than any processes of thinking, is still synonymous with education. this commonly held belief is reflected in the current government’s stance towards homeschooling at the lower secondary level. lower secondary level children whose parents wish to educate them at home must apply to one of the few pilot index schools where the ministry of education is monitoring the process. depending on the will of the officials and the ministers in office, the experiment can be stopped at any time. as such, homeschooling at the lower secondary level has an ambivalent status: neither impossible nor accepted. families who decide on this method of learning and living have to face the uncertainty of the very near future, living from one year to the next. financial demands multiply and circumstances divert them further away from their formal schools towards those that are centrally selected to implement the experiment. this status at the lower secondary level persists despite positive results submitted from all of the pilot index schools to the ministry for almost 10 years now. concluding remarks anthropology of education is not yet firmly established in the czech republic, and the tracking of anthropology of learning is virtually non-existent. this is partially due to the ongoing beliefs of both academics and research boards that topics connected with pedagogy and education are the domain of research teams associated with pedagogical faculties; another reason is that the legacy of communist uniformity continues to dominate the minds and imaginations of many, including social science academics (illich, 1971). 8 see http://ceskomluvi.cz/diskutujte-o-budoucnosti-viceletych-gymnazii, accessed 18.7.2014 9 see pisa survey 2012 on (http://www.pisa2012.cz/?a=vystupy, accessed 19.9.2014 http://ceskomluvi.cz/diskutujte-o-budoucnosti-viceletych-gymnazii http://www.pisa2012.cz/?a=vystupy international electronic journal of elementary education vol.8, issue 1, 161-174, 2015 172 after all, they are themselves the products of a uniform education system, and thus have difficulties imagining otherwise. it is mainly due to culturally creative thinkers that educational alternatives came into existence worldwide, and as such became of interest to social scientists. in the czech republic, the research that connects social science with pedagogy mainly concerns minorities (kašparová & souralová, 2013; nekorjak, souralová, vomastková, 2011) or gender (jarkovská, 2013). the outcomes of this research are similar to others throughout the world – namely that education strives to reduce ethnocentrism, while at the same time it is shaped by it and inevitably also reproduces it (kašparová & souralová 2013; meeusen, de vroome & hooghe, 2013). the emic encounters described in this paper confirm several important facts associated with education and schooling. although these findings have to be read within the legal context of the czech law, they represent a navigation of personal strategies similar to the situation elsewhere in the world (ricci, 2012). first, there is an unresolved ongoing philosophical social and political debate on the meaning of education and schooling. the state, the guardian of the metacapital (bourdieu, 1998), has the final decisive power to accept or reject alternatives. however, the legal guarantee of a right is not in itself a token of acceptance or inclusion. as is the case in the czech republic, the law can contain discriminatory conditions that transform a universal right into a right for a select few. second, parents who opt to homeschool often change their professions so as to accommodate for changes, or share educational responsibilities in order to be able to sustain their professions. as such, homeschooling is practiced by families throughout the whole spectra of society, being functionally adopted by single parents as well as by large two-parent families, living in urban as well as rural settings. what connects the families is that their desire for change relates to their early childhood experience (not necessarily negative), motivating them to be able to imagine and to fight for a difference both for themselves and for their children. despite their own education and training, they are able to step aside, envision an alternative, and assemble agency to make it happen. nevertheless, a homeschooling parent is never free from ties to the formal school and state system. structural adjustments ensure at least a partial penetration of the formal system by the parent into the teaching and learning of the children. it is not an accident that the greatest challenge described by homeschooling parents is to deschool themselves. third, since in homeschooling families there is little or no spatial and institutional division between family life and learning, most homeschoolers do not differentiate between socialization and learning, as described by cohen (2000) and rival (2000). as such, homeschooling resembles the learning strategies of pre-industrialized societies: sibling learning and responsibility, stress upon observation, practice oriented, and the child as an initiator of its own progress (lancy, 2008). yet the resemblance fails in one of the major aspects of the village-learning model: the immediate society does not play the role of a teacher, since usually it does not share the same values. homeschoolers are frequently too different, and are thus excluded from local schools as well as some local (mostly rural) communities; they are forced to create virtual communities via social networks. however, it is difficult to draw conclusions about the development of the issue. there is not enough data within this geographical context yet. in a decade or two, followup studies will be possible and we will be able to tell how the fusion of education and socialization will project itself into the lives of both homeschoolers and the broader society. at the moment, homeschooling opens a world of alternatives for those who seek it actively enough and challenges the practices of post-socialist inclusive education. • • • the right to be included: homeschoolers combat the structural discrimination / kašparová 173 references anderson, b. (1983). imagined communities. london: verso. bartoňová, m. & vítková, m. (2013). vzdělávání žáků se speciálními vzdělávacími potřebami vii. 1. vyd. brno: paido. berger, p. l., luckmann. t. (2001). sociální konstrukce reality. brno: centrum pro studium demokracie a kultury. bledsoe, c. (2000). the cultural transformation of western education in sierra leone. in: levinson, b.a.u. (ed). schooling the symbolic animal: social and cultural dimensions of education. new york: rowman and littlefield publishers. bourdieu, p. (1986.) the three forms of capital. in: richardson, j.g. (ed) handbook of theory and research for the sociology of education. new york: greenwood press. (1998). teorie jednání. praha: karolinum. bourdieu, p. & passeron, j. (1990) reproduction in education, society and culture. london: sage publications. cohen, a.y. (2000). the shaping of men’s minds: adaptations to imperatives of culture. in: levinson, b.a.u. (ed). schooling the symbolic animal: social and cultural dimensions of education. new york: rowman and littlefield publishers. ezzy, d. (2002). qualitative analysis. practice and innovation. london: routledge. hammersley, m. & atkinson, p. (1996). ethnography: principles in practice. london: routledge and kegan paul. harkness, s., super, c.m., et al. (2010). parental ethnotheories of children’s learning. in: lancy, d.f., bock, j., gaskins, s. (eds). the anthropology of learning in childhood. plymouth: altamira press. holt, j. (1995). how children learn. da capo press; revised edition. (1997). growing without schooling: a record of a grassroots movement. holt associates. chapman, k. (1986). the sociology of schools. tavistock publications ltd. illich, i. (1971). deschooling society. new york: harper and row. jarkovská, l. (2013). gender před tabulí: etnografický výzkum genderové reprodukce v každodennosti školní třídy. praha: sociologické nakladatelství. kašparová, i. (2015) homeschooling, freedom and control in czech education. in: global dialogue, university of berkeley: international sociological association, 2015, roč. 5, č. 3, s. 31-32. (2014). how to educate czech children: social network as a space of parental ethnotheories. in: sgem conference on anthropology, archaeology, history, philosophy: conference proceedings. sofia: sgem (2012). children, work and education: first steps to unschooling as a result of current institutional child-care crisis in the czech republic. belgrade international conference on education proceedings. 15-17 november 2012, p. 31-42 belgrade: tomorrow people organization. kašparová, i., souralová, a. (2014). „od lokální k cikánské škole“: homogenizace školní třídy a měnící se role učitele. orbis scholae 1/2014. (2013). teaching at a gypsy school: the process of teacher and pupil segregation in the czech republic. belgrade international conference on education proceedings. 14-16 november 2013, p. 119-130, belgrade: tomorrow people organization. kostelecká, y. (2003). domácí vzdělávání (dizertační práce). praha: pedagogická fakulta univerzity karlovy v praze. (2005). legislativní úprava domácího vzdělávání v zahraničí a její komparace se situací v české republice. pedagogika, 55(4), 354–367. international electronic journal of elementary education vol.8, issue 1, 161-174, 2015 174 (2010). home education in the post-communist countries: case study of the czech republic. international electronic journal of elementary education, 3(1), 29 44. lancy, d. f. (2010). children’s learning in new settings. in: lancy, d.f., bock, j., gaskins, s. (eds.) the anthropology of learning in childhood. plymouth: altamira press. (2008). the anthropology of childhood. cherubs, chattel, changelings. cambridge: cambridge university press. lancy, d.f., bock, j., gaskins, s. ed. (2010). the anthropology of learning in childhood. plymouth: altamira press. malinowski, b. (1929/2005). the sexual life of savages in north western melanesia. keesinger publishing. mead, m. (1928). coming of age in samoa: a psychological study of primitive youth for western civilization. new york: william morrow. meeusen, c. & de vroome, t. & hooghe, m. (2013). “how does education have an impact on ethnocentrism? a structural equation analysis of cognitive, occupational status and network mechanism.” international journal of intercultural relations. 37(5): 507-522. mcdonald, j. & lopes, e. (2014). how parents home educate their children with an autism spectrum disorder with the support of the schools of isolated and distance education. international journal of inclusive education. 18(1). nekorjak, m., souralová, a., & vomastková, k. (2011). uvíznutí v marginalitě: vzdělávací trh, „romské školy“ a reprodukce sociálně prostorových nerovností. sociologický časopis, 47(4), 657680. ogbu, j. u. (2000). understanding cultural diversity and learning. in: levinson, b.a.u. (ed). schooling the symbolic animal: social and cultural dimensions of education. new york: rowman and littlefield publishers. ray, p. h., & anderson, s. r. (2000). the cultural creatives: how 50 million people are changing the world. new york: three river press. reed-danahay, d. (2000). habitus and cultural identity: home/school relationships in rural france. in levinson, b.a.u. (ed). schooling the symbolic animal: social and cultural dimensions of education. new york: rowman and littlefield publishers. ricci, c. (2012). the willed curriculum, unschooling, and self-direction: what do love, trust, respect, care, and compassion have to do with learning? ricci publishing. rival, l. (2000). formal schooling and the production of modern citizen in the ecuadorian amazon. in: levinson, b.a.u. (ed). schooling the symbolic animal: social and cultural dimensions of education. new york: rowman and littlefield publishers. stahl, g. (2015). identity, neoliberalism and aspiration: educating white working-class boys. london: routledge. straková, j., spilková, v., simonová, j. (2013). názory učitelů základních škol na potřebu změn ve školním vzdělávání. orbis scholae, 7(1), 79-100. williamson, b. (1979) education, social structure and development. london: the macmillan press ltd. students’ performance calibration in a basketball dibbling task in elementary physical education international electronic journal of elementary education, 2013, 5(2), 185-198. issn:1307-9298 copyright © iejee www.iejee.com multicultural children’s literature and teacher candidates’ awareness and attitudes toward cultural diversity yuko iwai university of wisconsin, la crosse, usa received: 9 september 2012 / revised: 17 december 2012 / accepted: 6 january 2013 abstract the purpose of this study was to look at elementary/middle school pre-service teachers’ perceptions of multicultural and diversity issues through multicultural children’s literature. nineteen pre-service teachers in a foundation of literacy course explored multicultural children’s literature and involved group/class discussions and a project over the semester. data was collected using the preand postcultural diversity awareness inventory (cdai), open-ended responses, quick writings, projects, and class interactions. the results showed that while they were not familiar with multicultural children’s literature at the beginning of the semester, at the end of the semester they developed positive attitudes toward learning about them, identified them as crucial to foster children’s awareness of diversity, and planned to use them in their future classrooms. they also increased their positive attitudes toward multicultural and diversity issues. in particular, they were most aware of two categories, cultural diversity and multicultural environment, in the cdai. keywords: teacher candidates, multicultural children’s literature, cultural diversity, teacher education, multicultural education. introduction the k-12 school demographic trends have been changing. teachers serve more culturally and linguistically diverse students. according to aud et al. (2011), the number of students whose first language is not english has increased at a rapid speed. for instance, the population of english language learners for ages 5 – 17 increased from 4.7 to 11.2 million from 1989 to 2009 (aud et al., 2011). while the percentage of caucasian students dropped from 68 to 56 between 1989 and 2009, the percentage of hispanic and other students, including asian, hawaiian, alaska native, pacific islander, american indian, or two or more races, increased from 11 to 22 percent and from 4 to 8, respectively (aud et al., 2011). u.s. census bureau estimates that over 50 percent of children will be children with diverse cultural and linguistic backgrounds by 2023 (cited in dillon, 2006). these statistics demonstrate that teachers serve more diverse students.  yuko iwai, department of educational studies, university of wisconsin la crosse, 1725 state street, la crosse, wisconsin, 54601, usa. phone: +16087855407; e-mail: yiwai@uwlax.edu http://www.iejee.com/ international electronic journal of elementary education vol.5, issue 2, 185-198, 2013 186 teachers must prepare for teaching these diverse learners (chicola, 2007; ukpokodu, 2002). they will encounter more diverse learners in their classrooms in the near future. as teacher educators, we must make sure that we promote both teachers’ and teacher candidates’ cultural awareness and diversity and teach children the importance of multiculturalism. however, despite the need for teaching diversity in schools and fostering students’ awareness and respect for others with different cultural and linguistic backgrounds, teachers do not normally see the importance of diversity while teaching. colby and lyon (2004) point out that pre-service teachers are more likely to be unaware of the significant role of multiculturalism in education and have little understanding of the impact of their personal beliefs on classroom interactions, discussion, and practices. teachers also tend to pay less attention to the diversity of students and less likely to integrate diversity issues in their teaching. such educators’ attitudes toward multicultural education would generate a negative impact on students. research also shows the significant role of multicultural literature in schools (cai, 2008). multicultural literature is defined as books that describe people and events about countries and cultures other than the dominant ones. however, teacher candidates do not fully understand multicultural literature and how it can be integrated in their future classrooms because embedding multicultural and diversity pedagogy into teacher preparation programs have not actively implemented (gay, 2000). teacher candidates feel unprepared to teach children of diversity and lack foundations of multicultural education (barksdale et al., 2002). while scholars recognize the significant role of multicultural education and of teaching educators diversity issues, few researchers examine these areas in relation to multicultural children’s books. thus, this research aims to explore teacher candidates’ awareness and attitudes toward cultural diversity by exploring multicultural children’s literature. three main questions guide this study: (a) what are teacher candidates’ perceptions of multicultural literature?; (b) how does exploring multicultural children’s literature impact teacher candidates’ awareness and attitudes toward cultural diversity?; and (c) which category of the cultural diversity awareness inventory (cdai) (cultural awareness, cultural diversity family, cross-cultural communication, assessment, and multicultural environment) are teacher candidates aware of the most? review of literature multicultural education gay (2003) states that people who come from other regions, such as asia and the middle east, to the u.s. are different from the immigrants from western and northern europe in the earlier periods and that people may hold biases and feel anxieties about the people from these unfamiliar countries, cultures, and languages. she calls for embedding multicultural education in schools in order to augment diverse students’ academic achievements and assist other students in becoming meaningful members of the global society. banks describes multicultural education as “an idea or concept, an educational reform movement, and a process” and includes the assumption of providing all students with equal opportunities to learn in school (2010, p.3). according to nieto (2000), multicultural education refers to “a process of comprehensive school reform and basic education for all students. it challenges and rejects racism and other forms of discrimination in schools and society and accepts and affirms the pluralism (ethnic, racial, linguistic, religious, economic, and gender among others) that students, their communities, and teachers reflect” (p.305). she further explains seven important elements of multicultural education: multicultural education is (1) multicultural children’s literature and teacher candidates’ awareness and attitudes / iwai 187 antiracist education, (2) basic education, (3) essential for all students, (4) pervasive nature to schools and communities, (5) education for social justice, (6) a process, and (7) critical pedagogy (nieto, 2000). sleeter and grant (1987) use a different definition, within which ‘multicultural’ includes persons with disabilities; smith (2009) also uses a more extensive definition by including all exceptional individuals. singleton (1996) suggests that multicultural education should address not only particular social groups, but differing perspectives generally including gender, culture, age, and class. as described, various views of multicultural education exist. they give educators insight about being sensitive to this growing diverse society. they call for the importance of teaching multicultural education in teacher education (cai, 1998, 2010; ford & quinn, 2010; ukpokodu, 2002). teacher candidates’ cultural and diversity awareness scholars conducted research on multicultural education and cultural diversity among preservice teachers. frye, button, kelly, and button (2010) compared perceptions of education students before and after a literacy methods course, in which they learned about and experienced culturally responsive teaching. they surveyed 55 elementary teacher candidates, comprising 32 undergraduate and 24 graduate students. the survey included four parts; two of them were analyzed in the study. they are adaptations of siwatu’s culturally responsive teaching self-efficacy (crtse) and culturally responsive teaching outcome expectancy (crtoe) measurements. on each question, students rated themselves on a scale from 0 (lowest) to 100 (highest); each participant’s total and average score was calculated. results indicated that teacher candidates’ dispositions toward culturally responsive teaching significantly increased after taking the literacy methods course, more so for undergraduate than graduate students. the authors remarked that the survey enabled students to assess themselves and observe personal growth. moreover, they were hopeful that multicultural education could increase the likelihood that teaching candidates will actually implement culturally responsive teaching. however, the authors also pointed out that teacher candidates’ perceived increases in multicultural dispositions might not imply actual increases in practice. larke (1990) conducted a study with 51 female pre-service teachers in the elementary program. she examined their cultural awareness by asking them to complete the cultural diversity awareness inventory (cdai) developed by henry in 1986, which measures an educator’s cultural awareness. larke looked at how the participants responded to each item statement organized in five categories: (a) cultural awareness, (b) culturally diverse family, (c) cross-cultural communication, (d) assessment, and (e) multicultural environment. she summarized that more than 90 percent of the participants are aware that their future students may come from different cultural backgrounds and that it is important to identify students by ethnic groups; however, approximately two-thirds of them feel uncomfortable with people who have different values. regarding the culturally diverse family category, while the participants express positive attitudes toward working with diverse students’ parents, larke points out that the participants feel that the parents know little about assessing their children, which she documents as a critical element for the parents who make their child’s academic decisions such as gifted students and students with special needs. about the cross-cultural communication category, the participants value the children’s own language; but at the same time, they reveal that they are uncomfortable with people who speak non-standard english. international electronic journal of elementary education vol.5, issue 2, 185-198, 2013 188 the results of the assessment category demonstrate that the participants viewed the use of languages as playing an important role in child’s assessment. the last category of the multicultural environment showed the participants were positive about creating multicultural environments by providing the children with opportunities to share their cultures, making program adaptations to accommodate diversity need, and including more diverse materials in the classrooms. milner, flowers, moore, moore, and flowers (2003) duplicated the larke’s study. ninetynine pre-service teachers in the teacher education program participated in the study and completed the original cdai. in the study of milner et al., they found some consistencies with the results of larke’s study. for instance, most of the participants in the study of milner et al. were aware of having diverse students in their classrooms. approximately two-thirds of the participants in the study of milner et al. agreed that the teachers need to build interactions with parents outside of school activities. the participants in both studies also were positive about their support of inclusion of esl for english language learners in the regular curriculum. on the other hand, some results of the study by milner et al. did not concur with those of the study by larke. while 62 % of the participants in the study of milner et al. indicated they disagreed with the statement about correcting children’s spoken language by modeling without explanation, approximately half of the participants in larke’s study agreed with this statement. brown (2004a) examined the influence of instructional methodology on cultural diversity awareness of teacher education students in a cultural diversity course. this study took place at an urban university in the midwest with 109 participants. the course was aimed to provide college students with an opportunity to examine and discuss diversity in k-12 classrooms. the class enrollment was divided into two groups: the experimental group and the control group. the experimental group received treatments over different class sessions, including reducing resistance, increasing cognizance of self and others, exploring the interlocking relationship between and within cultural groups, identifying the effects of prejudice on those links, and developing a sense of community. the relationship between pretest and posttest scores of the students was measured by the revised cdai, which henry redeveloped in 1995 based on his extensive research. the results of brown’s study showed that there were positive changes in both groups, but cultural diversity awareness increased more for the experimental group than the control group. brown (2004b) also examined the relationship between self-concept perceptions and cultural diversity awareness and sensitivity among pre-service teachers. brown used two instruments (the tennessee self-concept scale: 2, and the cultural diversity awareness inventory [cdai]) to measure the participants’ self-concept perceptions and cultural awareness in the beginning and at the end of the course. brown found a positive relationship between changes in self-concept perceptions and factors of cultural diversity awareness. as students’ total diversity awareness increased, so did their total self-concept perceptions. multicultural children’s books many multicultural children’s books exist. multicultural children’s literature is “a literature about racial or ethnic minority groups that are culturally and socially different from the white anglo-saxon majority in the united states, whose largely middle-class values and customs are most represented in american literature” (norton, 1995, p.560). multicultural children’s literature refers to books that reflect diverse cultures other than the major culture (botelho & rudman, 2009). bishop (1997) also defines multicultural literature as works “that reflect the multicultural children’s literature and teacher candidates’ awareness and attitudes / iwai 189 racial, ethnic and social diversity that is characteristic of our pluralistic society and of the world” (p.3). researchers also assert the significance of using multicultural literature in schools because it is a powerful device, which helps students develop multiple perspectives about their cultures and provides them with insights about understanding other cultures and people (hefflin & barksdale-ladd, 2001). multicultural literature also cultivates one’s awareness “to other cultural customs and values, promotes communication with people from other countries, and enhances experiences involving theirs and others’ cultures” (lowery & sabisburns, 2007, p.50); it also helps them reflect on themselves and others, change their attitudes, and understand different cultures (cai, 2008). multicultural literature is valued in education especially in this global society and with the increases in the population of diverse students. scholars conducted studies to understand the effectiveness and role of multicultural literature in various settings. montero and robertson (2006) analyzed perceptions of multicultural literature among preand in-service teachers over the semester. other researchers also examined teacher candidates’ attitudes and awareness of diversity using multicultural literature (e.g., escamilla & nathenson-mejía, 2003; howrey & whelan-kim, 2009; szecsi, spillman, vázquez-montilla, & mayberry, 2010). denicolo and franquiz (2006) looked at culturally diverse students’ understandings about multicultural books in a fourth-grade bilingual language arts classroom. in the study of evans (2010), elementary students increased their awareness of diversity after receiving critical, interactive read-alouds from multicultural children’s books. method research questions/research designs three research questions guided this study. descriptions of the research designs follow. the first research question was as follows: what are teacher candidates’ perceptions of multicultural literature? the researcher collected qualitative data in her literacy class over a semester. the second research question was: how does exploring multicultural children’s literature impact teacher candidates’ awareness and attitudes toward cultural diversity? the researcher asked: are there any differences between the pre-cultural diversity awareness inventory (cdai) and post-cultural diversity awareness inventory results? she used onesample t-test on the mean differences to examine whether the mean of the mean differences is equal to zero or not. she examined differences between the pre-cdai and post-cdai scores for each student for each item. the third research question was: which category of cultural diversity among cultural awareness, cultural diversity family, cross-cultural communication, assessment, and multicultural environment are teacher candidates aware of the most? five categories exist in the cdai: (a) cultural awareness, (b) cultural diversity family, (c) cross-cultural communication, (d) assessment, and (e) multicultural environment. the average score for each category in both pre-test and post-test was calculated to examine the trends of the teacher candidates’ awareness of cultural diversity. participants nineteen students, including 18 females and one male, enrolled in a literacy class participated in this study. they consist of one asian and 18 caucasian students. they majored in the elementary/middle school teacher education program (sophomore or junior standing) at a university in the midwestern area in the united states. these teacher candidates pass through four stages in the teacher education program: the pre-field block, field experience one block, international electronic journal of elementary education vol.5, issue 2, 185-198, 2013 190 field experience two block, and student teaching block. these participants were in the first stage of the teacher education program, the pre-field block. instruments cultural diversity awareness inventory (cdai) the cultural diversity awareness inventory (cdai) was originally developed by gertrude b. henry in 1986. it measures educators’ awareness of cultural diversity and multicultural education. it uses a five-point likert scale (1 = strongly disagree, 2 = disagree, 3 = neutral, 4 = agree, and 5 = strongly agree) with some reverse responses intermittently. the cdai has 28 items. later, he revised the original cdai in 1995 by refining wording for the statements along with the professional panel review. a panel of professional experts reviewed all items for content validity, and more than five hundred educators participated in his study. based on the advice and feedback from them, henry revised some items. the overall internal consistency reliability for the revised cdai was .90 (cronbach alpha), using a larger sample of educators. he also confirmed validity using feedback from the panel. the revised cdai was used in this study. the researcher used the following five categories, which brown (2004a) and larke (1990) modified based on the previous work of professional experts in multicultural education including banks (1997): (a) cultural awareness, (b) culturally diverse family, (c) cross-cultural communication, (d) assessment, and (e) multicultural environment. for instance, a sample statement for the cultural awareness category is “i believe my culture to be different from some of the children i serve.” statements for the culturally diverse family include “i believe other than the required school activities, my interactions with parents should include unplanned activities (e.g., social events, meeting in shopping centers), or telephone conversations.” cross-cultural communication statements include “i believe english should be taught as a second language to non-english speaking children as a regular part of the school curriculum.” in the assessment category, the cdai includes statements such as “i believe that translating a standardized achievement from english to another language to be questionable since it alters reliability and validity.” the last category of multicultural environment includes statements such as “i believe it is my responsibility to provide opportunities for children to share cultural differences in foods, dress, family life and/or beliefs.” among 28 items, two items were parallel items, indicating 3 (neutral) being the best answer. for a statistical purpose, these two neutral items were excluded from the study. the reported cronbach’s alpha calculated in this study was .69. questionnaires preand post-questionnaires were used. the questionnaire includes the following open-ended questions. (1) what is multicultural literature? (2) do you know any multicultural children’s books? if so, please write titles of the books. (3) do you know any authors who write multicultural children’s books? if so, please write their names. (4) why do you think multicultural literature is important in education? (5) assume you are a classroom teacher. what particular ethnic group(s) would you like to focus on in your teaching? why? (6) how would you use multicultural literature in your teaching? any particular teaching/reading approaches? in what subjects? the participants completed the questionnaires at the beginning and end of the semester. quickwriting notes the participants provided quickwriting notes several times over the semester. in the literacy course, the instructor shared multicultural children’s books with them. they explored and read multicultural children’s books on their own and participated in guided class/group multicultural children’s literature and teacher candidates’ awareness and attitudes / iwai 191 discussions. they were asked to share their thoughts and comments about the topics covered in class. they were reminded that they did not need to be concerned about spelling or handwriting, but to focus on putting their thoughts on paper. multicultural children’s book project the participants worked on a multicultural children’s book project in the literacy course. they learned about what multicultural children’s books are, why it is important to include them in teaching, how to determine high quality multicultural children’s books, and useful resources to select appropriate multicultural children’s books for children. in particular, they learned how to examine multicultural literature based on the criteria by yokota (1993) and cullinan and galda (1994). data collection and analysis for the first research question, the researcher first organized and collected the data and then transcribed it. after preparation for the data analysis was complete, she conducted “a preliminary exploratory analysis” (creswell, 2008, p.250). she explored the data to get the general sense of it. while exploring the data, she took notes in the margins of the transcripts about some key words, comments, and/or ideas that came to her mind. next, she coded the data by segmenting and labeling. she read the transcripts and then highlighted the materials by finding some trends about the participants’ cultural diversity awareness. after this process, she reviewed her notes and reduced a number of the codes by categorizing the similar codes into one code that embraced them. the researcher continued this process until she achieved saturation. for the second research question, a one-sample t-test was used to examine any differences between the results of the preand post-cultural diversity awareness inventory (cdai). the .05 significance level was set. the researcher looked at a mean difference (postcdai minus pre-cdai score) for each student for each question. for the third research question, after the quantitative data was collected, the researcher organized it based on the five categories: (a) cultural awareness, (b) culturally diverse family, (c) cross-cultural communication, (d) assessment, and (e) multicultural environment for precultural diversity awareness inventory (cdai) and post-cdai scores. the means for all five categories were calculated. she compared the means among the five categories between the pre-cdai and post-cdai scores to determine which category of the cdai the teacher candidates were most aware of among the five categories. results research question 1: what are teacher candidates’ perceptions of multicultural literature? at the beginning of the semester, the teacher candidates were not familiar with multicultural children’s books and did not possess in-depth knowledge about it, nor did they know authors and particular books of multicultural children’s literature. some examples of their perceptions of multicultural children’s books were “multicultural children’s books show the culture of one’s cultural background” and “multicultural children’s literature is literature that involves different cultures.” their definitions of multicultural children’s books were general, and they only saw them as books about people ethnically different from themselves. as we explored books, they began to realize that multicultural children’s books include not only books about ethnic groups, but also people with different beliefs and religions. the teacher candidates self-reported limited knowledge about and reactions to the multicultural children’s books. before they took this literacy course, they acknowledged that international electronic journal of elementary education vol.5, issue 2, 185-198, 2013 192 they did not know many books about diversity. they reported, “i didn’t realize how many multicultural children’s books are written in different languages…i am also shocked by how many multicultural books are out there.” “i didn’t realize how many multicultural books were out there.” “i was amazed how many diversity books there were.” “i was very surprised to see all the resources available on multicultural literature.” part of their limited knowledge about multicultural children’s books was because they were not exposed to them when they were in k-12 school settings. they shared that “i wish i would have been exposed to books pertaining to diverse students.” “multicultural literature was something i was not really exposed to in elementary school that i wish i would have been.” while the teacher candidates were not familiar with multicultural children’s books, they were positive and excited to learn and develop their knowledge about them. they shared “i am really glad we are learning and get insight into many different types of multicultural literature.” “i think it is very interesting to learn about multicultural literature.” another finding was that the teacher candidates saw a strong connection between multicultural children’s books and education. they shared that it is important for children to learn about various types of diversity books. they wrote, “they are crucial in the classroom…” “it is very beneficial for the children of this generation,” “this type of reading is very important to bring into the classroom to both educate and introduce different cultures/races to children....” they concluded that multicultural children’s books “allow children to have appreciation for their differences,” “help them learn about others’ cultures and traditions that they may not have learned about otherwise,” “help them become more accepting of other cultures and people,” and “give them a sense of pride in their own culture.” not only did the teacher candidates develop an understanding of the important role of multicultural children’s books in education, they also plan to use them in their future teaching. they showed that they plan to incorporate these books in their classrooms and use them. they shared “i will definitely incorporate it [multicultural children’s literature] in my classroom.” many pre-service teachers also see themselves using various books across subjects such as social studies, history, geography, reading, art, and english. for example, students shared their idea of teaching the distance between the united sates and laos where the story in a hmong book took place in a geographical setting. in subject area teaching, they viewed that they would use a variety of reading approaches including k-l-w chart, shared reading, and group reading. research question 2: how does exploring multicultural children’s literature impact teacher candidates’ awareness and attitudes toward cultural diversity? the following hypothesis was examined: are there any differences between the pre-cultural diversity awareness inventory (cdai) and post-cultural diversity awareness inventory results? using the .05 level of significance, the researcher recorded a statistical positive difference between the pre-cultural diversity awareness survey and post-cultural diversity awareness survey results (p-value = .016). research question 3: which category of the cultural diversity awareness inventory (cdai) (cultural awareness, cultural diversity family, cross-cultural communication, assessment, and multicultural environment) are teacher candidates aware of the most? the average scores of all five categories for the pre-cultural diversity awareness inventory (cdai) were as follows: 4.00 for (a) cultural awareness, 3.71 for (b) culturally diverse family, 3.24 for (c) cross-cultural communication, 3.47 for (d) assessment, and 4.06 for (e) multicultural environment. these results show that the teacher candidates were most aware multicultural children’s literature and teacher candidates’ awareness and attitudes / iwai 193 of creating multicultural environments for diverse students. the next greater category was in the cultural awareness. on the other hand, the average scores for the five categories for the post-cdai were reported as follows: 4.18 for (a) cultural awareness, 3.91 for (b) culturally diverse family, 3.12 for (c) cross-cultural communication, 3.47 for (d) assessment, and 4.06 for (e) multicultural environment. these results show that while the mean of the category for multicultural environment remained high, the teacher candidates were aware of cultural diversity the most at the end of this study. discussion research question 1: what are teacher candidates’ perceptions of multicultural literature? this paper explored the teacher candidates’ perceptions of multicultural literature. they first were not familiar with multicultural children’s books or did not identify in-depth understandings of their definitions. they could not identify multicultural children’s books; and they were surprised to learn the vast number of multicultural children’s books published and available for parents, teachers, and children. they probably did not experience a lot of opportunities to read these books and/or learn about diverse students when they were in k-12 school settings. however, as the teacher candidates explored multicultural children’s literature through their own research and as they examined them using the criteria for high-quality multicultural children’s books, they began to understand the importance of using them in their teaching. they had positive attitudes toward learning about diversity, cultures, traditions, foods, and people through children’s books. lowrey and sabis-burns (2007) also claim that pre-service teachers, who were uncomfortable reflecting on diversity issues at the beginning of the semester, developed positive attitudes toward diversity issues and learning about multicultural literature by the end of the semester. similar results were found in another study, which explored elementary pre-service teachers’ perceptions of multicultural literature and reported an increase of their positive perceptions (colby & lyon, 2004). in addition, the teacher candidates saw advantages to using multicultural children’s books for children in schools. they shared that incorporating them into teaching helps the students appreciate their own cultures, understand and respect other cultures and traditions, and broaden views of diverse people. furthermore, many teacher candidates were very positive about utilizing multicultural children’s books in their future classrooms. they planned to use them as much as possible for various subjects. not only did they think about using these books in their teaching, but they also formulated specific ideas about how to incorporate them in different subjects using various literacy approaches. research question 2: how does exploring multicultural children’s literature impact teacher candidates’ awareness and attitudes toward cultural diversity? the following hypothesis was examined: are there any differences between the pre-cultural diversity awareness inventory (cdai) and post-cultural diversity awareness inventory results? the results of the one-sample t-test showed a positive statistical difference between the preand post-cdai results. while the increase of the scores was not so significant, the results still show the importance and effects of exploring multicultural children’s books to facilitate the teacher candidates’ awareness of the value of cultural diversity. the result of this study was aligned with that of brown (2004a), which indicated the increase of pre-service teachers’ awareness toward cultural diversity on the post-cdai at the end of the semester. in the study by frye, button, kelly, and frye (2010), pre-service teachers in literacy methods international electronic journal of elementary education vol.5, issue 2, 185-198, 2013 194 courses increased their awareness toward culturally responsive teaching at the end of the semester after learning about it with various approaches, such as exploring multicultural literature and working with children. the findings of this study also concur with montero and robertson (2006), who reported pre-service teachers’ increased awareness toward reading and using multicultural literature in their future teaching over the semester. research question 3: which category of cultural diversity among cultural awareness, cultural diversity family, cross-cultural communication, assessment, and multicultural environment are teacher candidates aware of the most? the results of this study showed the multicultural environment as the highest score of the categories in the pre-cultural diversity awareness inventory (cdai) the teacher candidates reported. on the other hand, in the post-cdai, the cultural awareness category had the highest mean score among the five categories. the second highest mean scores for preand post-cdais were cultural awareness and multicultural environment, respectively. in other words, teacher candidates were most likely be aware of these two elements among the five categories over the semester. some trends were found among this study and previous studies using cdai. for example, the studies of larke (1990) and milner et al. (2003) demonstrated that many pre-service teachers indicated their strong cultural awareness, such as “being aware of differences between the teacher and student.” with regard to the category of multicultural environment, the studies by larke (1990) and milner et al. (2003) support the findings of this study. for example, larke showed strong positive attitudes about four statements, such as “educators should provide opportunities for children to share cultural differences” and “students should be involved in a regular rotating schedule for job assignments” in the category of multicultural environment. milner et al. (2003) reported a slightly different result from that of larke on the statement on “educators place displays and materials showing at least three cultural groups.” in other words, twothirds of the participants agreed with the statement in larke’s study; approximately half of the participants in the milner et al. study indicated neutral on this item. however, their studies had similar findings on the other three statements in this category. furthermore, the results of this study imply that pre-service teachers’ higher awareness was observed in both cultural diversity and multicultural environment, which seemed to be influenced by their literacy course content, including interactions and explorations of multicultural literature and class/group discussions. it may also imply that because they have not experienced actually administering assessment, working and interacting with diverse students and their families in schools, they may not have developed their awareness of other three cultural diversity areas––assessment, cross-cultural communication, and culturally diverse family––as well as they did in cultural awareness diversity and multicultural environment. conclusions this study explored perceptions of teacher candidates’ awareness of multicultural children’s literature. it also examined their awareness and attitude toward cultural diversity using the cultural diversity awareness inventory (cdai). the results demonstrated that they were not exposed to multicultural children’s books. however, they responded very positively toward learning about these books, considered them key tools for children to foster children’s awareness of diversity and respect and tolerant toward differences, and planned to utilize them in their future classrooms across subjects by including various literacy approaches. multicultural children’s literature and teacher candidates’ awareness and attitudes / iwai 195 despite the limited number of the participants, this study offers important insight about the teacher candidates’ awareness and attitudes toward multicultural education and cultural diversity. in order to understand and scrutinize educators’ attitudes toward multicultural education and cultural diversity, the following recommendations are offered for further studies: to examine how the educators’ cultural and educational backgrounds, including ethnicity and past educational experiences, impact their views of cultural diversity; to investigate how their field practices with children through teaching and interacting with their mentor teachers impact their awareness of cultural diversity; and to compare the awareness and attitude toward cultural diversity between preand in-service teachers. . . . yuko iwai, ph.d., is an assistant professor of literacy/reading in the department of educational studies at the university of wisconsin – la crosse, united states. her research interests include reading comprehension, teacher education, multicultural education, and english language learners. references aud, s., hussar, w., kena, g., bianco, k., frohlich, l., kemp, j., tahan, k. (2011). the condition of education 2011 (nces 2011-033). u.s. department of education, national center for education statistics. washington, dc: u.s. government printing office. banks, j.a. (2010). multicultural education: characteristics and goals. in j.a. banks & c.a. banks (eds.), multicultural education: issues and perspectives (7th ed.), 3-30. hoboken, nj: john wiley & sons. barksdale, m.a., richards, j., fisher, p., wuthrick, m., hammons, j., & grisham, d. (2002). perceptions of preservice elementary teachers on multicultural issues. reading horizon, 43(1), 27-48. barnes, c. j. (2006). preparing preservice teachers to teach in a culturally responsive way. the negro educational review, 57(1-2), 85 – 100. bishop, r. s. (1997). selecting literature for a multicultural curriculum. in v.j. harris (ed.), using multiethnic literature in the k-8 classroom (pp.1-19). norwood, ma: christopher-gordon. botelho, m., & rudman, m. (2009). critical multicultural analysis of children's literature. new york, ny: routledge. brown, e. l. (2004a). what precipitates change in cultural diversity awareness during a multicultural course: the message or the method? journal of teacher education, 55(4), 325 – 340. brown, e. l. (2004b). the relationship of self-concepts to change in cultural diversity awareness: implications for urban teacher educators. the urban review, 36(2), 119 – 145. cai, m. (1998). multiple definitions of multicultural literature: is the debate really just “ivory tower” bickering? new advocate, 11(4), 311-324. cai, m. (2008). transactional theory and the study of multicultural literature. language arts, 85(3), 212220. cai, m. (2010). culturally responsive teaching: theory, research, and practice (2nd ed.). new york, ny: teachers college press. international electronic journal of elementary education vol.5, issue 2, 185-198, 2013 196 chicola, n.a. (2007). a view beyond tolerance: teacher candidate experiences with culturally responsive education. international journal of learning, 14(8), 211-219. colby, s. a., & lyon, a. f. (2004). heightening awareness about the importance of using multicultural literature. multicultural education, 11(3), 24-28. cullinan, b., & galda, l. (1994). literature and the child (3rd ed.). new york, ny: harcourt brace. denicolo, c. p., & franquiz, m. e. (2006). “do i have to say it?”: critical encounters with multicultural children’s literature. language arts, 84(2), 157-170. dillon, s. (2006, august 27). in schools across u.s., the melting pot overflows. new york times, a7, 16. escamilla, k., & nathenson-mejía, s. (2003). preparing culturally responsive teachers: using latino children’s literature in teacher education. equity & excellence in education, 36, 238–248. evans, s. (2010). the role of multicultural literature interactive read-alouds on student perspectives toward diversity. journal of research in innovative teaching, 3(1), 92-104. ford, t. n., & quinn, l. (2010). first year teacher education candidates: what are their perceptions about multicultural education? multicultural education, 17(4), 18 – 24. frye, b., button, l., kelly, c., & button, g. (2010). preservice teachers’ self-perceptions and attitudes toward culturally responsive teaching. journal of praxis in multicultural education, 5(1), 1 – 22. gay, g. (2000). culturally responsive teaching: theory, research, and practice. new york, ny: teachers college press. gay, g. (2003). the importance of multicultural education. educational leadership, 61(4), 30-35. hefflin, b. r., & barksdale-ladd, m. a. (2001). african american children’s literature that helps students find themselves: selection guidelines for grades k-3. the reading teacher, 54(8), 810-819. henry, g. b. (1986). cultural diversity awareness inventory. mainstreaming outreach project. hampton university, hampton, va. henry, g. b. (1995). determining the reliability and validity of the cultural diversity awareness inventory. (doctoral dissertation). retrieved from proquest dissertations and theses database. (umi no. 9539219) howrey, s. t., & whelan-kim, k. (2009). building culturally responsiveness in rural, preservice teachers using multicultural children’s literature project. journal of early childhood teacher education, 30, 123-137. larke, p. j. (1990). cultural diversity awareness inventory: assessing the sensitivity of preservice teachers. action in teacher education 12(3), 23 – 30. lowery, r. m., sabis-burns, d.l. (2007). from borders to bridges: making cross-cultural connections through multicultural literature, multicultural education, 14(4), 50-54. milner, h., flowers, l. a., moore, j., moore, j. l., & flowers, t. a. (2003). preservice teachers’ awareness of multiculturalism and diversity. the high school journal, 87(1), 63 – 70. montero, m.k., & robertson, j.m. (2006). master class in teaching children’s literature. journal of children’s literature, 32(2), 27-35. nieto, s. (2000). affirming diversity: sociopolitical context of multicultural education (3rded.). new york, ny: longman. norton, d. e. (1995). through the eyes of a child: an introduction to children’s literature (4thed.). englewood cliffs, nj: merrill singleton, d. l. (1996). multicultural education: the key to global unity. paper presented at the annual meeting of the world council for curriculum and instruction, amritsar, india. multicultural children’s literature and teacher candidates’ awareness and attitudes / iwai 197 sleeter, c. e., & grant, c.a. (1987). an analysis of multicultural education in the united states. harvard educational review, 57(4), 421– 444. smith, e. b. (2009). approaches to multicultural education in preservice teacher education: philosophical frameworks and models for teaching. multicultural education, 16(3), 45-50. szecsi, t., spillman, c., vázquez-montilla, e., & mayberry, s.c. (2010). transforming teacher cultural landscapes by reflecting on multicultural literature. multicultural education, 17(4), 44-48. yokota, j. (1993). issues in selecting multicultural literature for children and adolescents. language arts,70, 156-167. ukpokodu, n. (2002). breaking through preservice teachers’ defensive dispositions in a multicultural education course: a reflective practice. multicultural education, 9(3), 25-33. international electronic journal of elementary education vol.5, issue 2, 185-198, 2013 198 www.iejee.com this page is intentionally left blank http://www.iejee.com/ 277 copyright © www.iejee.com issn: 1307-9298 © 2022 published by kura education & publishing. this is an open access article under the cc by ncnd license. (https://creativecommons.org/ licenses/by/4.0/) international electronic journal of elementary education january 2023, volume 15, issue 3, 277-290 improving the methodological, analytical, and cultural impact of behavior analysis via utilization of group design methods and statistical analyses mark r. dixona,*, zhihui yib, amanda n. chastainc, meredith t. matthewsd abstract introduction the multi-decade debates within the field of behavior analysis as to the possible value and threat of group design methodology and statistical analyses on the purity of the field have weakened the discipline’s maximal impact on the world. this paper rebukes the concerns and suggests that through such adoption behavior analysis may likely better achieve its world-changing ideals, and pragmatic initiatives. we begin with a historical trace of the current debate and describe the pros and cons to design/analysis inclusion, frame such matters within the context of contemporary issues which applied behavior analysts find themselves concerned, and ultimately put forward means by which broader, and perhaps more impactful, research questions can be asked and interpreted. one of the most distinguishable characteristics of the field of behavior analysis is its reliance on a research method approach entitled “single-subject design” (cooper et al., 2007; derosa et al., 2019). this exploratory approach is not to be confused with a case study (bolgar, 1965) whereby a single individual is studied in an uncontrolled manner and simply reported upon afterwards. single-subject design (ssd) approaches to research questions instead systematically analyze the repeated effects of an independent variable on a dependent variable across one or more individual subjects (cooper et al., 2007). this cluster of techniques are employed within the field because of the concerns that many of the field’s founders had regarding the limitations of traditional research methods – often described as group or statistical research methods (sidman, 1960; skinner, 1956). in the early days of behavior analysis, b. f. skinner himself recommended this departure from traditional psychological statistical methods and analysis because: keywords: group design methodology, research design, statistics, singlesubject design received : 1 february 2023 revised : 7 february 2023 accepted : 7 march 2023 doi : 10.26822/iejee.2023.300 a,* corresponding author: mark r. dixon, university of illinois chicago, usa. e-mail: mrdixon@uic.edu orcid: https://orcid.org/0000-0002-0671-1183 b zhihui yi, university of illinois chicago, usa. e-mail: zyi7@uic.edu c amanda n. chastain, university of illinois chicago, usa. e-mail: achast2@uic.edu d meredith t. matthews, university of illinois chicago, usa. e-mail: mmatth22@uic.edu 278 january 2023, volume 15, issue 3, 277-290 you cannot easily make a change in the conditions of an experiment when twenty-four apparatuses have to be altered. any gain in rigor is more than matched by a loss of flexibility. we were forced to confine ourselves to processes which could be studied with the baselines already developed in earlier work. we could not move on to the discovery of other processes or even to a more refined analysis of those we were working with. no matter how significant might be the relations we actually demonstrated, our statistical leviathan had swum aground (skinner, 1956, pp. 113114) essentially, skinner was concerned that the analysis of the group did not provide meaningful analysis of any of the group members, and as such little could be discerned about the behavior of the single subject. and to skinner, that single subject mattered as it was the level of analysis that appeared to be necessary for evaluating and changing behavior. today the utilization of ssd reaches far beyond behavior analysis into other clinical and helping professions such as social work and health care (see bloom et al., 2009 for a textbook length treatment). there is no debate we wish to have over the utility of the ssd as a method within the field of behavior analysis, as the approach has been of great utility for decades in crafting a precise analysis of the controlling variables on an individual’s behavior. furthermore, we make no dismissal over clear discoveries that such a research tradition has allowed for in the field’s history (e.g., functional analysis and intervention; see beavers et al., 2013 for a review of literature of functional analysis). we do, however, have concern that the overreliance on ssd and omitting the inclusion of more traditional research designs has marginalized the impact the field is having on matters of great concern to behavior analysts. for our work to be taken more seriously beyond the walls of our own discipline, we may need to start writing, speaking, and describing our results in broader non-technical language, and look carefully at what is gained and lost by the specific research design chosen by the researcher. this approach is not meant to imply defeat or suggest minimizing of our research endeavors, but rather simply to greatly embrace the very core elements of what it is to be a pragmatic behaviorist – functional utility. beyond the initial proclamations by skinner regarding research methods, many well-known scholars have spoken out against the incorporation of group designs into the field. in the late 1990s, a group of presenters at the annual association for behavior analysis convention debated the adoption of these “non-behavioral” methods, and their views were eventually assembled within a special section of one of the field’s peer-reviewed journals. in this special section of papers, a range of opinions are presented with repeated concerns being made about loss of understanding the behavior of the individual person (branch, 1999; perone, 1999), and inferencing beyond the data (ator, 1999; branch, 1999; davison, 1999), and the additional distraction that statistical analyses create (perone, 1999) as a drift away from ssd could have for the field. only an occasional pro-group design approach was presented (crosbie, 1999), and statements as such hinted at a potential fractioning that might be underway even within the field of behavior analysis. more recent discussions on the superiority of ssds have echoed similar sediments (kyonka et al., 2019). examples of group design methods and statistical analyses will occasionally appear in behavior analytic journals today (e.g., dixon et al., 2022; jang et al., 2012, sutton et al., 2022; silverman et al., 2007; yi et al., 2022), yet they are a minority compared to the continued use of ssds throughout. even with such examples appearing within our own collection of scholarly journals, most behavior analytic textbooks fail to describe the rationale and usage of group methods and analyses for behavior analysts (cooper 2007; mayer et al, 2019; sidman, 1960; yet see belisle et al, 2021 as an exception) – thus potentially limiting an awareness of value and an understanding of how to construct research questions utilizing group designs. the current social and political environment often has placed the field of behavior analysis in its crosshairs. we are increasingly being described as a field of insensitive determinants of client autonomy (kirkham, 2017; mcgill & robinson, 2021), responsible for the development of alleged trauma in former clients (kupferstein, 2018), insensitive to racial injustices (e.g., čolić et al., 2022; zarcone et al., 2019), and behind trends of interest that need to be more fully addressed and analyzed (e.g., defelice & diller, 2019; fontenot et al., 2019; kornack et al., 2019; morris et al., 2021; wang et al., 2019). even our most heavily dominated applied appendage – autistic care – is being challenged as non-effective (united states of america department of defense, 2021). critics from within and beyond the field itself seem to believe that perhaps behavior analysis has not aged well in a fast-changing and culturally evolving society. we are clear of the risks that any field of inquiry may encounter when it too quickly drifts from historical roots because of modern themes and current interests. yet on the other hand, a field which ignores the critique of itself by its own members suggests that a possible reappraisal may be indeed necessary. and furthermore, if the field’s reaction to such criticism is with rhetoric and not data, increased dismissal of the utility and value of the overall field may result. even those who speak up about creating change in the discipline (e.g., jaramillo & nohelty, 2022; mathur & rodriguez, 2022; pritchett et al., 2022; wright, 2019), doing better than in the past (e.g., baires et al., 2022; li et al., 2019), or improving inclusivity (e.g., deochand & costello, 2022; levy et al., 2022; lovelace et al., 2022) cannot and should not rest 279 improving the methodological, analytical, and cultural impact of behavior analysis / dixon, yi, chastain & matthews after such assertions alone, but only after producing data by which to support such claims. we believe that only through data that change will occur at the magnitude of impact that appears desired, and most importantly the type of data that will yield the greatest change-making potential will be gathered using between group, large sample sized research designs and statistical analyses. many of the most impactful contributions in terms of scalability to improving the human condition have occurred when behavior analysts have adopted non-ssd approaches to demonstrating effects of the independent variable. one example involves the use of contingency management for the treatment of substance use disorders (e.g., dunn et al., 2008; higgins et al., 1994; higgins et al., 1991; ledgerwood et al., 2008). in many of these published studies, a comparison is made between groups of individuals assigned to either a traditional treatment condition in which participants receive drug treatment as usual (e.g., standard relapse prevention support, health risk education, group meetings, and individual counseling sessions), or a contingency management condition wherein abstinence behaviors resulted in payment in the form of vouchers exchangeable for community retail items (see higgins et al., 2019 for a recent review of the literature). related studies on analyzing the choice making of drug users have also centered around non-ssd methods and analyses (e.g., heil et al., 2006; nighbor et al., 2019; thrailkill et al., 2022; yoon et al., 2007). another example of behavioral solutions that have been quite successful at achieving widescale acceptability and adoption is using relational framing techniques to treat mental and physical health conditions under the auspices of acceptance and commitment therapy (dixon et al., 2023). this treatment approach has almost exclusively utilized between-subjects research methods (see twohig et al., 2007 as an outlier in its use of ssd), and gathered enough data to be deemed as effective enough for the world health organization (who, 2020) to distribute act self-help material in 21 languages “for anyone who experiences stress, wherever they live and whatever their circumstances” (p. 5). a final example comes from positive behavior intervention and supports (pbis; horner & sugai, 2015) whereby socialculture and behavioral supports are implemented school-wide (tier 1 and tier 2) and at the level of the individual (tier 3) to promote improved educational and social outcomes. many documented successes of this work are presented with group designs speaking to comparisons made between non-pbis exposed and pbis exposed student groups of varying demographics, whereby the pbis exposed students tend to fair better regarding social-emotional functioning, behavioral concerns, academic performance, bullying and peer rejection, and prosocial behavior (bradshaw et al., 2010; bradshaw et al., 2012; horner et al., 2009; waasdorp et al., 2012). many more examples of behavior analytic researchers who have stretched beyond the ssd research tradition can be found in the context of functional analysis (kurtz et al., 2013), gambling (habib & dixon, 2010), and the good behavior game (joslyn et al., 2019). all of this is not to suggest that ssd themselves do not yield utility for better understanding of human behavior. we completely agree that ssd has a crucial role to play in the behavior analysis of today and tomorrow. however, we also must accept that in order to advance beyond our current limited impact we have made to changing the world through behavioral science (dixon et al., 2018), that we should look carefully at what many of our most successful endeavors all appear to have in common – group research designs. single-subject research designs have limitations related to the generalizability of the study findings as well as the methodological constraints that limit the use of inferential statistical methods. conclusions can hardly be made regarding a group or groups of subjects, as the baseline logic (cooper et al., 2020) underneath most ssds fundamentally focuses on inferencing the likelihood that a procedure is responsible for producing the observed changes at an individual level. this does not speak to the likelihood that a similar effect can be observed when such a procedure is applied to the population, the group of individuals upon whom behavior analysts wish to bring socially significant changes. to a certain degree this limitation is mitigated through systematic replications. population-level inference is usually drawn by first randomly sampling the target population to create a group of subjects and then using the changes observed in the group under the procedure to make inferences about the likelihood of whether the population will respond in similar ways. additionally, the field of applied behavior analysis (aba) takes pride in its continued use of technical terminology which has largely allowed behavior analysts to effectively communicate with other behavior analysts. however, this technical language may be a barrier preventing behavior analysis to reach professionals from other disciplines (becirevic et al., 2016). historically, this insular vocabulary and research approach were the very reason the field crafted its own scientific journals (e.g., journal of applied behavior analysis, journal of the experimental analysis of behavior) to combat the inability to publish its research using ssds in more traditional psychology journals that were departing from a behavioral tradition (gollub, 2002). all the positive features of group design and analysis do not imply that shortcomings of the approach fail to exist. issues of sample representation, clinical/practical significance, effect sizes, maintenance, functional control, and generalization all remain ripe for continued debate. in conclusion, we believe that the 280 january 2023, volume 15, issue 3, 277-290 benefits of incorporating group designs and analyses outweigh the limitations – and as such, behavior analysts can advance further towards having a positive impact on the world by incorporating these sorts of methods into the means by which they speak to matters of interest, react to critics, and advocate for behavioral solutions to non-behavioral audiences. group designs in behavior analytic settings in contrast to single-subject designs, quantitative studies using group designs use variables among participants within one group or across multiple groups to examine the impact of independent variables on dependent variables. in order to aggregate these data, various descriptive and inferential statistical methods are used to provide a relatively objective interpretation. there are many nuances in how group design studies are categorized and the different optimal statistical methods that go with them, such as complex mixed designs better analyzed using structural equation modeling (duncan, 1969) and time series designs (gottman et al., 1969). in this section, though, we primarily focus on three types of group design in their basic forms: within-subject group design, between-subject group design, and mixed group design. we also provide a brief description of the design, common statistical methods used, a sample question that can be studied using this design, and how this research method can be used to address some contemporary issues surrounding our field. within-subject group design the concept of a within group research design is that individual subjects are evaluated multiple times during the experiment. such a design is more similar to an ssd than other sorts of group designs, whereby the individual subject is only examined once and compared to other subjects. here in a within group design, a group of subjects may be exposed to one same independent variable multiple times, a range of levels of an independent variable, or a combination of variables. for example, 20 children with attentiondeficit/hyperactivity disorder (adhd) may be exposed to behavioral interventions for 8 weeks, and also medication for 8 weeks. the order of delivery of treatment may be randomized across the entire 20 children, and after exposure to both (i.e., 16 total weeks), an analysis could be made as to which sort of treatment was better in terms of outcomes on a dependent variable (e.g., performance, attention in class, parent reports of homework completion). variations of this basic framework could include comparing low and high doses of adhd medication, comparing behavioral treatment alone to behavioral treatment with medication, a period of no-treatment to that of behavioral treatment, or even low and high doses of behavioral interventions alone. as these examples illustrate, in a within-subject group design, dependent variables are collected from participants within one same group assignment, sometimes across multiple time points. participants are usually selected based on similar criteria (e.g., demographic compositions, existing conditions, exposure to similar interventions), and statistical methods are used to compare the relationships of group-level measures across multiple time points or to identify patterns and relationships among different variables. using within-subject group design, behavior analysts craft purpose statements such as evaluating treatment progress over time for one group of participants, identifying relationships between outcome measures and potential predictors, and evaluating the extent to which measurements used are reliable and accurate. in contrast to ssd which heavily relies on visual inspection of the data obtained to determine effects of the intervention, group designs supplement the visual graphical differences that are plotted through the use of statistical procedures. most commonly, these questions can be answered using the data obtained through repeated measure t-tests, simple linear regression, and correlation analysis. to comprehensively discuss these statistical methods is beyond the scope of the current paper, and interested readers are highly encouraged to reference statistics textbooks in psychology or education (e.g., howell, 2012). here we present examples with from published peer-reviewed articles to briefly discuss these study’s conceptualization and statistical analysis process, as well as suggestions on how similar investigations can be made to address some emerging research questions and challenges the field faces. treatment progress overtime consider the following scenario: a behavior analyst is interested in evaluating whether a comprehensive treatment model (ctm) newly introduced at their clinic effectively produces measurable gains among a group of autistic learners. although such a question can be examined by individually evaluating the outcome measures of each learner, it might be difficult to aggregate such data due to insurance reimbursement or comparisons across agencies when some learners are showing improvement in outcome measures, and some are showing decreases. furthermore, with varying baseline measures, it is difficult, if not impossible, to control for the potential differing effect of the intervention on learners with varying abilities. although interpretations can be made by categorizing learners into different groups and evaluating the change in the outcome measure’s level, trend, and variability, as routinely done in visual analysis (cooper et al., 2020), such interpretation is largely subjective in nature although some tools have been constructed to improve objectivity (dowdy et al., 2022). 281 improving the methodological, analytical, and cultural impact of behavior analysis / dixon, yi, chastain & matthews a within-subject group design will be a good fit in scenarios like this. in this case, the investigator will gather all learners’ (i.e., population) or randomly select a subgroup of learners’ (i.e., sample) data before and after the implementation of the ctm. here we would compare each learner’s follow-up measure against their baseline measures, and the most common will use repeated measure t-tests to detect whether there are statistically significant changes between the two time points. for example, yi et al. (2022) investigated the impact of aba service embedded within the student’s individualized education program (iep) using data obtained from a public school. a repeated measure t-test showed a statistically significant difference between students’ performance at the beginning and the end of the school year. figure 1 illustrates the comparison being made in this scenario. figure 1. illustration for using within-subject group design to evaluate changes for a group of participants between baseline and follow-up. within-subject designs using similar methods to evaluate participants’ outcomes over time can have meaningful impact in the contemporary field of behavior analysis. there has been increasing attention in using behavior analytic principles to address social issues such as systematic racism (shea et al., 2022), as well as critical reflections on the research and clinical practice of behavior analysis among the black community (čolić et al., 2022; lovelace et al., 2022). čolić et al. (2022) synthesized black caregivers’ experience when it comes to autism care and provided specific accounts on racism manifested across its multiple stages. čolić et al. provided multiple examples of how to address institutional racism and offered specific recommendations for aba providers to combat racial bias. sevon (2022) also proposed similar recommendations on increasing awareness on antiblack racism. an important step the field should take is to diligently listen to the voice from the community, consumers, and stakeholders and act accordingly to design and implement the behavior-changing system to dismantle these issues. such endeavor can be strengthened by using within-subject research designs. for example, as an extension to čolić et al., researchers might design and implement an intervention package among service providers consisting of awareness training on black cultural values and intersectionality between blackness and autism. they could subsequently incorporate behavior skill training on strengthening partnerships among stakeholders. a study can be conducted by first gathering qualitative and quantitative data on black caregiver experience within behavior analytic settings. after the implementation of the intervention package among service providers, multiple waves of follow-up measures can be taken. using withinsubject group designs, statistical analysis can be done to detect whether the intervention package produced measurable improvements on black caregivers’ experience. surely a single-subject approach could also be crafted here, however external validity is inherently reduced. predictors for outcome another area where within-subject group design can be used is to identify predictors for certain outcome measures. consider the following scenario: a behavior analyst who works at a local school district is interested in exploring whether there is a relationship between the amount of aba service received and students’ progress. in order to advocate for more resources devoted to aba, the education team needs to reasonably demonstrate a relationship between the dosage of aba and the amount of progress. in this situation, ssds cannot easily answer this question. the outcome of aba is often measured across multiple weeks or months. with alternating treatment designs, the short exposure to each dosage condition is not powerful enough to produce meaningful changes that can be detected. with reversal designs across multiple dosage levels, the potential confound of sequence effect and carryover effect is so large that it is very difficult to attribute measured gains to a specific condition. an alternative is to visually inspect a scatterplot with the treatment dosage on the x-axis and the amount of progress on the y-axis. however, when the number of learners is low and with high variability in outcome measures, it might be difficult to visually interpret the trend of the progress as the dosage increases along the x-axis. a within-subject group design would be a good fit in this situation. instead of relying on ssds or visual analysis of the scatterplot, a linear regression analysis can be done to identify whether the dependent variable can be reliably predicted by a single or 282 january 2023, volume 15, issue 3, 277-290 multiple independent variables. for example, yi et al. (2022) was interested in identifying factors that could predict participants’ gain in school readiness skills. using a cohort of 17 autistic students within a public school, the researcher conducted a simple linear regression on the dosage of peak-based instruction on participants’ gain in bracken school readiness scale (bsra; bracken, 2007). results showed that the amount of peak-based instruction was a statistically significant predictor for their gain on bsra, f(1,14) = 5.31, p = .036. the dosage of peak-based instruction accounted for 27.80% of the variance observed in bsra gain. studies and analyses like this can deepen our understanding among several issues regarding the training for the next generation of behavior analysts and ensuring high quality of care. several recent studies analyzed data released by the behavior analyst certification board (bacb) on certification outcomes among accredited and verified course sequences in this field (dubuque & kazemi, 2022; matson & konst, 2014). by comparing certification outcomes among multiple applicant characteristics (e.g., program mode and accreditation status), researchers reported trends in the number of applicants in the last decade and differences observed among applicants experienced different modes of learning (e.g., in person, remote, hybrid). an extension of this body of work would be to use within-subject group designs and to use analyses such as linear and logistic regression models in identifying the environmental factors that are mostly likely to impact the educational outcome. researchers need to first gather more comprehensive applicant data, such as demographic information, social economic status, educational information (e.g., program mode, curriculum design, faculty-student ratio), fieldwork and supervision experience, and continue education. researchers also need to collect more comprehensive outcome measures besides the board exam pass rate, such as consumer satisfaction, and apply appropriate statistical tests to identify predictors of these outcome variables. similarly, the field has become increasing aware of staff burnout and its detrimental impact on the quality of care (plantiveau et al., 2018). by using similar group design method, researchers can identify predictors of staff burnout and develop corresponding strategies to improve the quality of care. psychometric properties behavior analysts have a long history in designing measurement systems for behavior changes overtime. focusing on the individual’s learning history, the field has long cautioned against standardized testing due to concerns on the inability to individualize the assessment’s process, which might more accurately reflect on the behavior observed (ayllon & kelly, 1972; koegel et al., 1997). as the field continues to evolve and expand, concerns have been raised on the reliability of many assessments aba providers use in clinical settings. the field of psychometrics studies the construction and application of assessment tools and an assessment’s psychometric properties describe how well it measures what it claims to measure. most commonly, researchers evaluate the instrument’s validity and reliability, and, often time for a newly developed instrument, the extent to which the instrument yields similar outcome to established measures (nunnally & bernstein, 1994). in a systematic review conducted by ackley et al. (2019), only four of the 18 aba-based assessments reported data supporting its reliability. evaluating psychometric properties of aba-based assessment is not only a valid scientific objective, but also offers many benefits such as increasing the external validity of the field, simplifying assessment process, and increasing dissemination beyond behavior-analytic journals (sutton et al., 2022). issues like this cannot be answered by ssd, and within-subject designs can be useful in studying the psychometric properties of the instrument. for example, sutton et al. (2022) evaluated the convergent validity and internal consistency of the peak comprehensive assessment. lenoir et al. (2022) evaluated the convergent and age appropriateness of the children’s psychological flexibility questionnaire. research studying psychometric properties is critically important to our field. an inaccurate or skewed measurement system is likely to render whatever conclusions made based on the observation or whatever progress captures via data collection invalid. this need is further amplified as the field keeps expressing interests on social issues, such as cultural humility and cultural responsiveness (kolb et al., 2022; wright, 2019). the behavior being measured can no longer be limited to simple operant classes. when a measurement system is developed and used to capture a group of behavior constituting a dynamic behavior system, the lack of psychometric studies on these instruments is concerning. luckily, researchers are beginning to pay more attention during the development process and often seek to obtain feedback to revise their early draft. for example, gatzunis et al. (2022) developed a culturally responsive supervision self-assessment (crss) tool for supervisors to self-reflect on their cultural responsiveness during supervisions. gatzunis et al. gathered feedback on crss and collected social validity data for its final form. an extension of their work would involve rigorous psychometric analysis of crss. using within-subject group designs, researchers can extend this work by collecting responses among a large number of supervisors. crss’s three domains can be verified by calculating the internal consistency among all items within the same domain. furthermore, factor analysis can be used to examine whether the construct of the instrument correspond 283 improving the methodological, analytical, and cultural impact of behavior analysis / dixon, yi, chastain & matthews to its theoretical underpinning, with the rationale being that items measuring the same domain should converge while items from different domains should be relatively independent. test-retest reliability can be examined by administering the crss twice with one same group of participants and calculating the correlation between the two outcomes. convergent validity can be examined by comparing the outcome of crss with established measures. content validity can be examined by synthesizing the input from a group of subject matter experts. and most importantly, researchers can compare whether the self-reported crss outcome corresponds to perceptions from the supervisee. between-subject group design in contrast to within-subject group designs, betweensubject group designs compare dependent variables collected between two or more groups of participants. participants are usually categorized into multiple groups based on conditions and demographic characteristics, or are intentionally assigned to different groups which, later on, are exposed to different conditions (e.g., treatment options, waitlists). here, the analysis primarily focuses on detecting the differences between the groups, which in turn, speaks to the impact independent variables have on dependent variables. researchers can answer questions such as whether an added component of act can increase parental adherence to an online aba caregiver training program (yi & dixon, 2021), comparing the efficacy of relational training procedures on intelligence (may & st. cyr, 2021), and whether autistic individuals perform differently during skill assessments compared with neurotypical peers (dixon et al., 2017). most commonly, these questions are answered using independent sample t-tests or analysis of variance (anova), depending on the number of groups. figure 2 illustrates the comparison being made in this scenario. figure 2. illustration for using between-subject group design to compare differences among multiple groups. compared with within-subject group designs, between-subject group designs have several methodological advantages and challenges. in a between-subject group design, one major concern is the inherited differences between the two groups. suppose the researcher wants to compare two types of aba intervention on participants skill gain across six months. had two groups not being equal at the baseline condition, one can make the argument that any observed differences in skill gain might be attributed to the differing foundational learning skills between the two groups, rather than the different intervention. in within-subject group designs, differences among participants are less of a concern as participants are each compared against themselves, thus controlling for this difference. another disadvantage for between-subject group designs is the requirement of the sample size. a between-subject group design using two groups of participants effectively double the number of participants required. this also leads to its potential insensitivity in detecting the treatment effect. more variability is inherently introduced with a larger number of participants. this variability often leads to smaller power in the statistical method used, decreasing its ability in detecting smaller changes. in other words, with all things being equal, studies using between-subject group designs need to produce a larger effect to avoid type-ii errors. at the same time, between-subject groups design also offer many methodological advantages. it is generally more flexible than within-subject group designs in the statistical methods used (keppel, 1982), and can avoid sequence effects, which could be detrimental in certain within-subject design studies. when evaluating the differences across multiple treatment conditions using a within-subject research design, the order in which these conditions are exposed to participants might have a significant impact on the dependent variable. at the same time, a previously exposed condition might have carryover effects on the latter condition. this concern is similar to that of reversal designs used in ssds. when using betweensubject group designs, however, sequence effect is less of a concern since each group is independently exposed to its own condition. comparing treatment outcomes a common application of between-subject group designs in behavior analytic settings is to compare outcomes of multiple treatment options. this is arguably one of the most important applied questions the field needs to answer: what works and what works better. although ssds such as alternating treatment designs and component analysis provide powerful demonstration on the impact of behavior-change procedures with an individual, and allows a direct comparison between different treatment conditions, aggregating such findings at a group level would better address the question on what is likely to work 284 january 2023, volume 15, issue 3, 277-290 when similar intervention is applied to a larger group of individuals. for example dixon et al. (2021) evaluated the impact of relational training procedures on participants’ intelligence. the researcher randomly assigned a group of 17 autistic participants into two groups: a comprehensive aba (c-aba) group involving relational training procedures and a traditional aba (t-aba) group receiving instructions based on contingencybased learning and generalization, but not content that incorporated derived relational responding. in addition, 11 participants currently on the waiting list for aba services served as a convenient waitlist control group. all participants’ iq was measured at baseline and after 12 weeks of intervention. a one-way anova was conducted to compare the differences in participant’s iq change score. results showed a statistically significant difference in participant’s iq change score among the three groups, f(2,26) = 5.80, p = .008. post-hoc analysis indicated a statistically significant difference between participants in the c-aba group and t-aba group (p = .042), and participants in the c-aba group and the waitlist control (p = .009). no statistically significant difference was detected between participants in the t-aba group and those in the waitlist control (p = .841). studies like this using between-subject group designs are extremely important as functional utility is at the very core for a pragmatic behaviorist: questions concerning what works and what does not work to produce behavior change. with new intervention approaches and competing treatment options being developed daily, partitioners are ultimately tasked with providing the most appropriate and effective care. to answer the “what works” question requires using between-subject group designs. for example, there has been ongoing debate on optimal parameters of error correction and prompting strategies in discrete trial training. yet often time, research remained at the individual level, with different studies reporting different outcomes. this speaks to the issue mentioned above, as it is difficult to synthesize outcomes from multiple or even a single study ssd with participants showing varying outcomes. an alternative would be to randomly assign participants into multiple groups, with each group exposed to one study condition. for example, participants in group a will always receive errorless teaching procedures while participants in group b will always receive least-to-most prompting. researchers can then compare outcome measures between the two groups, such as trial to criteria, number of targets mastered, and social validity data. often time, such analyses can be strengthened by introducing within-participants variables and this research design is called mixed group design. mixed group designs mixed group designs usually involve analyses that are conducted both at the within-subject level and at the between-subject level. they are often used in longitudinal studies involving multiple treatment conditions. here participants are compared against their peers with different group assignments and against themselves across different timepoints. a wide range of statistical models can be used to detect the effect of group assignment and time, as well as to explore the interaction effect of the two independent variables. general linear models and mixed-anova are often used in studies using group designs. figure 3 illustrates the comparison being made in this scenario. figure 3. illustration for using a 2 (time 1 vs time 2) x n (group 1 vs group 2 vs … vs group n) mixed group design to compare differences among multiple groups at both timepoints. randomized controlled trials among the few studies in the field of behavior analysis that utilize mixed group designs, the majority of them fall under the category of randomized controlled trials (rcts). rcts are widely accepted as a good standard in conducting casual analysis on the treatment’s outcome with agreed upon procedural safeguards in maintain its internal and external validity (e.g., the consort statement; schulz et al., 2010). in an rct, eligible participants are randomly assigned into multiple groups of conditions with dependent measures captured throughout the study at various timepoints. during the analysis, researchers analyze the trajectory of dependent variables within each group, as well as comparing them among all groups at various timepoints. for example, sanders et al. (2020) conducted a rct evaluating the impact of a rapid aba assessment and treatment protocol among hospitalized autistic 285 improving the methodological, analytical, and cultural impact of behavior analysis / dixon, yi, chastain & matthews children. sanders et al. randomly assigned 36 eligible participants into two conditions. those in the treatment group received a latency-based functional analysis and corresponding function-based behavior reduction plan. those in the control condition received no active behavioral intervention. participants clinical functioning, length of hospitalization, and perception from the medical team were evaluated before and after discharge. results showed preliminary support of incorporating aba procedures in in-patient hospital settings as those assigned to the treatment group demonstrating more improvements at a statistically significant level. in another example, a re-analysis of the dixon et al. (2021) study was conducted using mixed-anova to explore the potential interaction effect between group assignments (c-aba vs t-aba vs waitlist) and time (baseline bs follow-up; yi et al., 2021). results showed a statistically significant main effect of time and a statistically significant interaction effect. rcts can be one of the most powerful tools for applied researchers, especially in addressing concerns on aba’s overall effectiveness (united states of america department of defense, 2021). a longitudinal rct with multiple waves of data tracking participants overall development will provide strong evidence on the intervention’s effectiveness or the lack of. conclusions the exponential rise in the number of behavior analytic professionals signals an extremely bright future for the field of behavior analysis. the growth of the discipline alone is a metric of utility that our science has on saving the world around us (dixon et al 2018). the time is ripe to couple this rise in popularity of the discipline with a rise of impact and verification that yes indeed – this field matters. we believe that a slight pivot from the reliance of ssds to a greater adoption of group design methodology could produce great influences for our field to be taken seriously by outsiders. training programs in behavior analysis should broaden their coursework in research designs to include some of methods noted here within. clinicians should begin to more carefully examine how to optimize a blend of routine care with research techniques such as regression models, waitlist controls, and environmental comparison studies. activists within the field wishing to champion a cause, should come forward with data – as such will more quickly alter cure idle hands and silent majorities. our field has been defined as an enterprise deeply entrenched in pragmatic utilitarianism. therefore, it is time to make peace with the pragmatic gains that can be accomplished via the occasional adoption of group designs into the field of behavior analysis at a more robust level than historically has occurred. author note funding provided in whole or in part by the autism program of illinois and the illinois department of human services. references ackley, m., subramanian, j. w., moore, j. w., litten, s., lundy, m. p., & bishop, s. k. (2019). a review of language development protocols for individuals with autism. journal of behavioral education, 28(3), 362-388. https://doi.org/10.1007/s10864019-09327-8 ator, n. a. (1999). statistical inference in behavior analysis: environmental determinants? the behavior analyst, 22(2), 93-97. https://doi. org/10.1007/bf03391985 ayllon, t., & kelly, k. (1972). effects of reinforcement on standardized test performance. journal of applied behavior analysis, 5(4), 477-484. https:// doi.org/10.1901/jaba.1972.5-477 baires, n. a., catrone, r., & may, b. k. (2022). on the importance of listening and intercultural communication for actions against racism. behavior analysis in practice, 15(4), 1042-1049. https://doi.org/10.1007/s40617-021-00629-w beavers, g. a., iwata, b. a., & lerman, d. c. (2013). thirty years of research on the functional analysis of problem behavior. journal of applied behavior analysis, 46(1), 1-21. https://doi.org/10.1002/ jaba.30 becirevic, a., critchfield, t. s., & reed, d. d. (2016). on the social acceptability of behavior-analytic terms: crowdsourced comparisons of lay and technical language. the behavior analyst, 39(2), 305-317. https://doi.org/10.1007/s40614016-0067-4 belisle, j., stanley, c. r., & dixon, m. r. (2021). research methods for the practicing behavior analyst. emergent press llc. bloom, m., fischer, j., & orme, j. g. (2009). evaluating practice : guidelines for the accountable professional (5th ed.). allyn and bacon. bolgar, h. (1965). the case study method. in b. b. wolman (ed.), handbook of clinical psychology. mcgraw-hill. bracken, b. a. (2007). bracken school readiness assessment third edition. the psychological corporation. 286 january 2023, volume 15, issue 3, 277-290 bradshaw, c. p., mitchell, m. m., & leaf, p. j. (2010). examining the effects of schoolwide positive behavioral interventions and supports on student outcomes: results from a randomized controlled rffectiveness trial in elementary schools. journal of positive behavior interventions, 12(3), 133-148. https://doi.org/10.1177/1098300709334798 bradshaw, c. p., waasdorp, t. e., & leaf, p. j. (2012). effects of school-wide positive behavioral interventions and supports on child behavior problems. pediatrics, 130(5), 1136-1145. https://doi. org/10.1542/peds.2012-0243 branch, m. n. (1999). statistical inference in behavior analysis: some things significance testing does and does not do. the behavior analyst, 22(2), 8792. https://doi.org/10.1007/bf03391984 čolić, m., araiba, s., lovelace, t. s., & dababnah, s. (2022). black caregivers’ perspectives on racism in asd services: toward culturally responsive aba practice. behavior analysis in practice, 15(4), 1032-1041. https://doi.org/10.1007/s40617021-00577-5 cooper, j. o., heron, t. e., & heward, w. l. (2007). applied behavior analysis. pearson education, inc. cooper, j. o., heron, t. e., & heward, w. l. (2020). analyzing behavior change: basic assumptions and strategies. in applied behavior analysis third edition. pearson education. crosbie, j. (1999). statistical inference in behavior analysis: useful friend. the behavior analyst, 22(2), 105-108. https://doi.org/10.1007/bf03391987 davison, m. (1999). statistical inference in behavior analysis: having my cake and eating it? the behavior analyst, 22(2), 99-103. https://doi. org/10.1007/bf03391986 defelice, k. a., & diller, j. w. (2019). intersectional feminism and behavior analysis. behavior analysis in practice, 12(4), 831-838. https://doi. org/10.1007/s40617-019-00341-w deochand, n., & costello, m. s. (2022). building a social justice framework for cultural and linguistic diversity in aba. behavior analysis in practice, 15(3), 893-908. https://doi.org/10.1007/s40617-02100659-4 derosa, n. m., novak, m. d., morley, a. j., & roane, h. s. (2019). comparing response blocking and response interruption/redirection on levels of motor stereotypy: effects of data analysis procedures. journal of applied behavior analysis, 52(4), 1021-1033. https://doi.org/10.1002/ jaba.644 dixon, m. r., belisle, j., rehfeldt, r. a., & root, w. b. (2018). why we are still not acting to save the world: the upward challenge of a post-skinnerian behavior science. perspectives on behavior science, 41(1), 241-267. https://doi.org/10.1007/ s40614-018-0162-9 dixon, m. r., hayes, s. c., & belisle, j. (2023). acceptance and commitment therapy for behavior analysts: a practical guide from theory to treatment. taylor & francis ltd. dixon, m. r., paliliunas, d., barron, b. f., schmick, a. m., & stanley, c. r. (2021). randomized controlled trial evaluation of aba content on iq gains in children with autism. journal of behavioral education, 30(3), 455-477. https://doi.org/10.1007/s10864019-09344-7 dixon, m. r., paliliunas, d., weber, j., & schmick, a. m. (2022). a large-scale naturalistic evaluation of the aim curriculum in a public-school setting. behavior analysis in practice, 15(1), 156-170. https://doi.org/10.1007/s40617-021-00569-5 dixon, m. r., rowsey, k. e., gunnarsson, k. f., belisle, j., stanley, c. r., & daar, j. h. (2017). normative sample of the peak relational training system: generalization module with comparison to individuals with autism. journal of behavioral education, 26(1), 101-122. https://doi.org/10.1007/ s10864-016-9261-4 dowdy, a., jessel, j., saini, v., & peltier, c. (2022). structured visual analysis of single-case experimental design data: developments and technological advancements. journal of applied behavior analysis, 55(2), 451-462. https:// doi.org/10.1002/jaba.899 dubuque, e. m., & kazemi, e. (2022). an investigation of bcba exam pass rates as a quality indicator of applied behavior analysis training programs. behavior analysis in practice, 15(3), 909-923. https://doi.org/10.1007/s40617-021-00660-x duncan, o. d. (1969). some linear models for two-wave, two-variable panel analysis. psychological bulletin, 72, 177-182. https://doi.org/10.1037/ h0027876 287 improving the methodological, analytical, and cultural impact of behavior analysis / dixon, yi, chastain & matthews dunn, k. e., sigmon, s. c., thomas, c. s., heil, s. h., & higgins, s. t. (2008). voucher-based contingent reinforcement of smoking abstinence among methadone-maintained patients: a pilot study. journal of applied behavior analysis, 41(4), 527538. https://doi.org/10.1901/jaba.2008.41-527 fontenot, b., uwayo, m., avendano, s. m., & ross, d. (2019). a descriptive analysis of applied behavior analysis research with economically disadvantaged children. behavior analysis in practice, 12(4), 782-794. https://doi.org/10.1007/ s40617-019-00389-8 gatzunis, k. s., edwards, k. y., rodriguez diaz, a., conners, b. m., & weiss, m. j. (2022). cultural responsiveness framework in bcba® supervision. behavior analysis in practice, 15(4), 1373-1382. https://doi.org/10.1007/s40617-022-00688-7 gollub, l. r. (2002). between the waves: harvard pigeon lab 1955–1960. journal of the experimental analysis of behavior, 77(3), 319-326. https://doi. org/10.1901/jeab.2002.77-319 gottman, j. m., mcfall, r. m., & barnett, j. t. (1969). design and analysis of research using time series. psychological bulletin, 72, 299-306. https:// doi.org/10.1037/h0028021 habib, r., & dixon, m. r. (2010). neurobehavioral evidence for the “near-miss” effect in pathological gamblers. journal of the experimental analysis of behavior, 93(3), 313-328. https://doi.org/10.1901/jeab.2010.93-313 heil, s. h., johnson, m. w., higgins, s. t., & bickel, w. k. (2006). delay discounting in currently using and currently abstinent cocaine-dependent outpatients and non-drug-using matched controls. addictive behaviors, 31(7), 1290-1294. https://doi.org/10.1016/j.addbeh.2005.09.005 higgins, s. t., budney, a. j., bickel, w. k., foerg, f. e., donham, r., & badger, g. j. (1994). incentives improve outcome in outpatient behavioral treatment of cocaine dependence. archives of general psychiatry, 51(7), 568-576. https://doi. org/10.1001/archpsyc.1994.03950070060011 higgins, s. t., delaney, d. d., budney, a. j., bickel, w. k., hughes, j. r., foerg, f., & fenwick, j. w. (1991). a behavioral approach to achieving initial cocaine abstinence. american journal of psychiatry, 148(9), 1218-1224. https://doi. org/10.1176/ajp.148.9.1218 higgins, s. t., kurti, a. n., & davis, d. r. (2019). coucherbased contingency management is efficacious but underutilized in treating addictions. perspectives on behavior science, 42(3), 501524. https://doi.org/10.1007/s40614-019-00216-z horner, r. h., & sugai, g. (2015). school-wide pbis: an example of applied behavior analysis implemented at a scale of social importance. behavior analysis in practice, 8(1), 80-85. https:// doi.org/10.1007/s40617-015-0045-4 horner, r. h., sugai, g., smolkowski, k., eber, l., nakasato, j., todd, a. w., & esperanza, j. (2009). a randomized, wait-list controlled effectiveness trial assessing school-wide positive behavior support in elementary schools. journal of positive behavior interventions, 11(3), 133-144. https://doi.org/10.1177/1098300709332067 howell, d. c. (2012). statistical methods for psychology. cengage learning. jaramillo, c., & nohelty, k. (2022). guidance for behavior analysts in addressing racial implicit bias. behavior analysis in practice, 15(4), 11701183. https://doi.org/10.1007/s40617-021-00631-2 jang, j., dixon, d. r., tarbox, j., granpeesheh, d., kornack, j., & de nocker, y. (2012). randomized trial of an elearning program for training family members of children with autism in the principles and procedures of applied behavior analysis. research in autism spectrum disorders, 6(2), 852-856. joslyn, p. r., donaldson, j. m., austin, j. l., & vollmer, t. r. (2019). the good behavior game: a brief review. journal of applied behavior analysis, 52(3), 811815. https://doi.org/10.1002/jaba.572 keppel, g. (1982). design and analysis: a researcher's handbook (2nd ed.). prentice-hall. kirkham, p. (2017). “the line between intervention and abuse” – autism and applied behaviour analysis. history of the human sciences, 30(2), 107-126. https://doi.org/10.1177/0952695117702571 koegel, l. k., koegel, r. l., & smith, a. (1997). variables related to differences in standardized test outcomes for children with autism. journal of autism and developmental disorders, 27(3), 233243. https://doi.org/10.1023/a:1025894213424 kolb, r. l., robers, a. c., brown, c., & mccomas, j. j. (2022). beyond cultural responsivity: applied behavior analysis through a lens of cultural humility. in handbook of special education research (vol. i, pp. 144-157). routledge. 288 january 2023, volume 15, issue 3, 277-290 kornack, j., cernius, a., & persicke, a. (2019). the diversity is in the details: unintentional language discrimination in the practice of applied behavior analysis. behavior analysis in practice, 12(4), 879-886. https://doi.org/10.1007/s40617-01900377-y kupferstein, h. (2018). evidence of increased ptsd symptoms in autistics exposed to applied behavior analysis. advances in autism, 4(1), 1929. https://doi.org/10.1108/aia-08-2017-0016 kurtz, p. f., fodstad, j. c., huete, j. m., & hagopian, l. p. (2013). caregiverand staff-conducted functional analysis outcomes: a summary of 52 cases. journal of applied behavior analysis, 46(4), 738-749. https://doi.org/10.1002/jaba.87 kyonka, e. g. e., mitchell, s. h., & bizo, l. a. (2019). beyond inference by eye: statistical and graphing practices in jeab, 1992-2017. journal of the experimental analysis of behavior, 111(2), 155-165. https://doi.org/10.1002/jeab.509 ledgerwood, d. m., alessi, s. m., hanson, t., godley, m. d., & petry, n. m. (2008). contingency management for attendance to group substance abuse treatment administered by clinicians in community clinics. journal of applied behavior analysis, 41(4), 517-526. https:// doi.org/10.1901/jaba.2008.41-517 lenoir, c., hinman, j. m., yi, z., & dixon, m. r. (2022). further examination of the children’s psychological flexibility questionnaire (cpfq): convergent validity and age appropriateness. advances in neurodevelopmental disorders, 6(2), 224-233. https://doi.org/10.1007/s41252-02200259-5 levy, s., siebold, a., vaidya, j., truchon, m.-m., dettmering, j., & mittelman, c. (2022). a look in the mirror: how the field of behavior analysis can become anti-racist. behavior analysis in practice, 15(4), 1112-1125. https://doi.org/10.1007/ s40617-021-00630-3 li, a., gravina, n., pritchard, j. k., & poling, a. (2019). the gender pay gap for behavior analysis faculty. behavior analysis in practice, 12(4), 743-746. https://doi.org/10.1007/s40617-019-00347-4 lovelace, t. s., comis, m. p., tabb, j. m., & oshokoya, o. e. (2022). missing from the narrative: a sevendecade scoping review of the inclusion of black autistic women and girls in autism research. behavior analysis in practice, 15(4), 1093-1105. https://doi.org/10.1007/s40617-021-00654-9 mathur, s. k., & rodriguez, k. a. (2022). cultural responsiveness curriculum for behavior analysts: a meaningful step toward social justice. behavior analysis in practice, 15(4), 10231031. https://doi.org/10.1007/s40617-021-00579-3 matson, j. l., & konst, m. j. (2014). early intervention for autism: who provides treatment and in what settings. research in autism spectrum disorders, 8(11), 1585-1590. https://doi.org/10.1016/j. rasd.2014.08.007 may, b. k., & st. cyr, j. (2021). the impact of the peak curriculum on standardized measures of intelligence: a systems level randomized control trial. advances in neurodevelopmental disorders, 5, 245-255. https://doi.org/10.1007/ s41252-021-00199-6 mayer, g., sulzer‐azaroff, b., & wallace, m. d. (2019). behavior analysis for lasting change. sloan publishing. mcgill, o., & robinson, a. (2021). “recalling hidden harms”: autistic experiences of childhood applied behavioural analysis (aba). advances in autism, 7(4), 269-282. https://doi.org/10.1108/ aia-04-2020-0025 morris, c., goetz, d. b., & gabriele-black, k. (2021). the treatment of lgbtq+ individuals in behavioranalytic publications: a historical review. behavior analysis in practice, 14(4), 1179-1190. https://doi.org/10.1007/s40617-020-00546-4 nighbor, t. d., zvorsky, i., kurti, a. n., skelly, j. m., bickel, w. k., reed, d. d., naudé, g. p., & higgins, s. t. (2019). examining interrelationships between the cigarette purchase task and delay discounting among pregnant women. journal of the experimental analysis of behavior, 111(3), 405-415. https://doi.org/10.1002/jeab.499 nunnally, j. c., & bernstein, i. h. (1994). psychometric theory (3rd ed.). mcgraw-hill. perone, m. (1999). statistical inference in behavior analysis: experimental control is better. the behavior analyst, 22(2), 109-116. https://doi. org/10.1007/bf03391988 plantiveau, c., dounavi, k., & virués-ortega, j. (2018). high levels of burnout among early-career board-certified behavior analysts with low collegial support in the work environment. european journal of behavior analysis, 19(2), 195-207. https://doi.org/10.1080/15021149.2018.143 8339 289 improving the methodological, analytical, and cultural impact of behavior analysis / dixon, yi, chastain & matthews pritchett, m., ala’i-rosales, s., cruz, a. r., & cihon, t. m. (2022). social justice is the spirit and aim of an applied science of human behavior: moving from colonial to participatory research practices. behavior analysis in practice, 15(4), 1074-1092. https://doi.org/10.1007/s40617-02100591-7 sanders, k., staubitz, j., juárez, a. p., marler, s., browning, w., mcdonnell, e., altstein, l., macklin, e. a., & warren, z. (2020). addressing challenging behavior during hospitalizations for children with autism: a pilot applied behavior analysis randomized controlled trial. autism research, 13(7), 1072-1078. schulz, k. f., altman, d. g., moher, d., & the, c. g. (2010). consort 2010 statement: updated guidelines for reporting parallel group randomised trials. trials, 11(1), 32. https://doi.org/10.1186/1745-621511-32 sevon, m. a. (2022). schooling while black: analyzing the racial school discipline crisis for behavior analyst. behavior analysis in practice, 15(4), 12471253. https://doi.org/10.1007/s40617-022-00695-8 shea, p., johnson, p., & togade, d. (2022). using relational frame theory to examine racial prejudice: a tool for educators and an appeal for future research. behavior analysis in practice. https:// doi.org/10.1007/s40617-022-00767-9 sidman, m. (1960). tactics of scientific research: evaluating experimental data in psychology. basic books. silverman, k., wong, c. j., needham, m., diemer, k. n., knealing, t., crone-todd, d., ... & kolodner, k. (2007). a randomized trial of employmentbased reinforcement of cocaine abstinence in injection drug users. journal of applied behavior analysis, 40(3), 387-410. skinner, b. f. (1956). a case history in scientific method. american psychologist, 11, 221-233. https://doi. org/10.1037/h0047662 sutton, a., pikula, a., yi, z., & dixon, m. r. (2022). evaluating the convergent validity of the peak comprehensive assessment (pca): intelligence, behavior challenges, and autism symptom severity. journal of developmental and physical disabilities, 34(4), 549-570. https://doi. org/10.1007/s10882-021-09814-9 thrailkill, e. a., desarno, m., & higgins, s. t. (2022). intersections between environmental reward availability, loss aversion, and delay discounting as potential risk factors for cigarette smoking and other substance use. preventive medicine, 165, 107270. https://doi.org/10.1016/j. ypmed.2022.107270 twohig, m. p., shoenberger, d., & hayes, s. c. (2007). a preliminary investigation of acceptance and commitment therapy as a treatment for marijuana dependence in adults. journal of applied behavior analysis, 40(4), 619-632. https:// doi.org/10.1901/jaba.2007.619-632 united states of america department of defense. (2021). the department of defence comprehensive autism care demonstration auunal report 2021. https://health.mil/reference-center/ reports/2021/12/03/annual-report-on-autismcare-demonstration-program-for-fy-21 waasdorp, t. e., bradshaw, c. p., & leaf, p. j. (2012). the impact of schoolwide positive behavioral interventions and supports on bullying and peer rejection: a randomized controlled effectiveness trial. archives of pediatrics & adolescent medicine, 166(2), 149-156. https://doi. org/10.1001/archpediatrics.2011.755 wang, y., kang, s., ramirez, j., & tarbox, j. (2019). multilingual diversity in the field of applied behavior analysis and autism: a brief review and discussion of future directions. behavior analysis in practice, 12(4), 795-804. https://doi. org/10.1007/s40617-019-00382-1 world health organization. (2020). doing what matters in time of stress: an illustrated guide. https://www.who.int/publications/i/ item/9789240003927 wright, p. i. (2019). cultural humility in the practice of applied behavior analysis. behavior analysis in practice, 12(4), 805-809. https://doi.org/10.1007/ s40617-019-00343-8 yi, z., & dixon, m. r. (2021). developing and enhancing adherence to a telehealth aba parent training curriculum for caregivers of children with autism. behavior analysis in practice, 14(1), 58-74. https://doi.org/10.1007/s40617-020-00464-5 290 january 2023, volume 15, issue 3, 277-290 yi, z., koenig, j., & dixon, m. r. (2022). comparing low dosages of aba treatment on children’s treatment gains and school readiness. advances in neurodevelopmental disorders. https://doi.org/10.1007/s41252-022-00296-0 yi, z., schreiber, j. b., paliliunas, d., barron, b. f., & dixon, m. r. (2021). p < .05 is in the eye of the beholder: a response to beaujean and farmer (2020). journal of behavioral education, 30, 489-511. https://doi.org/10.1007/s10864-021-09435-4 yoon, j. h., higgins, s. t., heil, s. h., sugarbaker, r. j., thomas, c. s., & badger, g. j. (2007). delay discounting predicts postpartum relapse to cigarette smoking among pregnant women. experimental and clinical psychopharmacology, 15, 176-186. https://doi. org/10.1037/1064-1297.15.2.186 zarcone, j., brodhead, m., & tarbox, j. (2019). beyond a call to action: an introduction to the special issue on diversity and equity in the practice of behavior analysis. behavior analysis in practice, 12(4), 741-742. https://doi.org/10.1007/s40617-01900390-1 © 2022 published by kura education & publishing. this is an open access article under the cc by ncnd license. (https://creativecommons.org/ licenses/by/4.0/) 209 copyright © www.iejee.com issn: 1307-9298 international electronic journal of elementary education january 2023, volume 15, issue 3, 209-211 excessively positive narratives diminish autistic people of all ages nicholas liua,* abstract introduction evolving discourse about autistic individuals swims in murky territory, with ongoing debates over how the autistic community should represent itself and how the neurotypical population should engage with them. one tendency that has emerged is the depiction of autistic individuals as kind and well-intentioned but also simpleminded and guileless. this, in combination with the dominant imagery of autistic individuals as children, has created an atmosphere in which all autistic individuals are stripped of their human complexity by the neurotypical gaze, and by extension are not given the fulfillment, support, and resources needed to make possible a life of sufficient quality. current discourse about autism, which is often centered around children and inspirational stories, either excludes adults or infantilizes autistic people of all ages. for all the diversity of people on the autism spectrum, the overall message in popular media and the testimonial pages of many awareness-raising nonprofit groups (autism speaks, n.d.) is relentlessly formulaic: the protagonist suffers from debilitating symptoms, but with the right support system they eventually overcome many of their difficulties and make progress towards adjusting to society’s general expectation. this is a positive formula i abided by in my own personal retellings. materially, i had all the trappings of an “autism success story;" thanks largely to early detection and effective behavioral intervention based on the principles of applied behavior analysis (aba), i have now far surpassed the original prognosis that i would only have a 50/50 chance of eventually attending a “mainstream” k-12 school. instead of banging my head against the wall in frustration, i mutter profanities like a self-conscious new yorker; rather than hiding in the corner to play with the carpet at parties, i gingerly find contrived excuses to approach strangers, like the time i placed my drink on an occupied table because “i needed both hands to check something on my phone.” i am willing to compromise with friends and housemates, even if keywords: autism, autistic adults, narratives, applied behavior analysis received : 8 january 2023 revised : 23 january 2023 accepted : 8 march 2023 doi : 10.26822/iejee.2023.293 a,* correspondance details: nicholas liu, new york university, usa. e-mail: nicholasliu1536@gmail.com 210 january 2023, volume 15, issue 3, 209-211 that means sacrificing my principle of never letting a single morsel of food go to waste, no matter how far past the expiration date it has been lingering. life in general is a pleasant, even fulfilling, experience. because i surpassed my original prognosis so drastically, advocacy and media organizations have invited me to share my insight to parents, educators, and doctors hoping to advance the cause of supporting autistic people and promote aba as a humane and effective treatment, at least when applied correctly. i believed in my narrative of hard-won achievement, encapsulated in one 2016 seminar by its rather selfcongratulatory title: “my path from autism to the university of chicago.” but for all my waxing about how “tenacity” and “hope” helped me overcome autism’s most debilitating symptoms, i said little about the self-defeating, mean-spirited, and less traditionally virtuous tendencies i might still carry as an adult. this was not an intentional cover-up, at least consciously; my sense of worth hinged on my overcoming autism, and so i had convinced myself that it was overcome completely. by whitewashing our accounts of "overcoming" autism, we stifle discourse and awareness about the most severe problems and character flaws we possessed and may still possess in adulthood. when this happens, both autistic children and adults are deprived of the support and resources they need to live healthy and fulfilling lives. in my stories, i would mention to audiences my social anxiety, my compulsion to prove my intelligence to others, and my recurrent misunderstanding of non-verbal cues. but these are vague, generalized points that do not shed light on how problems borne from autism or from the experience of autism can actively complicate my quality of life and the strength of my character—or, in other words, make me human. i still agonize over the smallest faux pas i might have committed more than ten years ago, scrutinizing mental lists of witnesses who may or may not still remember and associate the incident with their opinion towards me. and yet, i am often quick to make moral and intellectual judgments of others, as if to convince myself that i am superior, somehow, over at least some people. but at least my judgement is less expressive and indiscriminate than it was before; in middle school, i comforted myself by verbally bullying classmates who i deemed to be “even lower” on the social hierarchy, including other neurodivergent peers. autism has made me both more empathetic and more selfish. i want to do what is kind and just because i know what it is like to experience cruelty and injustice; at the same time, i am so insecure of my social position and reputation that i sometimes step over others to aggrandize myself. to exclude nuance is to promote a false and problematic narrative of autistic people, with serious consequences. a workplace cannot accommodate an autistic adult, and a school cannot assist an autistic child, if they misunderstand who they seek to help— any attempt would only result in frustration, even trauma. the problem of nuance, or lack thereof, is not wholly the fault of me or a few others who might have once told a few varnished stories. the cause is also institutional. in popular media depictions, the characteristics projected on autistic people often appear to be the diametric opposites of negative characteristics that used to define the stereotypical autistic person. while autistic people are now less likely to be pigeonholed as crass, insensitive robots, they are increasingly viewed as innocent, endearingly eccentric victims who deserve the same protection and loving condescension as a gurgling infant. even when they do not embody such traits, paternalistic expectations for them to conform in that manner and shield them from the world strips them of their agency and creates self-fulfilling prophesies of dependency. this problem is reinforced by the high tendency of popular media and charitable organizations to depict autistic children rather than adults (stevenson et al., 2011), which causes their consumers to envision autism as an adolescent issue above all else, and autistic adults as perpetual children. rather than being harassed for flapping their left-hand mid-sentence, one autistic adult i know now receives comically infantilizing comments from peers of the same age. another autistic adult graduates from high school like most other people do, and a photo of him in his robes is posted on a subreddit for “cute and cuddly pictures” alongside all the cats and dogs and plump, beady-eyed bunnies. i may no longer have this issue in my day-to-day life, but that is only because i learned to conceal my autistic tendencies after incidents like having to explain why i do not need a special helper to follow me around in public at age 17. innocence and endearing eccentricity are not negative qualities by themselves, and representing autistic children is important. but when these things are used to reduce autistic people to a quarter-dimension of normal human complexity and intelligence, it creates a demoralizing, belittling experience for all of them. there are many autistic people who suffer from more explicit and brutal forms of hardship, discrimination, and oppression, especially if they come from lowerincome families, are nonwhite (mandell et al., 2009), and cannot access resources like reliable, correctly practiced aba-based therapy. sometimes, people like 13-year-old linden cameron or 3-year-old katie mccarron must even pay a blood price (fortin, 2020). poorly-conceived depictions of autism exacerbate these problems when much-needed attention is drawn away from their plight and redirected 211 excessively positive narratives diminish autistic people of all ages / liu towards shinier, more wholesome narratives. by overpromoting these kinds of narratives, we construct an idealization that encompasses everyone across the autism spectrum; and at the point in which we are no longer in touch with diverse and often harsh realities, we are fighting not for people, but for empty air. references autism speaks (n.d.). stories from the community. https://www.autismspeaks.org/profile?field_ profile_type_target_id=all&page=1 fortin, j. (2020, september 9). a police officer shot a 13-year-old with autism in salt lake city. the new york times. https://www.nytimes. com/2020/09/08/us/salt-lake-city-autismshooting.html mandell, d. s., wiggins, l. d., carpenter, l. a., daniels, j., diguiseppi, c., durkin, m. s., giarelli, e., morrier, m. j., nicholas, j. s., pinto-martin, j. a., shattuck, p. t., thomas, k. c., yeargin-allsopp, m., & kirby, r. s. (2009). racial/ethnic disparities in the identification of children with autism spectrum disorders. american journal of public health, 99(3), 493-498. https://doi.org/10.2105/ ajph.2007.131243 stevenson, j. l., harp, b., & gernsbacher, m. a. (2011). infantilizing autism. disability studies quarterly, 31(3). https://doi.org/10.18061/dsq.v31i3.1675 microsoft word 11_iejee_4_1_alshumaimeri international electronic journal of elementary education, 2011, 4(1), 185-195. issn:1307-9298 copyright © iejee www.iejee.com the effects of reading method on the comprehension performance of saudi efl students yousif alshumaimeri ∗∗∗∗ king saud university, saudi arabia abstract in this study i investigated the relative effects of different reading methods on the comprehension performance of saudi efl 10th grade male students. the scores of participants who read three comparable passages in three ways (oral, silent and subvocalizing) were compared. results revealed a significant difference between oral reading and subvocalization, and between oral reading and silent reading. oral reading had the greatest effect on comprehension performance among the three reading methods examined. all groups reported that oral reading was the most preferred reading method with the majority of respondents feeling the style best supported comprehension. feedback suggested that oral reading was preferred specifically because it helps in memorizing words and texts, concentration, and practicing and pronouncing words for real world encounters. it is recommended that second language teachers and students use all available reading methods in order to identify which method best serves their study objectives. keywords: oral reading, silent reading, subvocalization, reading comprehension, methods of reading, efl, saudi arabia introduction reading ability has always been viewed as critical to academic success (bernhardt, 1991; carrell, 1991; grabe & stoller, 2002; urquhart & weir, 1998). researchers investigating reading have attempted to look for components that affect reading performance as well as reading behaviors, such as oral reading, that distinguish proficient from less-proficient readers. oral reading is often viewed as a dated methodology and discouraged by efl/esl teachers (amer, 1997). while some researchers hold the opinion that oral reading is a way of wasting class time (hill & dobbyn, 1979), other scholars (cho & choi, 2008; gibson, 2008; rennie, 2000; reutzel, hollingsworth, & eldredge, 1994; white, 1982) point to potential benefits that can be gained from various oral reading techniques that allow for oral proofreading, pronunciation practice, and conversational fluency. ∗ yousif alshumaimeri,college of education, king saud university, riyadh, saudi arabia. e-mail: yousif@ksu.edu.sa international electronic journal of elementary education 186 for decades, investigators have emphasized the importance of oral reading to children in first language teaching situations not only as a means of encouraging children to read, but also of improving their reading comprehension (alshumaimeri, 2005; grabe, 1991; jackson & coltheart, 2001; juel & holmes, 1981; mccallum, sharp, bell, & george, 2004; prior & welling, 2001; rowell, 1976). according to al-qurashi, watson, hafseth, hickman, & pond (1995), in second language learning situations oral reading is the best way to teach pronunciation and word recognition during the early stages of second/foreign language acquisition, but reading comprehension is better strengthened by reading silently. reading silently has traditionally been viewed as the only way to train pupils to read on their own (al-qurashi et al., 1995). the underlying principle governing this viewpoint is that reading is normally a solitary activity best done in total silence without interruption for best concentration (alqurashi et al., 1995). while researchers continue to explore the effectiveness of oral reading on both language acquisition and comprehension, many questions remain unanswered. research on first language learning indicates that people often comprehend better after reading silently (bernhardt, 1983; leinhardt, zigmond, & cooley, 1981; wilkinson & anderson, 1995). however, other studies (teng, 2009) suggest comprehension scores do not differ significantly between silent and oral reading. further research on the relationship between reading methods and reading comprehension is needed in order to enhance efl teaching methodologies and to improve learning outcomes. this research furthers understanding of the relationship between reading method and comprehension. as such, results will benefit educational institutions and the efl researchers, teachers and students that support them. oral vs. silent reading methods reading is a crucial skill in learning and communication. current trends in education consider reading lessons to be an important early step in the development of mental and linguistic abilities. reading methods include reading silently, reading using subvocalization (forming the sounds of the words while reading silently), and reading orally to oneself. reading silently means reading without labial movements or the vibration of vocal cords. this method implies that graphic forms are visually perceived and then transformed into meanings and ideas without passing through the vocal stage. silent reading is usually seen as natural reading behavior and for decades has been associated with the idea of reading for comprehension. as reviewed by rennie (2000), academic work on reading pedagogy in the first half of the 20th century described the advantages, disadvantages and processes associated with both oral and silent reading (chall, 1967; russell, 1949). although russell (1949) found that in some places there was a system of reading called 'non-oral' which did not include oral reading instruction at any point in a child's reading development, most scholars agreed by the mid-twentieth century that both oral and silent reading activities were necessary for effective reading instruction. although the importance of oral reading to children learning a native language is widely accepted, the effectiveness of oral reading in second language classrooms continues to be debated. in her study of oral reading practices in the classroom, gibson (2008) found that teachers and learners were using oral reading in a variety of ways. the primary reasons for using the method were for practicing pronunciation and intonation. other reasons included for speaking practice, making graphemic-phonemic connections, diagnosing pronunciation problems, improving fluency and practicing reading skills. in the case of second language learning, gibson (2008) also found that 82% of autonomous learners read orally to themselves as part of private study. asian learners, in particular, commented that oral reading was especially important to them for practicing pronunciation. the effects of reading method on the comprehension / alshumaimeri 187 oral reading and comprehension hannon and daneman (2001) proposed four primary processes in reading comprehension: accessing relevant knowledge from long-term memory, integrating accessed knowledge with information from the text, making inferences based on information in the text, and recalling newly learned text material. in schema theory, a predominant theory of reading comprehension, reading comprehension is viewed as the process of interpreting new information and assimilating this information into memory structures (anderson & pearson, 1984; teng, 2009). as suggested by teng (2009), differences in native languages can affect second language (l2) oral reading for efl learners. reading in a second language requires more cognitive capacity for word identification than reading in one’s native language (l1). slower readers must employ greater cognitive resources than good readers in order to accomplish word recognition. taguchi and gorsuch (2002) found that while l1 readers tend to focus more on content words, l2 readers focus equally on content words and grammatical function words. as such, the limited cognitive capacity l2 readers allocate to word recognition tasks may impair their comprehension. in her study of efl students, amer (1997) states that oral reading by the teacher helps readers discover units of meaning that arise from multi-word phrases rather than meaning that is derived from individual words. oral reading also helps readers to see text as a whole with various levels of meaning rather than as a dissectible passage of graphic cues. amer suggests that, with appropriate practice, students will gradually begin to realize that a higher level of comprehension can be achieved by reading larger meaningful units of texts. oral reading performed by the teachers can additionally reinforce correct understanding of punctuation and intonation further strengthening student comprehension. in researching the relevance of oral reading fluency to reading comprehension, saieghhaddad (2003) conducted a study with 22 arabic and 28 hebrew native speakers, 19-25 years old, enrolled in intermediate efl courses. by analyzing participant’s oral reading skill with two texts, one in the participant’s native language and one in english, the researchers aimed to determine if there was a difference in the relationship between oral reading skill and reading comprehension. although there was no relationship found between oral reading fluency and reading comprehension in either arabic or hebrew reading (saiegh-haddad, 2003), in english, participants with oral reading fluency were found to have better reading comprehension. possible explanations for the above finding can be found in a study conducted by miller and smith (1985). conducting a study on comprehension after reading orally and silently, miller and smith (1985) tested 94 second to fifth graders who read either at a low level, medium level, or high level. the results suggest that poor readers are better at comprehending when reading orally as compared to reading silently, and are more adept at answering inferential questions than they are at answering literal questions (miller & smith, 1985). average readers in miller and smith’s (1985) study read silently more proficiently than poor readers and were able to answer inferential and literal questions equally well. good readers were found to be proficient at both oral and silent reading and best able to answer literal questions (miller & smith, 1985). the results of miller and smith’s (1985) study suggest not only that literal comprehension is the best indicator of reading competence, but also that poor readers do benefit from the use of oral reading in the classroom. in taiwan, teng (2009) studied the relationship between reading comprehension and reading methods and learning styles of efl 12th grade male students. teng (2009) found that most students can benefit from both silent and oral reading activities. being that some international electronic journal of elementary education 188 students in the study benefited more from oral reading than others, teng suggests that efl teachers be more flexible in selecting various reading methods for use in the classroom setting. teachers could support a mixture of oral and silent reading assignments that would allow students to engage in their preferred style. second language readers often read slowly and have under-developed oral production when compared to native speakers. oral reading practice was found by taguchi and gorsuch (2002) to be more effective than other reading methods at increasing reading speed and comprehension among beginning l2 readers. however, taguchi and gorsuch (2002) were doubtful that oral reading of passages can be effective for older l2 readers as they read orally less often and may be less comfortable with the method. oral reading, culture and environment the viewpoint that oral reading has limited benefit to learners does not take into account the differing social and cultural backgrounds of students. as mentioned previously, there is an enduring opinion that readers who read silently comprehend the most because they both read and think. however, this view implies that students are incapable of thinking while reading orally. it might be more accurate to say that a student’s attenuation to the social environment in which she is reading would have a greater bearing on her ability to concentrate than her inability to do both at once. alshumaimeri (2005) argues that oral reading is not necessarily a faulty reading method as suggested by nuttall (1996, but, rather, is an effective aid to comprehension. in a study conducted by alshumaimeri, oral reading was found to be used not only for decoding and relating written symbols to sounds, but also for comprehension. the criteria for effective reading comprehension included familiarity or comfortableness with the reading method, which aided the reader’s speed of comprehension. some informants in alshumaimeri’s study stated they would read orally when they were studying, which requires concentration, memorization, and comprehension, and would read silently when they read for enjoyment. furthering the body of work on l2 reading methods, this study investigates the effects of different reading methods on l2 student reading comprehension. the research questions are as follows: do different reading methods affect the comprehension of saudi students; which reading methods affect reading comprehension; and which reading methods do saudi students prefer and why? methods research design this research employs a classroom-based, quasi-experimental design in order to examine the effects of different reading methods on the comprehension performance of saudi students. in educational research, a quasi-experiment is more commonly used due to fixed school schedules and logistical problems (cohen, manion, & morrison, 2007). the different reading methods studied were oral, subvocalization, and silent reading. comprehension performance was determined from the students’ comprehension scores on multiple-choice tests. in order to minimize the effects of repetition, three different passages were selected from mccall-crabbs standard test lessons in reading, book d (1979). each group read each passage using one of the three reading methods (oral, subvocalization, and silent). the study participants always read passage 1 first, and then passages 2 and 3. however, in order to counterbalance the design of the study, the order of the reading method was rotated. for example, group 1 read passage 1 orally, passage 2 using subvocalization, and passage 3 silently, while group 2 read passage 1 using the effects of reading method on the comprehension / alshumaimeri 189 subvocalization, passage 2 silently, and passage 3 orally. in this way, the effect of passage difficulty or type of passage reading was minimized with regard to measuring the reading comprehension performance of the study students. each group was located in a different room during testing. the noise level during oral reading was not perceived as a distraction to comprehension as students read softly and the testing rooms were large. after reading each passage and taking the reading test for that passage, the students were asked to fill out a feedback slip asking about their preferred reading method and the reason behind their preference. participants participants in the study were 145 saudi male students with an average age of 16 years, in the first year of the secondary stage (10th grade) in a secondary school in riyadh, saudi arabia. like most saudi students they had studied english for four years, since the 6th grade in elementary school. participating students were expected to be fairly representative of the target population of saudi learners in terms of ability, interest, and age. however, one should acknowledge the limitation of drawing students from one school in riyadh. the participating students were distributed by the school management into four classes with the intention that each class should be a balanced mixed-ability class. the students’ level of language proficiency was considered to be a1 level of the european framework. passages three expository passages were selected from mccall-crabbs standard test lessons in reading, book d (1979). each passage was followed by five questions, posed in english, with four-option multiple choice answers. multiple choice questions are perhaps the most commonly used format in standardized reading comprehension tests. the procedure’s advantages lie in the simplicity of its scoring (koda, 2005). passage one, a school charity day, contains 141 words and describes a fundraising bazaar held at a children’s school. the second passage, the best way to lose weight, contains 139 words and provides advice on how to lose weight. the third passage, a carpenter story, contains 108 words and describes how one person became a carpenter. a reliability analysis was computed for each test using test/retest method (cohen, manion, & morrison, 2007). the reliability results were (pearson coefficient) 0.7462 for the first passage, 0.6715 for the second passage, and 0.6605 for the third passage. reliability was deemed sufficient given that the test only contained five items. feedback slips the feedback slip was a small piece of paper that was given to each student after completing each reading test (three feedback slips were collected per participant). it included three questions that asked students to write down (in l1) if the reading method they used supported their comprehension, to rank which reading methods they generally prefer, and to explain their choices. the purpose of the feedback slips was to help in understanding the effects of the different reading methods and to know which reading methods students prefer in everyday life. the number of responses collected was 227 out of 435 feedback slips distributed with a return rate of 52.2%. the low rate of return is believed to be because the slips were distributed after each test. some students returned the feedback slips blank because they had answered the question on the first slip and did not change their views. the slips were distributed after each test in order to provide equal opportunity for students to reflect on each reading method. international electronic journal of elementary education 190 procedure the research was conducted on a regular school day during the extra-curricular activity time (the last two periods of one day per week). the available time for testing was 110 minutes. each reading test was allocated 20 minutes followed by 5-7 minutes for filling out the feedback slips. the students were randomly assigned to their group. as described above, each group read a passage using each of the three different reading methods. all students were told to read the reading instructions carefully and to ask for clarification if needed. there were three teachers, one for each group, who helped administer the tests and explained the procedure clearly. needed materials were prepared beforehand and placed in envelopes according to the study design. the researcher supervised the administration by moving from one room to another to check that the procedures were followed according to plan and to answer any questions. the data collected consisted of the comprehension scores obtained from the five multiple-choice questions designed for each of the three passages as well as the data collected from the feedback slips. the data analysis was conducted in accordance with the research questions, all of which were concerned with comprehension performance as measured by the scores from the multiple-choice questions, the dependent variable. a oneway analysis of variance (anova) was used to test the differences between the reading methods and a post-hoc analysis using the scheffe test was conducted to locate the source of differences. then, two-way anova was used to test the differences between the groups with different reading methods. the study results are reported below. results the results obtained are presented in accordance with the research questions, beginning with the first research question. in order to answer the first research question (do different reading methods affect the comprehension of saudi students?) a one-way analysis of variance (anova) test was conducted with the post-hoc scheffe test. there was a significant difference at level 0.01 between the saudi students in comprehension performance according to the reading method. a significant difference was found between oral reading and subvocalization (mean difference 1.92, p < 0.01), and between oral reading and silent reading (mean difference 2.32, p < 0.01). the largest mean occurred for oral reading (9.65), which had the greatest effect on comprehension performance among the three reading methods included in the study. to answer the second research question, the data obtained from the feedback slips show the students’ responses to the question (does this reading method assist you in understanding this passage?). the results show that 57% of the students thought that oral reading helped them better comprehend the passage; whereas 26.2% and 17.9%, respectively, thought silent reading and subvocalization helped them understand the passage. the reading method that had the greatest positive effect on comprehension was oral reading with a mean value 9.65. subvocalization and silent reading had mean values of 7.72 and 7.33, respectively. these results indicate that oral reading helped students better understand passages. to answer the third research question (which reading methods do saudi students prefer and why?), the results obtained from the feedback slips show the ranked order of the preferred reading style of each group as well as an explanation of their choice of order. of all groups, 50.57% of students reported that oral reading was the most preferred reading method. subvocalization was ranked second with 22.76%, whereas silent reading was third with 14.02%. the effects of reading method on the comprehension / alshumaimeri 191 data obtained from the feedback slips is summarized in tables 1a, 1b, and 1c and indicate the reason the students preferred each method of reading. the rate of return (52.2%) of the feedback slips could indicate that the reason for preferring a reading method is static and that the learners felt they did not need to provide the same feedback after each passage. the results are presented according to each reading method. table 1a: the students’ responses for reasons for preferring oral reading no. students justification frequency percentage 1 it helps in memorizing and remembering new words 26 20.63% 2 i use this way for studying as it helps me understand and memorize the text 25 19.84% 3 it makes me concentrate more and understand the text 20 15.9% 4 in reading aloud it helps me understand more as i use three senses (sight, hearing, and speech) 15 11.9% 5 it helps in pronunciation practice and pronouncing the words better 13 10.32% 6 it makes the words more familiar and helps memorizing them and using them in conversations with others 12 9.52% table 1a: the students’ responses for reasons for preferring oral reading no. students justification frequency percentage 7 i read faster and understand more in reading aloud 8 6.34% 8 it helps in conversation and practice talking in a foreign language 7 5.55% total 126 100% table 1b: the students’ responses for reasons for preferring silent reading no. students justification frequency percentage 1 i read silently for leisure not for study 11 29.73% 2 it helps me understand and concentrate more 10 27.03% 3 i read faster and understand more 6 16.22% 4 i don’t like annoying other people when i read aloud 5 13.51% 5 i feel more relaxed when i read silently 5 13.51% total 37 100% table 1c: the students’ responses for reasons for preferring subvocalization no. students justification frequency percentage 1 it makes me concentrate more and understand the text 15 23.43% 2 i use this way for studying 15 23.43% 3 i do not annoy other people and concentrate than reading silently 11 17.2% 4 it helps in memorizing and remembering new words 10 15.63% 5 it helps in pronunciation practice and pronouncing the words better 7 10.94% 6 i read faster than reading aloud and keep my concentration 6 9.37% total 64 100% international electronic journal of elementary education 192 table 1a shows that just over 40% of the respondents preferred oral reading because it was perceived as aiding in memorizing and remembering new words. using oral reading as a means to improve conversational english or improve pronunciation was not a common justification among respondents. the usefulness of oral reading in improving conversational english was the least cited justification with only 5.5% of respondents choosing it as their primary reason for preferring oral reading. as seen in table 1b, nearly 30% of respondents indicated that they read silently for leisure, but not for study in justifying their preference for silent reading. such a result indicates that students associate silent reading with leisure reading and oral reading with language studies. many students, 27%, also indicated that understanding and concentration were heightened when reading silently. anxiety about annoying others and greater relaxation while reading silently were less cited reasons for preferring silent reading (13.5% and 13.5%, respectively). respondents with a preference for subvocalization cited the justifications of enhanced concentration and preferred method of studying (23.4% and 23.4%, respectively) as seen in table 1c. such results indicate subvocalization is a study habit perceived as enhancing students’ ability to concentrate on text while studying. as in the results presented in table 1a summarizing the justifications for preferring oral reading, pronunciation was not an often cited reason for employing subvocalization as a reading method (10.9%). a one-way anova with post-hoc scheffe test was used to explore the relationship between the preferred reading method and the comprehension performance of the students. a significant relationship was found between the preferred reading method and the students’ comprehension performance, (f = 5.919, sig. = .001). a significant difference was found in favor of the students who prefer oral reading over reading using subvocalization or silent reading (mean value 8.96). the results suggest that there is a relationship between the selected reading method and the comprehension performance of the saudi students. to evaluate if there is a significant difference between groups with regard to comprehension performance regardless of reading method, a two-way anova was conducted. the results indicate three things: the differences between the groups, regardless of reading methods, were not significant; the differences based on reading methods were significant; and the interaction effects between groups and reading methods were significant. discussion in summary, the study of the relative effects of different reading methods on the comprehension performance of saudi efl 10th grade students shows that there is a significant difference between the saudi students in comprehension performance according to reading method. these results support the literature suggesting oral reading can be a beneficial reading method when used in the l2 classroom (cho & choi 2008; gibson, 2008; rennie, 2000; reutzel, hollingsworth, & eldredge, 1994; white, 1982). if reading comprehension can be defined as the process of interpreting new information and assimilating this information into memory structures as schema theory suggests, this study indicates that oral reading aids comprehension by improving students’ ability to concentrate and memorize new words. in this study there was a significant difference between oral reading and subvocalization in regard to comprehension and between oral reading and silent reading. oral reading had the greatest positive effect on comprehension performance among the three reading the effects of reading method on the comprehension / alshumaimeri 193 methods included in the study. as found in the literature, students exposed to oral reading techniques in l2 classrooms report improved comprehension of reading material (amer, 1997; saiegh-haddad, 2003; warwick & mangubhai, 1983). although it should be reiterated that reading ability was not measured among the participants in this study, all participants were in the same grade and differences in reading ability would presumably have an insignificant impact on the study results. despite the negative opinion some scholars hold toward the use of oral reading in the language classroom (hill & dobbyn, 1979), this study demonstrates that oral reading can in fact assist some students in acquiring proficient language comprehension. oral reading, although often described as an effective method of learning for recognizing and pronouncing words with ease and fluency, was more often cited by the participants in this study as a means of strengthening memorization of new words and effective concentration as well as comprehension. the results of this study are in relation to student performance on a comprehension test and not in relation to language instruction or leisure reading. although many students indicated that understanding and concentration are heightened when reading silently, one third of respondents indicated that they read silently for leisure, but not for study. such a result indicates students associate silent reading with leisure reading and oral reading with language studies. this preference for silent leisure reading supports nuttall’s (1996) opinion that oral reading is uncommon outside the classroom. with regard to student preference for a particular reading style, all groups reported that oral reading was the most preferred reading method with subvocalization ranked second and silent reading third. these results suggest that most of the students hold the opinion that oral reading is an effective method for understanding the passages. in addition, participants indicated that oral reading was the preferred reading method not only because it helps in memorization and concentration, but it is also helpful for practicing and pronouncing words for real world encounters. this clear preference for oral reading for study purposes is partially due to traditional teaching methods, such as rote learning, that require learners to memorize information. respondents with a preference for subvocalization cited the justifications of enhanced concentration and preferred method of studying. such results indicate subvocalization is a study habit that is perceived as enhancing students’ ability to concentrate on text while studying. however, saudi students do not seem to support nuttall’s (1996) opinion that subvocalization is an ineffective reading method. participants in this study preferred subvocalization above silent reading as a reading method. if gibson (2008) and amer (1997) are correct in assuming that oral reading can be made a more effective learning device with greater systemization in the curriculum, the methods by which saudi teachers encourage and use oral reading in the classroom should be further researched. it is possible that the saudi scholastic environment provides an ideal setting for allowing oral reading methods to support significant gains in reading comprehension. this educational setting, coupled with a cultural appreciation for oral religious traditions and memorization through oral recitation, could foster strong tendencies among students to associate oral reading with concentration and memorization. as reported by alshuamimeri (2005), one interviewee reported feeling as if oral reading allowed for better understanding and concentration. further, the respondent suggested that arabic literature in particular is better appreciated and analyzed when read aloud. additionally, oral reading makes a strong impression because the reader hears as well as sees what is being read and the sense of hearing is effective in supporting comprehension. albar (1996) stated the capacity to learn a language is dependent on normal hearing more than international electronic journal of elementary education 194 any other trait suggesting that someone who reads aloud is more likely to understand what he is reading than someone who reads silently. conclusion in investigating the relative effects of different reading methods on the comprehension performance of saudi efl 10th grade students, the results showed that oral reading had the greatest positive effect on the comprehension performance of the study sample. this study also found that oral reading was the most preferred reading method. oral reading was perceived by students to aid in memorization and concentration. efl teachers can take from this finding that despite the relative inconsistencies in academic findings regarding the effect of reading method on comprehension, some students do in fact find oral reading to be beneficial in the l2 classroom. although such findings may be culturally or individually specific, greater flexibility in the design of second language teaching methodologies is warranted pending greater research on the subject. additionally, l2 students should use all available reading methods in order to identify which method best serves their study objectives. reading ability is acquired through practice, not through educational settings or teaching methods. when viewed as a continuum with beginners at one end and fluent readers at the other end, a student’s growing capacity is defined by his or her ability to rapidly understand and comprehend new lexicon and context. readers may find that while comprehension is not necessarily bolstered by practicing oral reading methods, memorization and concentration may be enhanced by employing oral reading techniques, either in the classroom or during private study. unfortunately, the arabic library is lacking material for children. making reading materials available from an early age should be prioritized in order to support a well rounded adult ability to comprehend written language. to further the findings of this study, additional research is needed on l2 learners of different ages and gender. • • • yousif alshumaimeri is an associate professor at the college of education, king saud university, saudi arabia. he is head of the tesol team at the department of curriculum and instruction and coordinator of the ma tesol program. his main research interests are in teaching reading skills and the use of pedagogic tasks. references albar, m. a. (1996). human development. saudi publishing & distributing house, jeddah, saudi arabia. al-qurashi, k., watson, m., hafseth, j., hickman, d,. & pond, r. (1995). ‘english for saudi arabia: first year secondary, teacher’s book’ .ministry of education press, riyad, saudi arabia. alshumaimeri, y. (2005). is reading aloud a faulty habit. paper presented at the second international conference: language, culture and literature, minia university, egypt. amer, a. (1997). the effect of the teacher's reading aloud on the reading comprehension of efl students. elt journal, 51(1), 43-47. anderson, r.c. & pearson, p.d. (1984). a schema-theoretic view of basic processes in reading comprehension. in p.d. pearson (ed.), handbook of reading research (pp. 255-292). new york: longman. bernhardt (1983). three approaches to reading comprehension in intermediate german. the modern language journal, 67, 111-115. bernhardt, e. b. (1991). reading development in a second language: theoretical, empirical, and classroom perspectives. norwood, nj: ablex. the effects of reading method on the comprehension / alshumaimeri 195 carrell, p. l. (1991). second language reading: reading ability or language proficiency? applied linguistics, 12(2), 159-79. chall, j. (1967). learning to read: the great debate, mcgraw-hill, new york. cho, k.s. & choi, d.s. (2008). are read-alouds and free reading “natural partners”? busan national university of education research institute, 36(5), 69-73. cohen, l., manion, l., & morrison, k. (2007). research methods in education (sixth ed.). us: new york: routledge. gibson, s. (2008). reading aloud: a useful learning tool? elt journal, 62(1), 29-36. grabe, w. (1991). current developments in second language reading research. tesol quarterly, 25(3), 375-406. grabe, w. & stoller, f. l. (2002). teaching and researching reading. london: longman. hannon, b., & daneman, m. (2001). a new tool for measuring and understanding individual differences in the component processes of reading comprehension. journal of educational psychology, 93(1), 103-128. hill, l. and dobbyn, m. (1979). a teacher training course for teachers of efl. london, uk: cassell. jackson, n. & coltheart, m. (2001). routes to reading success and failure. hove, uk: psychology press. juel, c. & holmes, b. (1981). oral and silent reading of sentences. reading research quarterly, 16(4), 545-568. koda, k. (2005). insights into second language reading: a cross-linguistics approach. cambridge, uk: cambridge university press. leinhardt, g., zigmond, n., & cooley, w. w. (1981). reading instruction and its effects. american educational research journal, 18, 343-361. mccall, w. & crabbs, l. (1979). standard test lessons in reading, book d. new york, us: teachers college, columbia university, mccallum, r. s., sharp, s., bell, s. m., & george, t. (2004). silent versus oral reading comprehension and efficiency. psychology in the schools, 41(2), 241-246. miller, s. d., & smith, d. e. p. (1985). differences in literal and inferential comprehension after reading orally and silently. journal of educational psychology, 77(3), 341-348. nuttall, c. (1996). teaching reading skills in a foreign language. uk: heinemann. prior, s. m., & welling, k. a. (2001). "read in your head": a vygotskian analysis of the transition from oral to silent reading. reading psychology, 22(1), 1-15. rennie, j. (2000). teaching reading: oral reading practices as a pedagogical tool in the primary school. the australian journal of language and literacy, 23(3), 197-211. reutzel, d. r., hollingsworth, p. m. & eldredge, j. l. (1994). oral reading instruction: the impact on student reading development. reading research quarterly, 29(1), 40-62. rowell, e.h. (1976). do elementary students read better orally or silently? the reading teacher journal, 29(4), 367-370. russell, d. h. (1949). children learn to read. boston: ginn and company. saiegh-haddad, e. (2003). bilingual oral reading fluency and reading comprehension: the case of arabic/hebrew (li) -english (l2) readers. reading and writing: an interdisciplinary journal, 16(8), 717-736. taguchi, e., & gorsuch, g. j. (2002). transfer effects of repeated efl reading on reading new passages: a preliminary investigation. reading in a foreign language, 14(1), 43-65. teng, y. (2009). the relationship of reading methods and learning styles to taiwanese 12th grade male students’ reading comprehension in english. a dissertation presented to the faculty of the school of education international and multicultural education department. the university of san francisco. urquhart, s. & weir, c. (1998). reading in a second language: process, product and practice. london: longman. warwick, b. e. & mangubhai, f. (1983). the impact of reading on second language learning. reading research quarterly, 19 (1), 53-67. white (1982). the english teacher’s handbook. walton on thames: nelson. wilkinson, i. a. g., & anderson, r. c. (1995). sociocognitive processes in guided silent reading: a microanalysis of small-group lessons. reading research quarterly, 30(4), 710-740. international electronic journal of elementary education, december 2016, 9(2), 433-450. issn:1307-9298 copyright © iejee www.iejee.com tutorial: teaching verbal behavior to children with asd einar t. ingvarsson a * a university of north texas and child study center, usa received: august, 2016 / revised: september, 2016 / accepted: october, 2016 abstract early and intensive behavioral intervention has been shown to result in favorable outcomes for children with autism spectrum disorder. procedures and practices based on and influenced by b. f. skinner’s verbal behavior (vb) have been increasingly integrated into eibi curricula in recent years. in this article, i give an overview of some basic tenets of vb as they pertain to behavioral interventions for children with asd, with a special emphasis on the relevance of basic behavioral principles to verbal operants. additionally, i provide a few examples of practical recommendations derived from vb. keywords: autism spectrum disorder, early and intensive behavioral intervention, verbal behavior introduction early and intensive behavioral intervention (eibi) has become a widespread treatment approach for children with autism spectrum disorder. several outcome studies, metaanalyses, and reviews of the literature support the effectiveness of eibi as an intervention for this population (eldevik et al., 2009; 2010; klintwall, eldevik, & eikeseth, 2015; lovaas, 1987). autism is generally believed to be a disorder of the brain and genetic factors are thought to play a crucial role in its etiology (rutter & thapar, 2014; volkmar & mcpartland, 2014). however, environmental experiences can compensate for deficits presumably caused by brain structure or function (thompson, 2007). this is especially true of experiences that occur early in life, due to the considerable neural plasticity that characterizes early brain development. thus, the systematic, comprehensive, and intensive environmental input characteristic of eibi seems to positively affect the developmental course of many children with asd. eibi is based on the science of applied behavior analysis (aba), and eibi is frequently referred to as aba therapy or treatment. aba is the applied “branch” of the science of behavior analysis, rooted largely in the work of b. f. skinner (1938; 1953). behavior * address for correspondence: einar t. ingvarsson, phd, bcba-d, 1155 union circle, department of behavior analysis, university of north texas, denton, texas 76201. phone: 940-565-2274. e-mail: einar.ingvarsson@unt.edu http://www.iejee.com/ international electronic journal of elementary education vol.9, issue 2, 433-450, december, 2016 434 analysts study environment-behavior relations in a broad sense, and decades of laboratory work have led to the development of general principles of behavior. aba researchers and practitioners apply these general principles to behavior of social importance (baer, wolf, & risley, 1968). behavioral scientists in the skinnerian tradition have sometimes been criticized for failing to account for complex human behavior, such as language, cognition, and social interactions (e.g., chomsky, 1959). however, several behavior analysts have studied complex behavior-environment relations extensively (e.g., hayes, barnes-holmes, & roche, 2001; horne & lowe, 1996; sidman, 1994). although he didn’t conduct much research on human behavior, skinner published several books and papers in which he interpreted complex behavior in terms of basic behavioral principles established in the laboratory. in the book verbal behavior (vb), published in 1957, skinner provided a conceptual interpretation of language and communication. in addition, he proposed new terms (e.g., mands, tacts, intraverbals) to categorize verbal behavior based on its function. skinner did not present any research on verbal behavior in the book. this may be part of the reason why its impact was somewhat limited in the years and decades after it was published (dymond, o’hora, whelan, & o’donovan, 2006; mcpherson, bonem, green, & osborne, 1984). however, the impact of skinner’s vb on research and practice has increased with time (love, carr, almason, & petursdottir, 2009; sautter & leblanc, 2006). the purpose of the current paper is to provide an overview of skinner’s analysis of verbal behavior as it pertains to behavioral interventions for individuals with autism spectrum disorder. because language and communication deficits are a defining feature of asd, skinner’s vb may be particularly important for this population. in addition to vb, the current discussion draws from sources that are consistent with, based on, and extend skinner’s work. this includes literature that may not have been directly influenced by skinner’s vb, but is nevertheless consistent with it (e.g., the literature on incidental teaching; hart & risley, 1978). i hope to outline the utility of vb, especially as it applies to early behavioral interventions for children with autism. this discussion assumes fundamental knowledge of basic principles of behavior such as reinforcement, stimulus control, and motivating operations, as well as common procedures (or classes of procedures) such as differential reinforcement, discrimination training, shaping, prompting, and fading. the definition of verbal behavior in the first chapter of verbal behavior, skinner defined verbal behavior as behavior whose reinforcement is mediated by another person. even though this definition points to a crucial defining feature of verbal behavior, it can be argued that it fails to distinguish between behaviors commonly recognized as verbal from others few would call verbal (e.g., the experimenter mediates the reinforcement for rat’s lever pressing in a skinner box). recognizing this, skinner added the caveat that the behavior of the listener (who mediates reinforcement for the verbal response) had to be specifically conditioned to reinforce particular speaker responses. thus, the listener reinforces responses that are consistent with the conventions of a verbal community. two aspects of skinner’s definition are especially noteworthy. first, it suggests that verbal behavior is neither special nor qualitatively different from behavior in general. thus, we should analyze verbal behavior like any other behavior, using concepts and principles derived from basic research. however, there may be additional concepts that are useful (see below). second, skinner’s analysis emphasizes response function rather than response form or topography. thus, the same response topography can have multiple functions, depending on the nature of the environmental variables that are operating in overview of verbal behavior / ingvarsson 435 each instance. conversely, different response topographies can have the same function. further, skinner’s analysis doesn’t differentiate between verbal responses in the form of spoken language, sign language, picture exchange, text messaging, gestures, or any other form, if the controlling variables are the same. verbal operants in vb, skinner (1957) proposed several verbal operants, which are concepts that categorize verbal behavior according to characteristic antecedents and consequences. essentially, the verbal operants are labels for categories of verbal responses, differentiated by the type of functional control involved. verbal operants may or may not occur in their pure form in everyday situations, but the as analytic tools they are nevertheless useful. for the current purposes, they provide a way to organize assessment, instruction, and intervention on language and communication. below, i provide brief definitions and examples of each verbal operant. mands. the mand is a verbal operant under the control of a motivating operation and reinforced by a characteristic reinforcer. one example of manding is asking for water when thirsty and receiving the water as a consequence. in this case, the thirst (or being deprived of fluids) is the motivating operation, and receiving the water is the characteristic reinforcer, specified by the response. mands can also specify escape from or avoidance of aversive stimulation. an example would be to ask a person to turn down music that is too loud. in this case, the loud music is the motivating operation, and the other person turning down the music is the characteristic consequence. thus, behavior that is referred to as “requesting” in everyday language is typically manding. tacts. the tact is a verbal operant under the stimulus control of a nonverbal stimulus and maintained by generalized (i.e., not specific) reinforcement. an example would be a child saying “cat” when a cat walks by, and the parent responds by saying, “that’s right!” the response is evoked by a nonverbal stimulus (the cat), and the reinforcer is generalized in the sense that it is not specified by the response. intraverbals. like tacts, intraverbal responses are also maintained by generalized reinforcement, which often takes the form of attention or other reinforcers inherent in social interactions and conversation. unlike tacts, intraverbal responses are evoked by other verbal stimuli. examples of intraverbal responses are answers to questions or completing a sentence that another person starts. an example is an adult asking, “how old are you?” and a child answering “i’m eight years old.” echoics. echoic behavior is functionally similar to intraverbal behavior, with one important difference. by definition, an echoic response has a point-to-point correspondence and formal similarity to the verbal stimulus that precedes it. an example is when a child says “cat” when an adult says “cat”. echoics can be viewed as a special case of intraverbal behavior (vargas, 1986), but are typically considered as its own category due to the imitative nature of the response. as we will see below, the point-to-point correspondence and formal similarity that characterize echoic behavior have important implications for behavioral interventions. autoclitics. autoclitic behavior refers to verbal behavior that modifies the effects of other verbal behavior. for instance, somebody might say “i think the party is tonight”, or they might say “i’m sure the party is tonight.” the sentence parts “i think” and “i’m sure” will likely influence the behavior of the listener in different ways. the listener might seek further confirmation before going to the party in the first case, but not in the second case. relatively little research has been conducted on autoclitic functions. due to the introductory nature of this article, i will not discuss autoclitics further. international electronic journal of elementary education vol.9, issue 2, 433-450, december, 2016 436 speaker and listener behavior the behavior of the speaker is considered verbal if it’s reinforced by a listener in accordance to the conventions of a verbal community. as defined above, all the verbal operants are examples of speaker behavior. however, the behavior of the listener is also crucial for a behavioral analysis of language and communication. listener behavior is any behavior that is under the control of verbal behavior of the speaker and mediates the reinforcement for that behavior. common examples are following instructions and identifying objects in the environment. for example, the speaker might ask the listener to hand him or her an object that is out of reach of the speaker. this mand is reinforced if the listener complies. in turn, the behavior of the listener might be reinforced through the speaker’s approval. in an educational situation, a teacher (the speaker) might ask a student (the listener) to point to a particular letter of the alphabet when presented with an array of letters a worksheet. this mand is reinforced through the student’s compliance with the teacher’s instruction. the students’ behavior is likely reinforced via the teachers’ approval and through successfully completing the assignment. these kinds of tasks are often referred to as receptive identification or receptive labeling in the context of eibi. based on strict interpretation of vb, only speaker behavior is considered verbal. accordingly, behaviors such as following instructions and receptive identification would not be considered verbal. despite this distinction, listener behavior is frequently included as an essential component of verbal behavior interventions and research on verbal behavior. schlinger (2008) argued that listener behavior that participates in verbal episodes should be treated as verbal behavior. in accordance with this argument, i will discuss listener behavior along with speaker behavior where applicable. verbal operants: conceptual strengths and weaknesses verbal operants can be used to organize language assessment and intervention in accordance to behavioral function. some of the ways in which verbal operants help with these tasks will be discussed below. however, it is not sufficient to simply label behavior as a mand, tact, or intraverbal. the labels do not take the place of an analysis of behavior in terms of basic behavioral principles (stimulus control, reinforcement, motivating operations). using the labels can give the illusion of functional analysis, but important functional variables might not have been identified. thus, it is important to avoid viewing vb as special approach that is separate from or superior to aba and eibi. rather, it offers additional tools and insights that may be useful in in the practice and science of behavior analysis as it pertains to verbal behavior. any behavior analyst making use of vb concepts should be fluent in basic behavioral principles. further, multiple instructional methods and procedures are relevant to teaching verbal behavior, as is the case for eibi in general (e.g., discrete trial instruction, incidental teaching, direct instruction, prompting and prompt fading, shaping, chaining, differential reinforcement, discrimination training, etc.). in the following section, i will discuss some examples of basic behavioral principles as they apply to skinner’s verbal operants. basic behavioral principles and verbal behavior motivating operations. motivating operations (mos) are environmental variables that influence behavior by altering the reinforcing or punishing value of other environmental events. mos can also directly evoke behavior because of a history of altering the value of the consequences for that behavior (laraway, snycerski, michael, & poling, 2003). mands are defined in terms of control by mos. therefore, it is important to ensure appropriate mo control during mand training (michael, 1993). in early intervention, therapists can either capture or contrive mos. both procedures rely upon identifying effective reinforcers and controlling access to them, and then delivering the reinforcers contingent on a specific overview of verbal behavior / ingvarsson 437 mand or an approximation to the mand. in everyday situations, mos can be captured by following the child’s lead and observing initiations towards preferred items or activities (for instance, pointing at or approaching toys). the therapist then acts to control access to the preferred items or activity, waits for the child to make an initiation response, and uses the opportunity to teach. the reinforcer (i.e., the item the child initiated towards) should then be delivered contingent on the target response or an approximation. if the target response doesn’t occur, the therapist can teach using prompting and prompt fading (e.g., constant prompt delay) or shaping. contriving mos works the same way, except that the therapist identifies preferred items and activities ahead of time and sets up the environment so that the child must emit a mand to access the item or activity. this can be a way to increase the number of mand training opportunities relative to capturing mos. for instance, the therapist might place highly preferred toys out of the child’s reach and wait for the child to make an initiation response towards the toys. mos can also be captured during preferred activities. the therapist might observe that a child enjoys when an adult pushes him or her on a swing. after pushing the child a few times, the therapist pauses, waits for a response, and continues pushing the swing contingent on the response. another approach to contriving an mo has been labelled the interrupted chain strategy (carter & grunsell, 2001). the client must complete a chain of responses to reach a terminal reinforcer, but the therapist manipulates the chain so that the client cannot complete it without emitting a mand. for instance, the child might prefer playing a video game, but the controller might be missing, so he or she must mand for it. one major benefit of capturing or contriving an mo is that teaching occurs at a moment when the client is highly motivated, therefore increasing the likelihood that a response will occur. additionally, responding will directly benefit the speaker at that moment in time, because it brings her in immediate contact with a highly preferred item or activity. mand training that involves capturing or contriving mos is similar to a well-established aba procedure called incidental teaching (hart & risley, 1978). however, although many instances of mand training qualify as incidental teaching, not all incidental or naturalistic teaching is necessarily mand training (i.e., it can involve other kinds of behavior). motivating operations are most relevant to mand training, because mands are defined in terms of mos. however, mos are certainly relevant to other verbal operants, as well as early intervention in general. the strength of motivating operations can affect stimulus control and generalization (lotfizadeh, edwards, redner, & poling, 2012). further, social motivation, or the extent to which social interactions function as reinforcers, is likely to affect generalization and maintenance of intraverbals, tacts, and listener responses in natural environments. we will return to this issue in the section on reinforcement below. stimulus control. stimulus control refers to the extent to which specific stimuli or stimulus compounds evoke behavior due to being correlated with the availability of reinforcement. thus, if a behavior is reinforced in the presence of a stimulus and reinforcement is withheld in its absence, the behavior will be more likely to occur in the presence of that stimulus in the future. the stimulus will come to serve as discriminative stimulus (sd) for that behavior (cooper, heron, & heward, 2007). ensuring appropriate stimulus control by verbal and nonverbal stimuli is a crucial issue in teaching verbal behavior. we will consider each category in turn. nonverbal stimulus control. control by nonverbal stimuli is involved in tacts and listener responses. with tacts, the response should be under the control of a nonverbal stimulus such as an object or action. one can also tact qualities and characteristics of objects such as color and shape, categories, abstract concepts such as emotions, and even private international electronic journal of elementary education vol.9, issue 2, 433-450, december, 2016 438 events such as pain. one everyday example of tacting is a tour guide who points out and names novel landmarks to visitors. in an eibi program a therapist might show a client a picture of a cow, and the child responds, “cow”. in both examples the controlling variables are broadly the same, in that verbal responses are under the control of nonverbal stimuli and maintained via generalized reinforcement not specified by the response (e.g., attention, approval, tokens, etc.). however, these situations are clearly different in one important respect: in the former case the speaker who emits the tact (the tour guide) is attempting to educate the visitors (most of whom probably don’t know the names of the landmarks), but in the latter case the teaching trial is a part of a program in which the speaker who emits the tact (the client) is being educated. it is useful to keep in mind that in naturalistic situations, such as when a speaker is showing a listener a novel object or location, tacts are emitted for the benefit of the listener. in educational situations, such as those involved in eibi, children are typically taught a variety of tacts that are judged to be important. however, the educational context does not necessarily share many similarities with naturalistic situations in which tacts are likely to occur, so generalization and maintenance of tacts may be limited in the absence of careful programming. establishment of a tact repertoire is an important building block in language acquisition. a strong tact repertoire that is under appropriate stimulus control provides one of the foundations of language comprehension, as that term is commonly understood. thus, it is important to ensure that tacts occur under appropriate stimulus control. research has suggested that individuals with autism may be more likely to display unwanted or restricted stimulus control (sometimes referred to as stimulus overselectivity; lovaas, koegel, & schreibman, 1979). a child may learn to tact an object shown in a picture, but when other exemplars of the object are presented, the response does not generalize. when further test trials are conducted to isolate various components of the original stimulus, it is revealed that the child learned to tact an irrelevant part rather than the essential features of the object (of course, the part is irrelevant from our perspective, not the child’s). the extent to which this occurs as a function of the disorder or due to inappropriate or insufficient teaching is unknown. in either case, behavior analysts can reduce the probability of restricted stimulus control and increase the likelihood of desired stimulus control by presenting multiple exemplars of stimuli from the very beginning, in which essential components, qualities, and features of objects remain constant, but all other components, qualities, and features can vary. certain types of listener responses, often referred to as receptive identification or receptive labeling, also occur under nonverbal stimulus control. however, these responses are under conditional stimulus control of both verbal and nonverbal stimuli. the verbal stimuli specify which nonverbal stimulus will serve as a discriminative stimulus (sd) for a listener response, and often take the form of an instruction. for instance, an adult might ask a child, “please hand me the potatoes”, while seated at the dinner table. in an eibi program, a therapist might present several cards depicting different shapes and ask the child to “point to the circle”. in these kinds of tasks, the instructions are often referred to as samples and the items as comparisons. the response requires the child to select the correct comparison out of an array. the verbal instruction functions as a conditional stimulus, because it determines the function of the items (i.e., which item functions as a discriminative stimulus on each occasion). these kinds of discriminations are often referred to as auditory-visual discriminations, because auditory and visual stimuli interact to produce the response. when teaching these kinds of responses, it is essential to ensure control by both the auditory (sample) and visual (comparison) stimuli. to achieve that, it is necessary to present multiple samples (instructions) in a random sequence and randomize the positions of the comparisons. grow, carr, kodak, jostad, and kisamore (2011) evaluated procedures to teach auditory-visual discriminations with children with overview of verbal behavior / ingvarsson 439 asd, and green (2001) has made important recommendations for instruction in this skill area. these types of listener responses are particularly important in language instruction, because they form some of the building blocks of understanding or language comprehension (schoneberger, 1991). verbal stimulus control. as noted above, verbal stimulus control occurs in intraverbal behavior. the most straightforward example of intraverbal behavior is when the responses are directly evoked by a verbal stimulus. an example is when an adult asks a child “what is your name”, and the child immediately answers, “mary”. for children who can vocally imitate words, this kind of response can be brought under proper stimulus control using common prompting and prompt-fading procedures (e.g., goldsmith, leblanc, & sautter, 2007; ingvarsson & hollobaugh, 2011; ingvarsson & le, 2011). the child may then be able to easily discriminate between questions such as “what is your name” and “how old are you”. however, finer discriminations can cause difficulties, such as, “how are you”, vs. “how old are you”, or “what is your mother’s name”, vs. “what is your father’s name” (sundberg & sundberg, 2011). failures to acquire these relatively subtle discriminations may reflect lack of stimulus control by all relevant components of the questions. for instance, the response “four” may be under the control of the word “you” only, rather than “how old are you”. when asked “how are you”, the child answers “four”. when multiple components of a verbal antecedents interact or combine to control an intraverbal response, this can be conceptualized either as conditional stimulus control (axe, 2008; sundberg & sundberg, 2011) or convergent stimulus control by stimulus compounds (eikeseth and smith, 2013). in either case, it is important to arrange teaching so that stimulus control by multiple components of the verbal antecedent is ensured. examples of procedures to establish appropriate stimulus control by complex sentences have been provided by ingvarsson, kramer, carp, petursdottir, and macias (2016), kisamore, karsten, and mann (2016), and braam and poling (1983). palmer (1991) pointed out that some answers to questions and other responses that on the surface look like intraverbal responses may instead be a result of a more extended process. when a response is not immediately available to us (e.g., we do not remember it), we are likely to engage in additional behavior that enables us to respond. skinner (1953) referred to this additional behavior as problem solving. problem solving can involve overt behavior and response products, or it may include covertly talking to oneself and visually imagining (i.e., the behavior we call “thinking” in everyday language). for instance, a person might arrive at the grocery story only to discover that the shopping list did not make the trip. most people would probably attempt to recall which items they have or have not used up, what they have noticed is needed in the past few days, and might even attempt to visually imagine the current state of their refrigerator. something similar might happen when we are asked a question that we don’t immediately know the answer to, such as “what is 1890 divided by 4?” or “what year did you last see radiohead?” in these examples, a chain of responses occurs between the verbal antecedent (the question) and the response (the answer). the controlling variables are therefore substantially different from intraverbal responses that are directly evoked by antecedent stimuli. in early behavioral intervention for children with autism, it may be necessary to specifically teach clients to engage in problem solving strategies when asked questions they don’t know the answer to. sautter, leblanc, jay, goldsmith, and carr (2011) and kisamore, carr, and leblanc (2011) evaluated procedures to teach typically developing preschool children to engage in problem solving strategies (self-prompting and visual imaging) when instructed to list members of a category (e.g., “tell me some kitchen items”). this is a case in which a verbal antecedent sets the occasion for multiple intraverbal responses (michael, palmer, & sundberg, 2011). international electronic journal of elementary education vol.9, issue 2, 433-450, december, 2016 440 echoic behavior is also defined in terms of verbal stimulus control. as noted above, echoic behavior has a point-to-point and formal correspondence to the preceding verbal stimulus. in eibi, echoic behavior is often referred to as verbal imitation. some of the research on the effectiveness of eibi has suggested that acquisition of verbal imitation is among the predictors of good intervention outcomes (sallows and graupner, 2005). perhaps this is because echoic behavior can provide the building blocks for other verbal behavior. if the child can imitate verbal responses, the task of bringing them under other kinds of control (intraverbal, tact, mand) becomes easier and learning from the everyday environment is more likely. further, establishing fluent echoic behavior ensures that the response topography is at strength in the client’s repertoire prior to attempting to teach other verbal operants (eikeseth & smith, 2013). echoic prompting and transfer-ofstimulus control procedures can then be used to bring the response topography under the desired stimulus control. for instance, when teaching the intraverbal response “moo”, the therapist can start by presenting the question, “what does a cow say?”, and then immediately state the vocal (echoic) prompt, “moo”. when the child reliably echoes, the prompt is gradually faded until the response occurs under the control of the question. eventually, it is necessary to intersperse other similar intraverbals (e.g., “what does a dog say”) to ensure appropriate stimulus control. the importance of distinguishing between “meaningful” versus “meaningless” or “rote” intraverbal behavior is worth emphasizing. for skinner, the meaning of verbal responses was to be found in their controlling variables (andery, micheletto, & serio, 2005). if the controlling variables are purely verbal (i.e., echoic prompts used to establish the intraverbal stimulus control), independent observers would probably consider the response rote and meaningless. an illustrative example is a second language learner who is taught, via echoic prompting, to emit various foreign language responses to common foreign language phrases. while the verbal stimulus control might be perfect, there is no guarantee that the responses are related to the corresponding native language verbal responses or to relevant nonverbal stimuli. thus, the person might be able to say the foreign language equivalent of “two” when asked (in the foreign language), “what is 1+1?” however, because the relevant nonverbal stimulus control has not been established, the person would not be able to emit that response under other conditions (e.g., tacting quantity when shown two items). therefore, as a rule, it is important to ensure that the intraverbal response topography is also taught as a tact or a listener response. for instance, if a child is taught to answer the question “what does a cow say?” the child should also be able to point to and/or tact a cow and should be able to identify the correct animal when asked “point to the animal that says moo” (ingvarsson, cammilleri, & macias, 2012). reinforcement. operant reinforcement refers to an increase in the frequency of behavior that occurs because the behavior was followed by a specific consequence (a reinforcer) in the past (catania, 1998). the behavior operates on the environment to produce the reinforcer (hence the term operant). some important considerations are suggested when reflecting on the nature of reinforcement maintaining verbal behavior. one issue concerns contrived versus natural reinforcement. contrived reinforcers are those that are introduced by the therapist or teacher to increase and maintain specific behavior, but would be unlikely to follow that behavior in the absence of the contrived contingency. for example, pointing at and tacting a novel object in the environment might be reinforced by the delivery of a token (a contrived reinforcer) in an eibi program, but in the natural environment such responses would likely be reinforced through social interactions with conversation partners (natural reinforcers). contrived reinforcers are often necessary in behavioral interventions because natural consequences (particularly those inherent in social interactions) may not be effective reinforcers for children with asd. however, overview of verbal behavior / ingvarsson 441 verbal behavior will not maintain in everyday environments unless the behavior contacts effective naturally occurring reinforcement. for children with asd, these naturally occurring reinforcers are relatively more likely to occur for mands than for other verbal operants, because mands specify reinforcers that are valuable for the individual at a given moment in time. thus, mands for preferred items and activities are relatively likely to maintain and generalize across environments if the relevant motivating operations continue to occur, reinforcement for the mands is sufficiently frequent, and the individual has acquired a robust and varied mand repertoire. however, for other verbal operants, establishing social interactions and the social behavior of others as reinforcers is likely to be crucial. listener behavior, tacts, and intraverbal responses are generally maintained by some aspects of social interactions with others and are unlikely to maintain in their absence. lack of effective social reinforcers seems to be a characteristic of autism, and some scholars have suggested that lack of social motivation may be the primary underlying feature of asd (chevallier, kohls, troiani, brodkin, & schultz 2012). it is clear that some social interactions (e.g., praise, tickles) can function as reinforcers for children with asd (kelly, roscoe, hanley, schlichenmeyer, 2014). however, the range of social stimuli that function as reinforcers may be restricted, and subtler social cues (e.g., facial expressions, nods, gestures) may not function as effective reinforcers to the same extent that occurs for those without autism. due to the apparent importance of social reinforcement in eibi, treatment manuals have typically included recommendations to establish social stimuli as conditioned reinforcers. conditioned reinforcers achieve their reinforcing value through associations with other reinforcers (e.g., primary reinforces such as food; williams, 1994). treatment manuals for eibi have included recommendations for pairing social interactions (such as praise statements) with the delivery of already effective reinforcers (such as edibles; anderson, taras, & cannon, 1996; leaf & mceachin, 1999). in the most common scenario, the therapist waits for a specific response from the client (e.g., a correct response in a teaching program) and when the response occurs, delivers the social stimulus (e.g., praise) followed immediately by an already effective reinforcer (e.g., food). this procedure has been described as response-contingent pairing (dozier, iwata, thomason-sassi, worsdell, & wilson, 2012; lepper & petursdottir, in press). relatively little research has focused on the effectiveness of this approach in early behavioral intervention for children with autism. another procedure that has been the target of increased research effort is to establish social stimuli as discriminative stimuli (sds) signaling the availability of already effective reinforcers, such as toys and edibles (lepper, petursdottir, & esch, 2013; lovaas et al., 1966; holth, vandbakk, finstad, grønnerud, & sørensen, 2009; isaksen & holth, 2009). therapists can establish social interactions as sds by presenting trials in which social interaction is present (sd trials) and not present (s-delta trials) in a random sequence. during both types of trials, preferred items (e.g., toys or edibles) are within the reach of the child, but the child is only allowed to access the reinforcers when social interaction is in place. the goal is to teach the child to observe the therapist and only reach for the reinforcers in the presence of specific stimuli, in this case social stimuli. while therapist social interactions may be established as a reinforcer through this procedure, it is likely that the therapist must continue to pair himor herself with preferred items and activities to maintain the value of the conditioned reinforcers. more applied research is needed, but this procedure seems to hold some promise as an approach to establish a wider range of reinforcers, social and otherwise, for individuals with asd. a third method to establish conditioned reinforcement is stimulus-stimulus pairing, which involves presenting a currently neutral stimulus (e.g., a speech sound) simultaneously or international electronic journal of elementary education vol.9, issue 2, 433-450, december, 2016 442 immediately prior to the delivery of a primary reinforcer (e.g., an edible item). no response is required from the child, other than orienting towards the therapist before the stimuli are presented. this procedure has been most frequently studied as a potential method to establish the reinforcing value of speech sounds. from a behavior analytic standpoint, it is likely that early language development includes both social and automatic reinforcement. with automatic reinforcement, producing speech sounds is reinforcing in and of itself (independent of socially mediated reinforcement) to the individual that emits the sounds. the conditioned reinforcing value is likely established via both operant and respondent processes that occur during adult-infant interactions. if a child with asd does not produce speech sounds to the extent seen in typically developing children, part of the solution may be to establish speech sounds as conditioned reinforcers via stimulusstimulus pairing. research on this procedure has yielded mixed results (schillingsburg, hollander, yosick, bowen, & muskat, 2015). however, it is possible that responsecontingent pairing (see above) may yield better results (lepper & petursdottir, in press). the ways in which conditioned reinforcement plays a potential role in verbal behavior are likely numerous. i will briefly describe one more case. in manding for information, a mand is reinforced by delivery of “information” (which often takes the form of verbal instructions, directions, or clarification) which in turn allow the individual to access the terminal reinforcer (lechago & low, 2015; sundberg, loeb, hale, & eigenheer, 2002). thus, the “information” reliably predicts the availability of reinforcement and takes on conditioned reinforcing function. for instance, if a child wants to watch a favorite movie but can’t find the tv remote control, the child might ask an adult where the remote is. the adult tells the child the location of the remote, and thus the child can find the remote and watch the movie. in this example, the mo is not having access to the favorite movie and the mand for information is the child asking where the remote is located. the information given by the adult predicts and signals access to the reinforcer (the movie), and will thus likely become a conditioned reinforcer. just as with other kinds of mand training, therapists can capture and contrive a variety of scenarios where the client must mand for information to access preferred items and activities. one common program (described by taylor and harris, 1995) involves placing a variety of items in front of the child and asking the child to tact the items. some of the items are known to the child while others are novel. when the child gets to the unknown items, the therapist prompts the child to ask “what’s that?” the therapist then answers the question, which in turn allows the child to complete the task. the therapist can then evaluate generalization to novel items and situations and whether the child acquires new tacts because of manding for information about novel items. novel and emergent verbal behavior. it’s safe to assume that a large proportion of everyday verbal behavior is not directly taught, but emerges as a function of previous learning and current stimulus conditions. critics from outside the field have sometimes assumed that behavior analysis lacks the tools to explain emergent behavior. however, there exist several theoretical approaches to study emergence from a behavioral perspective. a thorough treatment of this vital issue is beyond the scope of this paper, so a brief discussion of a few pivotal areas must suffice. functional independence and interdependence. an important aspect of skinner’s notion of verbal operants is their functional independence. thus, acquiring a particular topography as one verbal operant (e.g., a mand) will not automatically result in being able to emit the same topography under different conditions (e.g., those characteristic of the tact; lamarre & holland, 1985). the same is frequently true of listener and intraverbal relations (petursdottir, carr, lechago, & almason, 2008). thus, eibi service providers are well advised to assess and teach each response topography under all relevant environmental overview of verbal behavior / ingvarsson 443 conditions (e.g., as an echoic, mand, tact, and intraverbal). however, it is also the case that verbal operants are often interdependent (i.e., not independent). this is particularly true for advanced speakers. for instance, it may be enough for a verbally capable adult to acquire a word as a tact and it will occur as an intraverbal with no further training (e.g., dounavi, 2014). in early intervention for children with autism, it is advisable to conduct probes and assessments on a regular basis to evaluate functional independence and interdependence. further, behavior analysts may choose to conduct teaching programs in which multiple verbal operants are taught for each response topography (e.g., listener response, tact, and intraverbal). naming. according to the influential naming account (horne & lowe, 1996) the interaction of speaker and listener behavior in the same individual plays an important role in verbal development. a child is said to have acquired the naming capability when the acquisition of a listener response automatically results in the corresponding tact, and vice versa (miguel & petursdottir, 2009). this capability allows the child to acquire language at a rapid rate from naturally occurring interactions if adults and others in their environments provide frequent opportunities for learning (i.e., modeling the names of items and activities as they occur). however, the same may not be true of children with asd. it may be particularly important to evaluate the naming capability with this population and attempt to establish bi-directional listener and tact behavior by interspersing listener and tact teaching trials (greer & ross, 2008; sundberg & partington, 1998). recombinative generalization. one way in which complex novel verbal responses occur is through recombinative generalization, in which previously acquired elements (e.g., sounds, words, or short phrases) recombine to form novel responses under specific stimulus conditions (goldstein, 1983; suchowierska, 2006). for instance, an individual might learn to tact “red circle”, “yellow square”, and “blue triangle” in the presence of these specific color-shape combinations. if the person can then emit the correct phrase when presented with novel combinations, such as a red square, yellow triangle, blue circle, and so on, recombinative generalization has occurred. this approach is potentially efficient because many novel combinations can result from teaching a limited number of responses. to maximize instructional efficiency, the elements to be taught and tested can be arranged in a matrix (axe & sainato, 2010; frampton, wymer, hansen, & shillingsburg, 2016; kohler & malott, 2014; pauwels, ahearn, & cohen, 2015). in addition to offering a way to maximize instructional efficiency, recombinative generalization is attractive because it offers a potential explanation for the occurrence of novel combinations of previously learned response units in everyday behavior. derived relations. two influential theoretical accounts, stimulus equivalence (se) and relational frame theory (rft), have inspired considerable research that is relevant to verbal behavior (hayes et al., 2001; sidman, 1994). although these accounts differ in important ways, both offer theoretical frameworks to explain how novel stimulus relations can occur as a function of previously acquired relations. for instance, by learning the relation between a toy car and the spoken word “car” (a-b) and between a toy car and the printed word car (a-c), an individual will typically be able to relate the spoken word and the printed word without further training (b-c). thus, like recombinative generalization, these frameworks offer ways to increase the efficiency of learning as well as potential explanations for novel behavior. much of the research has, however, been limited to studying selection responses (e.g., pointing to or otherwise selecting stimuli) that are often quite arbitrary in the sense that the response topography is of little importance. this is adequate to study complex stimulus relations, but in verbal behavior, the response topography is usually important because it is related to environmental international electronic journal of elementary education vol.9, issue 2, 433-450, december, 2016 444 events in unique ways. nevertheless, there is little doubt that both se and rft can be integrated with skinner’s vb analysis (e.g., barnes-holmes, barnes-holmes, & cullinan, 2000). additional practical recommendations the preceding sections have included several practical recommendations, but i will conclude by offering some additional advice. some of these recommendations are specifically based on vb research, while other recommendations are more general in nature and apply to multiple skill areas targeted in eibi. three curriculum guides exist that are based on skinner’s vb, but also cover general aspects of eibi: teaching language to children with autism and other developmental disabilities (sundberg & partington, 1998), the assessment of basic language and learning skills-revised (ablls-r; partington, 2008), and the verbal behavior milestones assessment and placement program (vb-mapp; sundberg, 2008). several other highly influential eibi curriculum guides include a focus on verbal behavior but are not organized according to skinner’s verbal operants. these include: teaching individuals with developmental delays: basic intervention techniques (lovaas, 2003), a work in progress (leaf & mceachin, 1999), behavioral intervention for young children with autism (maurice, green, & luce, 1996), and making a difference: behavioral intervention for autism (maurice, green, & foxx, 2001). these documents provide resources for skills assessment and treatment goal selection. however, selection of treatment goals should also be informed by relevant developmental and ecological considerations (dyer & peck, 1987). although curricular sequence should be determined on an individual basis, general guidelines can be offered based on skinner’s verbal operants. an early goal should be to establish a functional mand repertoire. this is important for at least two reasons: first, as stated above, mands benefit the speaker because they are related to a current motivating operation. therefore, it is likely that reinforcement of communicative responses under these conditions will be especially effective. problem behavior often serves a communicative function in the sense that it is maintained by socially mediated reinforcement (hanley, iwata, & mccord, 2003). socially appropriate mands can replace problem behavior, and therefore mand training can form a part of a behavioral intervention plan. teaching an appropriate mand to replace problem behavior is often referred to as functional communication training (fct; carr & durand, 1985). note that in the early stages, the functional communication response may be general rather than specific (i.e., it does not specify a particular reinforcer). however, the responses can often be shaped into more specific and complex mands over time. for children who do not have an echoic repertoire it may be beneficial to use picture exchange communication (pecs) or signs to facilitate quick acquisition of communication responses. depending on their abilities and progress, some children may continue to use pecs (bondy & frost, 2001), signs, or speech generating devices (sgds), while with others, it may be feasible to gradually shape up vocal speech. it is sometimes possible to begin the process of shaping up speech by delaying access to reinforcement following non-vocal mands, prompt a vocal response, and then fade out the prompts over time (gevarter et al., 2015). when implementing behavioral interventions to increase vocal speech, the establishment of echoic behavior (generalized verbal imitation) should be an early priority. an echoic repertoire functions as a “minimal response repertoire”, which facilitates the acquisition of additional verbal operants (alessi, 1987; palmer, 2012). the desired verbal topography can be evoked as an echoic and then brought under the desired stimulus control. in other overview of verbal behavior / ingvarsson 445 words, therapists can take advantage of the client’s echoic repertoire to teach other verbal operants. in the early stages of intervention, it can be beneficial to combine echoic and mand training, thereby taking advantage of the current motivating operation to establish echoic behavior. this can be done in the context of incidental teaching (hart & risley, 1978), in which the therapist controls access to preferred items and activities and uses the child’s initiations towards the preferred items and activities as opportunities to teach. the child’s communicative responses are then reinforced by access to the specific preferred item or activity. with children who have no echoics, tacts, or mands, the program can start with the therapist reinforcing any vocal response. over time, the therapist starts delivering specific vocal models, and gradually shapes the child’s vocalizations. when the child can reliably echo sounds, the therapist can start shaping up words. when the child can successfully echo under these conditions, it may be possible to begin practicing and teaching vocal imitation in separate discrete trial sessions. when the child has acquired a robust echoic repertoire (i.e., he can vocally imitate novel words and phrases on the first opportunity), the therapist can take advantage of the echoic repertoire to teach new mands, tacts, and intraverbals as indicated for each client. as implied above, it is generally a good idea to establish echoic and mand repertoires prior to focusing on other verbal operants. when echoics and mands have been established, the next step would be to focus on listener and tact repertoires. in typical development, young children acquire listener responses (e.g., pointing, orienting towards stimuli) before tacts. however, it may not be most efficient to follow that sequence of programming in eibi. on the contrary, the relevant research literature suggests that tact training is somewhat more likely to lead to emergence of the corresponding listener responses than vice versa (petursdottir & carr, 2011). it can be argued that tacting cannot occur without some sort of listener response (at minimum, orienting towards the stimulus). therefore, listener responses may occur incidentally during tact trials and become members of a common response class. however, when listener responses are taught, it is not necessarily the case that tacts occur incidentally. this may explain the why teaching tacts is more likely to lead to emergence of listener responses than vice versa. as noted above, it is advisable to establish listener and tact responses before intraverbal behavior. however, it may be appropriate to teach a limited number of intraverbals prior to the establishment of tact and listener responses if those intraverbal responses are judged to be sufficiently important instructional targets (e.g., stating one’s name if asked). however, teaching of more advanced intraverbal responses is likely to be successful only after the child has acquired a robust echoic, listener, and tact repertoire. instruction can then proceed to more advanced verbal behavior, such as recall, various mands for information, abstract properties, syntax and grammar, various autoclitics, perspective taking, conversational exchanges, and so on. • • • references alessi, g. (1987). generative strategies and teaching for generalization. the analysis of verbal behavior, 5, 15. anderson, s. r., taras, m., & cannon, b. o. (1996). teaching new skills to young children with autism. in c maurice, g. green, & s. c. luce (eds.), behavioral intervention for young children with autism (pp. 181-194). austin, tx: pro-ed. international electronic journal of elementary education vol.9, issue 2, 433-450, december, 2016 446 andery, m. a., micheletto, n., & sério, t. m. (2005). meaning and verbal behavior in skinner's work from 1934 to 1957. the analysis of verbal behavior, 21, 163-174. axe, j. b. (2008). conditional discrimination in the intraverbal relation: a review and recommendations for future research. the analysis of verbal behavior, 24, 159-174. axe, j. b., & sainato, d. m. (2010). matrix training of preliteracy skills with preschoolers with autism. journal of applied behavior analysis, 43, 635–652. doi:10.1901/jaba.2010.43-635 barnes-holmes, d., barnes-holmes, y., & cullinan, v. (2000). relational frame theory and skinner's verbal behavior: a possible synthesis. the behavior analyst, 23, 69-84. braam, s. j., & poling, a. (1983). development of intraverbal behavior in mentally retarded individuals through transfer of stimulus control procedures: classification of verbal responses. the analysis of verbal behavior, 4, 279-302. bondy, a., & frost, l. (2001). the picture exchange communication system. behavior modification, 25, 725-744. carr, e. g., & durand, v. m. (1985). reducing behavior problems through functional communication training. journal of applied behavior analysis, 18, 111-126. carter, m., & grunsell, j. (2001). the behavior chain interruption strategy: a review of research and discussion of future directions. research and practice for persons with severe disabilities, 26, 37-49. catania, a. c. (1998). learning (fourth edition). upper saddle river, nj: prentice-hall. chevallier, c., kohls, g., troiani, v., brodkin, e. s., & schultz, r. t. (2012). the social motivation theory of autism. trends in cognitive sciences, 16, 231-239. chomsky, n. (1959). a review of skinner’s verbal behavior. language, 35, 26-58. cooper, j. o., heron, t. e., & heward, w. l. (2007). applied behavior analysis. upper saddle river, nj: pearson/merrill prentice hall. dounavi, k. (2014). tact training versus bidirectional intraverbal training in teaching a foreign language. journal of applied behavior analysis, 47, 165-170. dozier, c. l., iwata, b. a., thomason-sassi, j., worsdell, a. s., & wilson, d. m. (2012). a comparison of two pairing procedures to establish praise as a reinforcer. journal of applied behavior analysis, 45, 721-735. dyer, k., & peck, c. a. (1987). current perspectives on social/communication curricula for students with autism and severe handicaps. education and treatment of children, 10, 338-351. dymond, s., o’hora, d., whelan, r., & o’donovan, a. (2006). citation analysis of skinner's verbal behavior: 1984-2004. the behavior analyst, 29, 75-88. eikeseth, s., & smith, d. p. (2013). an analysis of verbal stimulus control in intraverbal behavior: implications for practice and applied research. the analysis of verbal behavior, 29, 125-135. eldevik, s., hastings, r. p., hughes, j. c., jahr, e., eikeseth, s., & cross, s. (2009). meta-analysis of early intensive behavioral intervention for children with autism. journal of clinical child & adolescent psychology, 38, 439-450. eldevik, s., hastings, r. p., hughes, j. c., jahr, e., eikeseth, s., & cross, s. (2010). using participant data to extend the evidence base for intensive behavioral intervention for children with autism. american journal on intellectual and developmental disabilities, 115, 381-405. frampton, s. e., wymer, s. c., hansen, b., & shillingsburg, m. a. (2016). the use of matrix training to promote generative language with children with autism. journal of applied behavior analysis. doi:10.1002/jaba.340 gevarter, c., o'reilly, m. f., kuhn, m., mills, k., ferguson, r., watkins, l., ... & lancioni, g. e. (2015). increasing the vocalizations of individuals with autism during intervention with a speechgenerating device. journal of applied behavior analysis, 49, 17-33. overview of verbal behavior / ingvarsson 447 goldsmith, t. r., leblanc, l. a., & sautter, r. a. (2007). teaching intraverbal behavior to children with autism. research in autism spectrum disorders, 1, 1-13. goldstein, h. (1983). recombinative generalization: relationships between environmental conditions and the linguistic repertoires of language learners. analysis & intervention in developmental disabilities, 3, 279-293. green, g. (2001). behavior analytic instruction for learners with autism: advances in stimulus control technology. focus on autism and other developmental disorders, 16, 72–85. greer, r. d., & ross, d. e. (2008). verbal behavior analysis: inducing and expanding new verbal capabilities in children with language delays. new york: pearson/allyn and bacon. grow, l. l., carr, j. e., kodak, t. m., jostad, c. m., & kisamore, a. n. (2011). a comparison of methods for teaching receptive labeling to children with autism spectrum disorders. journal of applied behavior analysis, 475-498. hanley, g. p., iwata, b. a., & mccord, b. e. (2003). functional analysis of problem behavior: a review. journal of applied behavior analysis, 36, 147-185. hart, b., & risley, t. r. (1978). promoting productive language through incidental teaching. education and urban society, 10, 407-29. hayes, s. c., barnes-holmes, d., & roche, b. (eds.). (2001). relational frame theory. a post-skinnerian account of human language and cognition. new york: kluwer academic / plenum publishers. holth, p., vandbakk, m., finstad, j., grønnerud, e., & sørensen, j. (2009). an operant analysis of joint attention and the establishment of conditioned social reinforcers. european journal of behavior analysis, 10, 143-158. horne, p. j., & lowe, c. f. (1996). on the origins of naming and other symbolic behavior. journal of the experimental analysis of behavior, 65, 185-241. ingvarsson, e. t., cammilleri, a. c., & macias. h. (2012). emergent listener responses following intraverbal training in children with autism. research in autism spectrum disorders, 6, 654664. ingvarsson, e. t., & hollobaugh, t. (2011). a comparison of prompting tactics to establish intraverbal responding in children with autism. journal of applied behavior analysis, 44, 659664. ingvarsson, e. t., kramer, r. l., carp, c. l., petursdottir, a. i., & macias, h. (2016). evaluation of a blocked-trials procedure to establish complex stimulus control over intraverbal responses in children with autism. the analysis of verbal behavior. doi:10.1007/s40616-016-0071-5 ingvarsson, e. t., & le, d. d. (2011). further evaluation of prompting tactics for establishing intraverbal responding in children with autism. the analysis of verbal behavior, 27, 75-93. isaksen, j., & holth, p. (2009). an operant approach to teaching joint attention skills to children with autism. behavioral interventions, 24, 215-236. kelly, m. a., roscoe, e. m., hanley, g. p., & schlichenmeyer, k. (2014). evaluation of assessment methods for identifying social reinforcers. journal of applied behavior analysis, 47, 113-135. kisamore, a. n., carr, j. e., & leblanc, l. a. (2011). training preschool children to use visual imagining as a problem-solving strategy for complex categorization tasks. journal of applied behavior analysis, 44, 255-278. kisamore, a. n., karsten, a. m., & mann, c. c. (2016). teaching multiply controlled intraverbals to children and adolescents with autism spectrum disorders. journal of applied behavior analysis. doi:10.1002/jaba.344 klintwall, l., eldevik, s., & eikeseth, s. (2015). narrowing the gap: effects of intervention on developmental trajectories in autism. autism, 19, 53-63. international electronic journal of elementary education vol.9, issue 2, 433-450, december, 2016 448 kohler, k. t., & malott, r. w. (2014). matrix training and verbal generativity in children with autism. the analysis of verbal behavior, 30, 170-177. doi:10.1007/s40616-014-0016-9 lamarre, j., & holland, j. g. (1985). the functional independence of mands and tacts. journal of the experimental analysis of behavior, 43, 5–19. laraway, s., snycerski, s., michael, j., & poling, a. (2003). motivating operations and terms to describe them: some further refinements. journal of applied behavior analysis, 36, 407-414. leaf, r., & mceachin, j. (1999). a work in progress. new york: autism partnership. lechago, s. a., & low, a. (2015). a review of the mand-for-information training research literature. international journal of behavior analysis & autism spectrum disorders, 1, 35-54. lepper, t. l., & petursdottir, a. i. (in press). effects of response-contingent stimulus pairing on vocalizations of nonverbal children with autism. journal of applied behavior analysis. lepper, t. l., petursdottir, a. i., & esch, b. e. (2013). effects of operant discrimination training on the vocalizations of nonverbal children with autism. journal of applied behavior analysis, 46, 656-661. doi:10.1002/jaba.55 lovaas, o. i. (1987). behavioral treatment and normal educational and intellectual functioning in young autistic children. journal of consulting and clinical psychology, 55, 3-9. lovaas, o. i. (2003). teaching individuals with developmental delays: basic intervention techniques. austin, tx: pro-ed. lovaas, o. i., freitag, g., kinder, m. i., rubenstein, b. d., schaeffer, b., & simmons, j. q. (1966). establishment of social reinforcers in two schizophrenic children on the basis of food. journal of experimental child psychology, 4, 109-125. lovaas, o. i., koegel, r. l., & schreibman, l. (1979). stimulus overselectivity in autism: a review of research. psychological bulletin, 86, 1236-1254. love, j. r., carr, j. e., almason, s. m., & petursdottir, a. i. (2009). early and intensive behavioral intervention for autism: a survey of clinical practices. research in autism spectrum disorders, 3, 421-428. lotfizadeh, a. d., edwards, t. l., redner, r., & poling, a. (2012). motivating operations affect stimulus control: a largely overlooked phenomenon in discrimination learning. behavior analyst, 35, 89. maurice, c., green, g., & foxx, r. m. (2001). making a difference: behavioral intervention for autism. austin, tx: pro-ed. maurice, c. e., green, g. e., & luce, s. c. (1996). behavioral intervention for young children with autism: a manual for parents and professionals. austin, tx: pro-ed. mcpherson, a., bonem, m., green, g., & osborne, j. g. (1984). a citation analysis of the influence on research of skinner's verbal behavior. the behavior analyst, 7, 157-167. michael, j. (1988). establishing operations and the mand. the analysis of verbal behavior, 6, 3-9. michael, j., palmer, d. c., & sundberg, m. l. (2011). the multiple control of verbal behavior. the analysis of verbal behavior, 27, 3-22. miguel, c. f., & petursdottir, a. i. (2009). naming and frames of coordination. in r. rehfeldt & y. barnes-holmes (eds.). derived relational responding: applications for learners with autism and other developmental disabilities (pp. 129-148). oakland, ca: new harbinger press. palmer, d. c. (1991). a behavioral interpretation of memory. in l. j. hayes & p. n. chase (eds.), dialogues on verbal behavior (pp. 261–279). reno, nv: context press. palmer, d. c. (2012). the role of atomic repertoires in complex behavior. the behavior analyst, 35, 59-73. overview of verbal behavior / ingvarsson 449 partington, j. w. (2008). the assessment of basics language and learning skills-revised: scoring instructions and iep development guide. pleasant hill, ca: behavior analysts. pauwels, a. a., ahearn, w. h., & cohen, s. j. (2015). recombinative generalization of tacts through matrix training with individuals with autism spectrum disorder. the analysis of verbal behavior, 31, 200–214. doi: 10.1007/s40616-015-0038-y petursdottir, a. i., & carr, j. e. (2011). a review of recommendations for sequencing receptive and expressive language instruction. journal of applied behavior analysis, 44, 859-876. petursdottir, a. i., carr, j. e., lechago, s. a., & almason, s. m. (2008). an evaluation of intraverbal training and listener training for teaching categorization skills. journal of applied behavior analysis, 41, 53-68. rutter, m., & thapar, a. (2014). genetics of autism spectrum disorder. in f. r. volkmar, s. j. rogers, r. paul, & k. a. pelphrey (eds.), handbook of autism and pervasive developmental disorders (fourth edition), (pp. 411-423). hoboken, nj: wiley. sallows, g. o., & graupner, t. d. (2005). intensive behavioral treatment for children with autism: four-year outcome and predictors. american journal on mental retardation, 110, 417-438. sautter, r. a., & leblanc, l. a. (2006). empirical applications of skinner's analysis of verbal behavior with humans. the analysis of verbal behavior, 22, 35-48. sautter, r. a., leblanc, l. a., jay, a. a., goldsmith, t. r., & carr, j. e. (2011). the role of problem solving in complex intraverbal repertoires. journal of applied behavior analysis, 44, 227-244. schoneberger, t. (1991). verbal understanding: integrating the conceptual analyses of skinner, ryle, and wittgenstein. the analysis of verbal behavior, 9, 145-151. shillingsburg, m. a., hollander, d. l., yosick, r. n., bowen, c., & muskat, l. r. (2015). stimulusstimulus pairing to increase vocalizations in children with language delays: a review. the analysis of verbal behavior, 31, 215-235. schlinger, h. d. (2008). listening is behaving verbally. the behavior analyst, 31, 145-161. sidman, m. (1994). equivalence relations and behavior: a research story. boston, ma: authors cooperative. skinner, b. f. (1938). the behavior of organisms: an experimental analysis. cambridge, ma: b. f. skinner foundation. skinner, b. f. (1953). science and human behavior. new york: the free press. skinner, b. f. (1957). verbal behavior. acton, ma: copley publishing group. suchowierska, m. (2006). recombinative generalization: some theoretical and practical remarks. international journal of psychology, 41, 514-522. sundberg, m. l. (2008). verbal behavior milestones assessment and placement program: the vbmapp. concord, ca: avb press. sundberg, m. l., loeb, m. hale, l., & eigenheer, p. (2002). contriving establishing operations to teach mands for information. the analysis of verbal behavior, 18, 15-29. sundberg, m. l., & partington, j. w. (1998). teaching language to children with autism and other developmental disabilities. pleasant hill, ca: behavior analysts, inc. sundberg, m. l., & sundberg, c. a. (2011). intraverbal behavior and verbal conditional discriminations in typically developing children and children with autism. the analysis of verbal behavior, 27, 23-43. taylor, b. a., & harris, s. l. (1995). teaching children with autism to seek information‐acquisition of novel information and generalization of responding. journal of applied behavior analysis, 28, 314. thompson, t. (2007). making sense of autism. baltimore: paul h. brookes. international electronic journal of elementary education vol.9, issue 2, 433-450, december, 2016 450 vargas, e. a. (1986). intraverbal behavior. in p. n. chase & l. j. parrott (eds.), psychological aspects of language (pp. 128-151). springfield, il: charles c. thomas. volkmar, f. r., & mcpartland, j. c. (2014). from kanner to dsm-5: autism as an evolving diagnostic concept. annual review of clinical psychology, 10, 193-212. williams, b. a. (1994). conditioned reinforcement: experimental and theoretical issues. the behavior analyst, 17, 261-285. international electronic journal of elementary education, 2015, 7(3), 371-382. issn:1307-9298 copyright © iejee www.iejee.com preschool children’s perceptions of the value of affection as seen in their drawings yunus güni̇ndi̇  aksaray university, turkey received: 3 january 2015 / revised: 15 march 2015 / accepted: 27 april 2015 abstract the purpose of this study is to examine the perceptions of children in preschool education with regard to the value of affection in the pictures they draw. the study involved 199 children aged 60 months old or above. the descriptive research method was used and data were collected with the draw-and-explain technique. during the collection of the data, the children were requested to draw a picture related to the value of affection and explain the picture they drew. the children’s explanations were recorded by the researcher. the study is one of the first to be conducted in turkey with preschoolers in this research area. the results showed that the children generally depicted human figures like family members, other children and friends, animals like butterflies and dogs, trees, flowers and grass, happy images such as hearts, balloons and balls, and abiotic images like clouds and sunshine, as well as other images like houses in their drawings. the children tended especially to feature people and objects in their immediate vicinity. keywords: preschool, drawing of value of affection, value education, perception of value of affection in children. introduction from birth, humans find themselves within a social existence and try to adapt themselves to their socio-cultural environment. this effort of adaptation continues throughout the child’s development. the basis of children’s social and emotional development is substantially built in the first years of life, as with all areas of development (günindi, 2011). therefore, early childhood constitutes the most critical part of life in terms of adopting values that children will need socially. because children in this period are going through the fastest stage of their development, where their personality is built, they are strongly affected by their immediate environment and open to any kind of learning. their development can therefore be supported with timely and efficient interventions. between the ages of zero and six a child begins to adopt the value judgments of the immediate society and the behaviors and habits that comply with its cultural texture. the  yunus günindi, aksaray university, adana yolu uzeri campus, aksaray, 68100, turkey, phone: +90 (382) 288-2271. e-mail: yunusgunindi@aksaray.edu.tr http://www.iejee.com/ international electronic journal of elementary education vol.7, issue 3, 371-382, 2015 372 child’s interaction with his/her peers and other individuals within the environment during the preschool period helps them gain many positive and negative behaviors, skills, manners and opinions. these build the basis of their value judgments. while information about values is learnt during a person’s whole life, the first knowledge is gained in early childhood (bilir & bal, 1989; bronson, 2000; davies, 2004; uyanık-balat & balaban-dağal, 2009; dereli-i̇man, 2014). many definitions have been proposed for the notion of ‘value’. although it is generally defined as an important criterion within cultures and societies, values tend to be understood to consist of objectives generated against the backdrop of the ideas, standards and targets adopted by a group, or behavioral patterns organized such that the individual can maintain his/her existence within the group in accordance with the standards that are considered right by all the individuals within the society. values can also correspond to behaviors and implementations which grow over a long period and to which society expects individuals to conform, or generalized ethical principles or beliefs which are considered right and useful by most of the members of a social group or society in order to maintain the existence, unity and continuity of that group or society; they can also reflect common feelings, opinions, targets and benefits, or more or less certain and systematic ideas enabling the individual’s interaction with the environment (türk, 2009; fichter, 2006; bolay, 2004; veugelers & vedder, 2003; kızılçelik & erjem, 1996; titus, 1994). studies conducted on values education in turkey and the world generally focus on topics like the provision of values, implementing values education programs, and the effect of the family on values education. children’s own opinions of values and the way they perceive and make sense of them seem not to have been studied adequately. existing studies involving interviews and surveys among teachers or families have tended not to favor drawing as a means to determine children’s perceptions (revell, 2002; veugelers & kat, 2003; berkowitz & bier, 2005; skaggs & bodenhorn, 2006; gökçek, 2007; husu & tirri, 2007; i̇nci, 2009; richardson, tolson, huang & lee, 2009; üner, 2011; öztürk samur, 2011; uyanık balat, özdemir beceren & adak özdemir, 2011). however, pictures provide the potential for children to present their world-views in their reactions against telling a story, relaying metaphors and both the description itself and their own descriptions. a child synthesizes his/her opinions and feelings about the subject with his/her observations and expresses them by means of colors, shapes and lines while drawing (malchiodi, 2013). children synthesize their observations from life with their opinions in their pictures, and reflect on what happens in their environment in the way they perceive it. drawing a picture is both an enjoyable activity and an explanation technique for children (hayes, symington & martin, 1994; johnson, 1993). while children are often uncomfortable answering questions asked in interviews, they express the same information willingly when asked to draw a picture (lewis & greene, 1983). drawing is also seen as an alternative means of expression for children who cannot express themselves verbally (chambers, 1983; rennie & jarvis, 1995). if pictures drawn by children are analyzed well, they can provide researchers with detailed information about their knowledge and development (yavuzer, 1997), perhaps more so than written or verbal texts. aspects of psychology have been lighting the way in the field of psychiatry and picture therapy (malchiodi, 2013). the objects drawn by children, colors and the painting materials they use can all provide important clues for diagnostic and therapeutic purposes. diagnosis and treatment are not included in this study, so these elements have not been evaluated in the children’s drawings. this research was conducted with the draw-and-explain technique to determine the perception of children in relation to the preschool children’s perceptions of the value of affection / günindi 373 value of ‘affection’. the literature review revealed no studies analyzing the drawings of children in relation to the value of ‘affection’. this study therefore fills a research gap. method the descriptive research method was used and data were collected with the draw-andexplain technique. the study group consisted of 199 children in total, who went to independent kindergartens in aksaray and were aged 60 months or above. measurement tool the draw-and-explain technique was used to determine the children’s perception of the value of ‘affection’ (brackett-milburn, 1999; shepardson, 2005). this technique observes children’s drawings and their explanations of these drawings. the draw-and-explain technique is a diagnostic method used to evaluate how children structure opinions and concepts (mcwhirter, collins, bryant, wetton & bishop, 2000). the children were asked to draw whatever came to their minds when ‘affection’ was mentioned, and to explain these drawings. the researcher recorded the children’s explanations according to pre-prepared codes applied to each drawing application process and environment the study was conducted within the academic year 2014–2015. after permission had been obtained, the researcher went to the participating institutions and informed the administrators about the study. after this information had been obtained, an implementation plan was drawn up together with the school managers to determine when and how the assessment instrument would be applied. while the children narrated their drawings, the narration was recorded. researcher’s role: before the research, the researcher visited the participating schools for three weeks to teach two-hour lessons about issues independent of the research (children were asked to draw responses to such questions as ‘what comes to your mind when you hear “environment”?’, ‘what does “hero” mean for you?”, ‘what does ‘being healthy’ mean for you?’, and so on). this was to enable the children to get used to the researcher so they could express themselves comfortably. finally, the researcher asked the children “what comes to your mind when you hear ‘affection’?” and they were asked to draw their responses. the children’s responses were not guided or interrupted by the researcher. after the children had completed their drawings, the researcher asked about them and what they wanted to express. each drawing was filed separately after the researcher had recorded each child’s narration of his or her drawing on a blank sheet and attached it to the reverse of each picture. data analysis the data obtained were analyzed using interpretative content analysis, a qualitative method (ball & smith, 1992; banks, 2001). the interpretative content analysis included determination and definition of themes, subjects and cases in the visual and written material obtained from the study (giarelli & tulman, 2003). the codes and themes acquired were reviewed by the researcher for validity and reliability and analyzed using the same procedures as different specialist researchers. the reliability formula suggested by miles and huberman (1994) was used to calculate the reliability of the research: reliability = agreement / (agreement + disagreement) the research reliability was calculated as 92%. values above 70% are considered reliable (miles & huberman, 1994), so this research can be considered reliable. in addition, to evaluate the consensus between the experts and the researcher a kappa international electronic journal of elementary education vol.7, issue 3, 371-382, 2015 374 (conformity) analysis was carried out. at the end of this assessment the kappa value was found to be 86%. the results can therefore be considered reliable. the use of verification strategies in qualitative research is important to increase reliability (morse et al., 2002). thus, all steps followed in the research are reported. the researcher made relevant notes about the application environment and implementation process in the form of short notes. notes not used as data enabled the researcher to review the implementation. the data obtained from the study were also reported in the context of a descriptive analysis and a percentage and frequency analysis. findings in total, 10 categories and 104 codes were obtained from the analysis of the data. children generally drew human figures like family members, other children and friends, animals such as butterflies and dogs, trees, flowers and grass, happy images such as hearts, balloon and balls, abiotic images like clouds and sunshine, and structures like houses. table 1. figures included in the drawings of the children participating in the study categories codes 60 months old or above f % family/relatives mother 56 28.1 father 46 23.1 siblings 57 28.6 themselves 35 17.6 grandmother 2 1 paternal grandmother 3 1.5 grandfather 16 8 uncle 4 2 maternal aunt 2 1 brother’s wife 3 1.5 other people friends 15 7.5 baby 4 2 child 34 17.1 visitor 10 5 teacher 9 4.5 greengrocer 3 1.5 plants tree 22 11.1 flower 68 34.2 grass 24 12.1 fruit 13 6.5 animals bird 2 1 butterfly 38 19.1 dog 11 5.5 cat 8 4 bear 2 1 rabbit 5 2.5 sheep 2 1 turtle 3 1.5 shark 1 .5 fish 4 2 chick 10 5 preschool children’s perceptions of the value of affection / günindi 375 table 1 (cont.). figures included in the drawings of the children participating in the study categories codes 60 months old or above f % animals ant 4 2 abiotic elements mountain 4 2 cloud 42 21.1 sun 84 42.2 river 5 2.5 sea 5 2.5 planet 2 1 sky 7 3.8 beach 2 1 star 4 2 buildings/vehicles home 65 32.7 school 6 3 car 13 6.5 plane 1 .5 road 6 3 farmstead 5 2.5 elevator 2 1 bicycle 1 .5 wall 3 1.5 pool 1 .5 statue 3 1.5 bulldozer 2 1 traffic lamp 4 2 ladder 5 2.5 motorcycle 2 1 semi 2 1 rocket 2 1 slide 3 1.5 rail 3 1.5 happy image balloon 6 3 entertainment 1 .5 smile 3 1.5 gift 2 1 heart 40 20.1 kiss 1 .5 cotton candy 2 1 money 2 1 affection 2 1 ball 7 3.5 kite 2 1 ankara 2 1 istanbul 2 1 play 3 1.5 toy 3 1.5 foods pastry 2 1 potato 3 1.5 nuts 3 1.5 egg 3 1.5 international electronic journal of elementary education vol.7, issue 3, 371-382, 2015 376 table 1 (cont.). figures included in the drawings of the children participating in the study categories codes 60 months old or above f % natural events/ seasons rainbow 15 7.5 rain 2 1 snow 2 1 winter 8 4 summer 3 1.5 others computer 3 1.5 paint 2 1 dress 2 1 bag 2 1 nail 1 .5 closet 4 2 ghost 1 .5 rope 1 .5 cage 1 .5 door 2 1 book 2 1 window 4 2 straw 2 1 chair 2 1 number 2 1 wheel 1 .5 television 2 1 vase 2 1 leaf 1 .5 snowman 6 3 the images included in the drawings and the frequencies of these images are given in table 1. 42.2% of the children who participated in the study depicted sunshine, 34.2% flower(s), 32.7% house(s), 28.6% sibling(s), 28.1% their mother, 23.1% their father, 21.1 cloud(s), 17.6% themselves and 17.1% other children in their drawings. the mostencountered animal in the pictures was the butterfly (19.1%), followed by the dog (5.5%) and the rabbit (2.5%). images that expressed happiness such as hearts (20.1%), balls (3.5%) and balloons (3%) were also observed. examples of children’s drawings are given in picture 1, picture 2, picture 3 and picture 4. picture 1 drawing, the child shows an unknown girl, herself, a flower, grass and a cloud. the child narrates her drawing as follows: “i am gifting a flower to a girl whom i don’t know. the girl to whom i gift the flower gets very happy and walks on air. then, she starts running, saying that she has to go. she falls down, because she stumbles while running. i help her with getting up. she thanks me. the fact that i give a flower to someone whom i don’t know, and help her, shows my affection for her, and it points at her affection for me when she thanks me.” smiling children and flowers, grass and blue clouds are also observed in the pictures of many other children. in picture 1, the value of ‘affection’ was described with the figures of an unknown girl, herself, flower, grass and cloud. picture 2 shows happy-looking people including the child’s mother, father and sibling, at home in sunny weather. a heart has also been drawn to intensify the impression of ‘affection’. preschool children’s perceptions of the value of affection / günindi 377 picture 1. picture 2. international electronic journal of elementary education vol.7, issue 3, 371-382, 2015 378 picture 3. in this drawing, a flower, a house, sunshine, a butterfly, a rabbit, a child, and some grass and clouds are shown. the child narrated the picture by saying: ‘affection means loving nature, flowers and animals.’ in the picture, the sun, flowers and smiling rabbit and butterfly stand out. . picture 4. preschool children’s perceptions of the value of affection / günindi 379 in picture 3, the value of ‘affection’ was described with flower, house, sun, butterfly, rabbit, child, grass and cloud figures. picture 4 depicts a friend, sunshine, a butterfly, some flowers, some stairs and a heart. the child narrated his/her picture by saying: ‘affection means loving your friend, mother, father, everyone and nature.” we can see that the heart image was used to intensify the impression of ‘affection’. when the records of the interviews held with children are reviewed, we can see that being with family members, other people and plants and animals has particular significance within children’s perceptions of the value of ‘affection’. discussion and suggestions ‘affection’ is described as “the feeling that directs a person to show close interest and attachment to a thing or a person” by the antalya governorship provincial directorate of national education, in the booklet entitled ‘values education in preschool’. in line with the ministry of national education, the booklet mentions the necessity of helping preschoolers adopt affection for nature, animals and plants (m.n.e., 2011). the participating children spoke about their drawings by making stories out of them. some of their narrations were as follows: child a: affection means hugging my mum and dad. child b: affection means things in nature loving each other. child c: affection means my mum, my dad and my siblings. it means flying birds in the sky. child d: affection means loving nature. affection means growing flowers and loving animals. child e: affection means sun’s chatting with sun’s friend, rabbit’s having a chat with rabbit’s friend, people loving both nature and animals. the interviews held with children show affection for family members, relatives, friends and even people they do not know, besides affection for nature, animals and plants. as stated, the literature review revealed no studies using the draw-explain technique with regard to the value of ‘affection’ for preschoolers. moreover, few studies were encountered involving preschoolers within the scope of values education; such studies that existed mostly involved children in primary and secondary schools, families or teachers. these studies examined the relationship between behavior and success and between the values education program and family participation, ethical maturity, and academic success, differences in value preferences between male and female students, the effect of values education on the exhibition of positive behaviors and level of socialemotional development, the values expected of children by families, and the effect of teachers relaying values as a result of the program applied for values education, using assessment instruments such as surveys and interviews (dilmaç, 1999; veugelers & kat, 2003; sarı, 2005; berkowitz & bier, 2005; skaggs & bodenhorn, 2006; gökçek, 2007; husu & tirri, 2007; katılmış, 2010; öztürk samur, 2011; uyanık balat, özdemir beceren & adak özdemir, 2011). pictures drawn by children can be used to determine their knowledge about a topic, their misconceptions and interests. the first researchers to study children’s pictures did so to determine their intelligence level (burt, 1921; goodenough, 1926; harris, 1963). some early theorists and researchers also examined children’s pictures in relation to developmental and personality traits within the process (koppitz, 1968; hammer, 1958; international electronic journal of elementary education vol.7, issue 3, 371-382, 2015 380 machover, 1949). studies conducted on children’s pictures and on how children perceive what happens in their environment have started to be used by educators in recent years. although the examination of children’s drawings is an effective method, the number of studies conducted on this topic is quite low. on this basis, drawings should be used also for different values in future studies within the scope of values education. pictures drawn by children can also be used to assess children’s opinions and perceptions, the causes of values-related problems they encounter in the school and family environment, and the solutions they think of to solve these problems. although a limited number of children were studied in relation to ‘affection’ in this study, the data obtained are valuable because to the researcher’s knowledge there are no existing studies on children’s perception of the value of ‘affection’ using their drawings. in the future similar studies should be carried out with different samples and the results compared. even though the self-expression of children by drawing is an effective subject for analysis, it has some limitations. interviews should be carried out with children and recorded, besides having them draw pictures in order to reduce limitations. • • • references ball, m. & smith, g. (1992). analyzing and visualizing data. london: sage. banks, m. (2001). visual methods in social research. london: sage. berkowitz, m. w. & bier, m. c. (2005). character education: parents as partners. educational leadership, 63(1), 64-69. bilir, ş. & bal, s. (1989). kütahya il merkezinde anaokuluna giden ve gitmeyen 4-6 yaşlar arasındaki çocukların kullandıkları ifadelerin sözdizimi yönünden incelenmesi: 6.ya-pa semineri kitapçığı. i̇stanbul: ya-pa yayınevi. bolay, s. h. (2004). değerlerimiz ve günlük hayat. değerler eğitimi merkezi dergisi, 1(1), 12-19. brackett-milburn, k. (1999). a critical appraisal of the draw and write technique. health education research, 14(3), 387-395. bronson, m. b. (2000). self-regulation in early childhood: nature and nurture. new york: guildford press. chambers, d. w. (1983). stereotypic images of scientists: the draw-a-scientist test. science education, 67(2), 255-265. davies, d. (2004). child development a practitioner’s guide. new york: guildford press. dereli-i̇man, e. (2014). değerler eğitimi programinin 5-6 yaş çocuklarin sosyal gelişimine etkisi: sosyal beceri, psiko-sosyal gelişim ve sosyal problem çözme becerisi. kuram ve uygulamada eğitim bilimleri, 14 (1), 249-268. dilmaç, b. (1999). i̇lköğretim öğrencilerine insani değerler eğitimi verilmesi ve ahlaki olgunluk ölçeği ile eğitimin sınanması. yayımlanmamış yüksek lisans tezi. i̇stanbul: marmara üniversitesi eğitim bilimleri enstitüsü. fichter, j. (2006). sosyoloji nedir (çev. nilgün çelebi). ankara: anı yayıncılık. giarelli, e. & tulman, l. (2003). methodological issues in the use of cartoons as data. qualitative health research, 13(7), 945-956. preschool children’s perceptions of the value of affection / günindi 381 gökçek, b. s. (2007). 5–6 yaş çocukları için hazırlanan karakter eğitimi programının etkisinin incelenmesi. yayınlanmamış doktora tezi. marmara üniversitesi eğitim bilimleri enstitüsü i̇lköğretim ana bilim dalı okul öncesi eğitimi bilim dalı, i̇stanbul. günindi, y. (2011). bağımsız anaokullarina ve anasiniflarina devam eden çocuklarin sosyal becerilerinin değerlendirilmesi. ahi evran üniversitesi eğitim fakültesi dergisi, cilt 12, sayı 1, nisan 2011, sayfa 133-144. hayes, d., symington, d., & martin, m. (1994). drawing during science activity in the primary school. international journal of science education, 16, 265-277. husu, j. & tirri, k. (2007). developing whole school pedagogical values – a case of going through the ethos of good schooling. teaching and teacher education, 23(4), 390-401. i̇nci, e. (2009). erken çocukluk dönemi öğrencilerinin değer yönelimlerinin sosyo-ekonomik düzey, yaş ve cinsiyet değişkenlerine göre incelenmesi. adnan menderes üniversitesi, sosyal bilimler enstitüsü, i̇lköğretim anabilim dalı yayınlanmamış yüksek lisans tezi. aydın. johnson, p. (1993). literacy through the book arts. chicago: heinemann. katılmış, a. (2010). sosyal bilgiler derslerindeki bazı değerlerin kazandırılmasına yönelik bir karakter eğitimi programı geliştirilmesi. yayınlanmamış doktora tezi. marmara üniversitesi eğitim bilimleri enstitüsü, i̇stanbul. kızılçelik, s. & erjem, y. (1996). açıklamalı sosyoloji terimler sözlüğü. i̇zmir: saray yayıncılık. lewis, d. & greene, j. (1983). your child’s drawings… their hidden meaning, london: hutchinson. malchiodi, a. c. (2013). çocukların resimlerini anlamak (çev: yurtbay, t.). nobel tıp kitapevleri, i̇stanbul. mcwhirter, j. m., collins, m., bryant, i., wetton, n. m., & bishop, j. n. (2000). evaluating ‘safe in the sun’, a curriculum programme for primary schools. health education research, 15(2), 203-217. m.n.e. (2011). values education in preschool’. antalya governorship provincial directorate of national education, 2011. miles, m. b. & huberman a. m. (1994). qualitative data analysis: an expanded source book. thousand oaks, ca: sage. morse, j.m., m. barrett, m. mayan, k. olson, & j. spiers. (2002). verification strategies for establishing reliability and validity in qualitative research. international journal of qualitative methods, 1(2), 13-22. öztürk, s. a. (2011). ayrıntılı değerler eğitimi programının 6 yaş çocuklarının sosyal ve duygusal gelişimine etkisi. yayımlanmamış doktora tezi. selçuk üniversitesi sosyal bilimler enstitüsü, konya. rennie, l. j. & jarvis, t. (1995). children’s choice of drawings to communicate their ideas about technology. research in science education, 25(3), 239-252. revell, l. (2002). childrens responses to character education. educational studies, 28(4), 421-431. richardson, r. c., tolson, h., huang, t. y. & lee y. h. (2009). character education: lessons for teaching social and emotional competence. children & schools, 31(2), 71-78. sarı, e. (2005). öğretmen adaylarının değer tercihleri: giresun eğitim fakültesi örneği. değerler eğitimi dergisi, 3(10), 73-88. shepardson, d. p. (2005). student’s ideas: what is an environment? journal of environmental education, 36(4), 49-58. skaggs, g. & bodenhorn, n. (2006). relationships between implementing cracter education, student behavior and student achievement. journal of advanced academics, 18, 1, 82-114. titus, d. n. (1994). value education in american secondary schools. paper presented at the kutztown university education conference. kutztown, pa, september 16. international electronic journal of elementary education vol.7, issue 3, 371-382, 2015 382 türk, i̇. (2009). değerler eğitiminde saygı. yayınlanmamış yüksek lisans tezi, gaziosmanpaşa üniversitesi sosyal bilimler enstitüsü, tokat. uyanık-balat, g. & balaban-dağal, a. (2009). okul öncesi dönemde değer eğitimi etkinlikleri. ankara: kök yayıncılık. uyanık-balat, g., özdemir-beceren, b. & adak-özdemir, a. (2011). the evaluation of parents’ views related to helping pre-school children gain some universal values. procedia social and behavioral sciences, 15, 908-912. üner, e. (2011). okul öncesi eğitim programındaki 36-72 aylık çocuklara farklılıklara saygı eğitimi kazandırmanın öğretmen görüşleri doğrultusunda değerlendirilmesi. yayınlanmamış yüksek lisans tezi, erciyes üniversitesi eğitim bilimleri enstitüsü. kayseri. veugelers, w. & vedder p. (2003). values in teaching. teachers and teaching: theory and practice, 9(4), 377-389. veugelers, w. & kat, e. (2003). moral task of the teacher according to students, parents and teachers. educational research and evaluation, 9(1), 75-91. yavuzer, h. (1997). resimleriyle çocuğu tanıma. i̇stanbul: remzi kitapevi. microsoft word 5_iejee_4_1_ray_meyer international electronic journal of elementary education, 2011, 4(1), 67-82. issn:1307-9298 copyright © iejee www.iejee.com individual differences in children’s knowledge of expository text structures: a review of literature melissa n. ray the pennsylvania state university, united states bonnie j. f. meyer ∗∗∗∗ the pennsylvania state university, united states abstract in this review of literature we examine empirical research of individual differences in younger readers’ knowledge and use of expository text structures. the goal of this review is to explore the influence of reader and text characteristics in order to better understand the instructional needs of elementary school readers. first we review research which has examined the influence of two textual characteristics: the hierarchical organization of macro-and micro-level propositions and the type of text structure (e.g. collection, comparison, problem-and-solution). then we review research of three reader characteristics: overall comprehension skill, age, and prior knowledge and how their influences may vary in relation to the aforementioned text characteristics. our review of research suggests that readers of all ages may benefit from explicit instruction in text structure, particularly less-skilled comprehenders. text structure instruction should focus on highly structured texts like comparison, causation, and problem-and-solution. keywords: text structure, structure strategy, expository text introduction informational texts can present a challenge for many elementary school readers. in comparison to narrative texts, young readers may have more difficulty comprehending expository texts (best, floyd, & mcnamara, 2008). this difficulty is likely the result of both the demands on readers’ prior knowledge (best et al., 2008) as well as lower levels of early exposure to expository texts (duke, 2000). an additional source of difficulty may be the structure of expository texts (coté, goldman, & saul, 1998). in comparison to narrative texts, a larger number of structures are used to describe the organization of expository texts. meyer (1975, 1985a) proposed five top-level structures: collection, description, comparison, ∗ bonnie j. f. meyer, department of educational and counseling psychology and special education, 204 cedar building, penn state, university park, pa 16802, united states. phone: 814-863-7501; fax: 814-863-1002. e-mail: bjm8@psu.edu international electronic journal of elementary education 68 causation, and response (problem-and-solution). when reading expository texts younger readers are confronted unfamiliar concepts that are organized in an unfamiliar way. difficulties in comprehension of informational texts may lead to trouble with learning from text, particularly in content areas. in order to meet the challenge of reading these texts, readers may employ a number of comprehension processes including: paraphrasing, elaborating (connecting to prior knowledge), and monitoring (coté et al., 1998). another way in which younger readers can successfully comprehend expository text is through their knowledge and use of text structure. readers who possess a “structure strategy” have knowledge how authors organize texts and seek to apply this knowledge to the text being read, while readers without this strategic knowledge approach text with a “default/list” strategy in which the text is viewed as a collection of loosely related ideas (meyer, brandt, & bluth,1980; see meyer & rice, [1982] for a more detailed description of the structure strategy). readers with structural awareness possess the ability to not only recognize an author’s organization, but also to engage in similar organizational processes in establishing their own mental representations of a text. awareness of text structure has been associated with better text recall in terms of the number of ideas remembered and their organization (meyer et al., 1980; taylor & samuels, 1983). structural awareness improves comprehension because it facilitates the construction of a coherent mental representation of text. coherence, the creation of clear relationships between and among textual ideas contained in one’s cognitive representation is considered an essential aspect of text comprehension (van dijk & kintsch, 1983; kintsch, 2004; van den broek, young, tzeng, & linderholm, 2004). readers with structural awareness develop a mental representation in which textual ideas are organized according to a defined hierarchical structure. this organization helps readers’ ability to remember ideas from a text, especially those propositions at the top of the hierarchy (britton, meyer, hodge, & glynn, 1980, meyer et al., 1980; taylor & samuels, 1983). although some younger readers possess knowledge of expository text structure (englert & hiebert, 1984; mcgee, 1982; richgels, mcgee, lomax, & sheard, 1987; smith & hahn, 1987; yochum, 1991), many do not (meyer et al., 1980; taylor, 1980; taylor & samuels, 1983). the extent to which children are structurally aware may vary as a function of both textual characteristics such as text structure (richgels et al., 1987; smith & hahn, 1989; yochum, 1991) as well as reader characteristics, such as age (englert & hiebert,1984; englert & thomas, 1987; garner & gillingham, 1987; mcgee, 1982; smith & hahn, 1989) and overall comprehension skill (meyer et al., 1980; englert & hiebert, 1984). intervention research has shown that with explicit instruction in the use of text structure, readers can improve both structural knowledge and comprehension (e.g. armbruster, anderson, & ostertag, 1987; meyer et al., 2002; meyer et al., 2010; williams et al., 2005; williams et al., 2007; see meyer & ray in this issue). however, there is evidence that instruction which is individualized to the instructional needs of the reader is more effective than less individualized instruction (meyer, wijekumar, & lin, 2011). in order to better understand elementary school readers’ instructional needs, it is necessary to have a clear understanding of sources of individual differences. purpose of this review of literature this review of literature examines sources of individual differences in children’s awareness of expository text structures. previous reviews of literature have described the contribution of reader and text variables on the processing of text structure (e.g. goldman & rakestraw, 2000) as well as the interaction between reader and text variables (roller, 1990). in this review, we investigate how sources of individual differences among readers interact with individual differences in children’s knowledge / ray & meyer 69 structural properties to influence both structural knowledge and use of this knowledge in comprehending expository texts. this review differs from roller (1990) in that we examined the relationship among multiple reader characteristics, while the review conducted by roller focused specifically on one individual difference: prior knowledge. we take the position that structural awareness varies not only as a function of the reader but also as the reading situation changes. this view reflects current perspectives on reading which stress the importance of the relationship between reader, text, and situational variables (rand reading study group, 2002). our goal is to provide a comprehensive picture of students’ instructional needs, by providing insights into variability of children’s structural knowledge. reader and textual characteristics we used the framework created by meyer and rice (1989) to guide the selection of reader and textual influences (see also meyer, 2003). meyer and rice proposed three sources of individual differences in text comprehension: the reader, the text, and the task. most relevant to the current review are reader and text variables. reader variables refer to the experiences and skills that readers bring with them to the text and include: age, verbal ability, and prior knowledge (meyer & rice, 1989). text variables refer to those variables which contribute to the content and organization of the text such as text structure, genre, and topic (meyer & rice, 1989). although there are several possible reader and text influences, we selected those influences which have been both frequently researched and are most relevant for instruction of elementary school readers. with regards to textual characteristics we focus on two major influences. the first is the overall quality of the hierarchical organization and the extent to which subordination of ideas is present. the second source of variability we examined is the type of top-level, rhetorical structure used to organize the text (e.g. comparison, problemand-solution). we include in this examination of rhetorical structure empirical studies which have examined readers’ sensitivity to text signaling. text signals are textual elements (e.g. headings, overviews, previews, and connectives) which call readers’ attention to the structure of the text (meyer, 1985b; lorch, 1989). although this review treats these text characteristics as separate elements, these aspects of text structure are not necessarily mutually exclusive. in this review, we distinguish these elements of structure in order to more closely examine readers’ knowledge of various types of top-level-structures as well as their ability to recognize the hierarchical relationships between macroand micro-level propositions contained in the text. this review does not cover issues of cohesion and its relationship to reader characteristics (see mcnamara, kintsch, songer & kintsch,1996; ozuru, dempsey, & mcnamara, 2009; voss & silfies, 1996). cohesion was excluded from the current review as it covers a broader set of textual characteristics than text structure. in relation to reader characteristics we focused on three sources of individual differences: overall comprehension skill, age, and prior knowledge. although each of these variables has been found to influence structural awareness, the nature of this influence is somewhat different. age and comprehension skill are associated with both the level of awareness a reader has and their ability/need to apply this knowledge. in contrast, prior knowledge is usually associated with the circumstances under which readers do and do not use knowledge of text structure. for each of these reader variables, we explore its unique contribution and how the influence of each of these variables changes in relation to the previous mentioned characteristics of the text. this review of literature does not represent an exhaustive review of those studies which have examined awareness of text structure; rather we have selected those studies which are representative of research on the influence of reader and textual characteristics. we chose to international electronic journal of elementary education 70 focus only on those studies which closely align with meyer’s (1975, 1985a) analysis and classification of expository text structures (for alternative approaches to the description and classification of text structure see alvermann & hague, 1989; frederiksen, 1975; van dijk & kintsch, 1983). we selected meyer’s approach because several studies of younger readers’ structural knowledge as well as interventions designed to teach text structure, reflect this approach. this allowed us to draw meaningful conclusions across studies. wherever possible, we include those studies which have investigated younger (elementary and middle school) readers. influence of text characteristics the overall quality of the hierarchical relationships between propositions within a text may influence readers’ ability to perceive and use text structure. the organization of expository text is centered on the hierarchical relationship between macropropositions, higher level ideas, and micropropositions, the lower level ideas which elaborate on them (weaver & kintsch, 1991). when the overall organization is compromised, comprehension and recall performance are negatively affected (kintsch & yarborough, 1982; taylor & samuels,1983; danner, 1976). kintsch & yarborough (1982) found that poorly constructed texts were associated with poorer performance on main idea questions in comparison to wellstructured text. similarly, danner (1976) and taylor and samuels (1983) reported that elementary school readers recalled fewer ideas when reading texts that lacked a clear organization. taylor and samuels found the effect of textual organization was related to readers’ level of structural awareness. readers classified as structurally aware had a greater number of ideas recalled with unscrambled over scrambled text, while readers classified as unaware had a similar number of ideas recalled for both scrambled and unscrambled passages (taylor & samuels, 1983). this finding suggests that in order for structural knowledge to benefit comprehension, the reader must be able to perceive the hierarchical organization of the text. text structure readers’ structural awareness and the benefits of structural knowledge may also vary in relation to the type of text structure being read. for adults, different text structures may have varying effects on the number of ideas recalled and their organization. meyer and freedle (1984) classified the five text structures described by meyer (1975) on a continuum reflecting the number and ordering of required schematic components contained within each structure. while some structures are less structured, such as collection and description, other structures are more structured, like causation and problem-and-solution. meyer and freedle found that more organized structures like comparison and causation were more facilitative of recall (in terms of number of ideas and their organization) than less organized structures like collection. other researchers have produced similar findings regarding the influence of less and more organized text structures. sanders and noordman (2000) and spooren, mulder, and hoeken (1998) found that problem-and-solution was associated with better performance on sentence recognition tasks in comparison to a listing structure (for spooren et al., 1998 this only occurred with texts in which the structure was marked). similarly, wylie and mcguinness (2004) found that the least structured text examined, generalization, was associated with lower recall of main ideas than were the more structured texts, such as comparison. previous research on the effect of type of structure on children’s knowledge and use text structure has produced mixed findings. englert and hiebert (1984) compared 3rdand 6th grade readers’ sensitivity to description, enumeration, sequence, and comparison text structures using a judgment task in which readers were asked to rate how well a list of individual differences in children’s knowledge / ray & meyer 71 sentences fit with the stimulus sentences provided. the researchers found that children had better judgments with the collection structure than the comparison, with performance lowest for comparison and description structures. this difference was present on targets (sentences that fit the structure) but not distractors (sentences that did not fit the structure) (englert & hiebert, 1984). similarly, in their study of 4th -, 6th-, and 8th-graders’ recalls, smith and hahn (1989) found that comparison was used less frequently to organize recall than enumeration and description. in both of these cases, levels of structural awareness were related to the age of the readers; older readers demonstrated greater knowledge of the comparison structure, than younger readers. this relationship will be further discussed in a later section. other researchers have shown patterns of children’s awareness that are somewhat similar to the patterns found with adults. richgels, mcgee, lomax, & sheard (1987) found that 6th grade readers were more sensitive to the comparison structure than the other structures examined (collection, causation, problem-and-solution). they found that causation texts posed the greatest challenge to 6th-grade readers; performances on a matching task as well as organizational ratings of recalls and compositions were lowest for causation (richgels et al., 1987). similarly, yochum (1991) found that 5th graders recalled more ideas when reading comparison texts than when reading attribution (collection) texts. however, this effect for text structure was not found on a comprehension test. these findings suggest that like adults, children may benefit from more organized text structures. however, unlike adults, children’s knowledge of text structure may still be developing, and those structures that are most structured like the causation structure, may pose a challenge. in addition to the overall rhetorical structure, textual signaling devices that explicitly indicate the structure of the text may also influence readers’ abilities to recognize and use the text structure. numerous studies which have examined the influence of text signaling on adult comprehension have demonstrated that signaling is associated with a greater number of ideas recalled and more organized recalls (kardash & noel, 2000; lorch & lorch, 1985; lorch, lorch, & inman, 1993; ritchey, schuster, & allen, 2008; for an extensive review of text signaling see lorch, 1989). research on younger readers’ sensitivity to signaling reveals similar effects. ohlhausen and roller (1988) in their examination of younger readers’ sensitivity to structural and content schemas found that texts which contained elements highlighting both the structural and content organization were associated with a greater number of correct main ideas identified in comparison to those texts which emphasized either the structure or the content. moreover readers exposed to a text which emphasized the text structure were better able to identify the structural organization (ohlhausen & roller, 1988). similarly, rossi (1990) found that structural schemas which highlighted macrosentences and/or the overall rhetorical structure were associated with higher performance on a test of comprehension and a larger number of text propositions produced in 5th-grade readers’ summaries. the effectiveness of signals has been found to vary in relation to the overall comprehension skill of the reader (meyer et al., 1980). nevertheless, previous research suggests that providing indices of the top-level structure of texts can facilitate readers’ structural awareness by making the structure more explicit. the relationship between reader and text characteristics in this section, we will examine previous research which has investigated sources of individual differences (reading ability, age, prior knowledge) in structural awareness and their relationship to text characteristics. although we will address these sources individually, many studies consider multiple reader and text influences simultaneously. table 1 contains a international electronic journal of elementary education 72 description of studies which have examined individual differences in younger readers’ knowledge of text structure. table 1. reader and text influences on structural knowledge authors date reader text text structure(s) danner 1976 age hq description taylor 1980 age, ca attribution meyer et al. 1980 ca, sa s comparison, problem-solution mcgee 1982 age, ca description taylor & samuels 1983 sa hq loman & mayer 1983 ca s englert & hiebert 1984 age, ca ts description, enumeration, sequence, comparison garner et al. 1986 age hq garner & gillingham 1987 age hq englert & thomas 1987 age, ld ts description, enumeration, sequence, comparison richgels et al. 1987 none ts collection, comparison causation, problem-solution ohlhausen & roller 1988 age s description smith & hahn 1989 age ts description, enumeration sequence, and comparison rossi 1990 ca s problem-solution yochum 1991 pk ts attribution, comparison vauras et al. 1994 age, ca description, functional/causal armand 2001 pk ts causation+collection/ casuation+comparison ray 2011 age, ca ts comparison, problem-solution note. reader variables: ca= comprehension ability, ld= learning disability, pk= prior knowledge, sa= structural awareness level; text variables: hq = hierarchical quality, s= signaling, ts= text structure individual differences in children’s knowledge / ray & meyer 73 reading comprehension skill readers’ overall comprehension abilities are associated with their levels of structural awareness. in general, previous research has found that readers classified as skilled comprehenders have greater structural awareness in comparison to less skilled readers (englert & hiebert, 1984; hiebert, englert, & brennan, 1983; mcgee, 1982; meyer et al., 1980; ray, 2011; taylor, 1980).within the same grade, skilled comprehenders recall more ideas after reading and produce recalls that more closely reflect the author’s organization (mcgee, 1982; meyer et al., 1980; ray, 2011; taylor, 1980). in addition, englert and hiebert (1984) found that on rating tasks, highly skilled readers were better able to identify statements that matched the structure of the stimulus sentences and to exclude those that did not. these findings suggest that skilled readers are more likely to have knowledge of text structure and to approach text with a “structure strategy”, and as a result, they are more likely to establish a well-organized mental representation of the text. this organized mental representation in turn improves recall. however, the benefit on recall may be restricted to those ideas located at the topic of the hierarchical structure with little difference among comprehension skill groups on recall of details (mcgee, 1982). this difference may reflect structural awareness, as text structure is likely most facilitative in retrieval of macro-level propositions (britton et al., 1980). although overall comprehension ability has been associated with greater structural awareness, the nature of this relationship remains unclear. it not clear whether greater comprehension skill causes higher levels of structural knowledge, or whether structural knowledge contributes to improvements in overall comprehension skill. intervention research (e.g., meyer et al., 2010) does indicate that instruction of text structure is associated with improved performance on standardized tests of reading comprehension. thus, it is possible that structural knowledge can predict as well as affect readers’ comprehension abilities. while the relationship between comprehension skill and structural awareness appears to be consistent across text structures (englert & hiebert, 1984; meyer et al., 1980), the effect of text signals may vary in relation to the overall comprehension skill of the reader. while some studies have found that both skilled and less killed readers benefit from explicit indicators of text structure (loman & mayer, 1983; rossi, 1990) others have not (meyer et al., 1980). meyer et al. (1980) found that text signaling benefited immediate recall for readers classified as “underachievers” (readers for whom a discrepancy existed between word reading and text comprehension) but did not have an effect on readers classified as high and low comprehenders. this benefit for signaling was found for only one of the two passages students read, a problem-and-solution text (meyer et al., 1981). there was no effect for signaling on the comparison text read. although rossi (1990) found that reading text with signaling helped both skilled and less skilled readers in comparison to reading texts with no signaling, skilled and less skilled readers differed with respect to which types of signaling were most beneficial. rossi examined the effect of multiple signaling conditions in which the presence of two types of signaling, underlining of macrostructure sentences (topic sentences) and headings which indicated the text structure, were manipulated. skilled readers performed better with signaling only in those signaling conditions in which the macro-sentences were underlined, while less skilled readers performed similarly across signaling conditions (rossi, 1990). the relationship between signaling and reading ability is complex. although text signaling may help readers to apply their knowledge of structure to a particular text, the international electronic journal of elementary education 74 effectiveness of these signals is relative to the readers’ need and ability to make use of them. highly skilled readers may not need text signaling (or certain text signals) in to order apply structural knowledge, while readers with particularly low comprehension skills may have such low levels of structural knowledge that they cannot take advantage of these signaling devices. age knowledge of expository text structures and structural properties of texts may increase with age as children gain more exposure to expository texts. studies which have compared differences in age groups’ awareness of the various expository text structures have found that levels of awareness increased with age (danner, 1976; englert & hiebert, 1984; englert & thomas, 1987; mcgee, 1982; ray, 2011; smith & hahn, 1989; taylor, 1980). in general, older readers recall more ideas after reading (danner, 1976; mcgee, 1982; ray, 2011; taylor, 1980), produce recalls which more closely reflect the author’s organization (danner, 1976; mcgee, 1982; ray, 2011; smith & hahn, 1989; taylor, 1980), and perform better on rating tasks measuring the identification of text structure (englert & hiebert, 1984; englert & thomas, 1987). this increase with age may be related to the relative ages of the readers. ray (2011) compared 4th-, 6th-, 7thand 9th-grade readers’ awareness of problem-and-solution and comparison texts using written recalls (problem-and-solution) and main idea statements (comparison). although structural awareness improved with age (particularly from 4th to 6th grades), fewer significant differences in performance were found among older readers (grades 6-9), suggesting that while structural awareness increases with age, there may be periods where little change occurs. lack of change was not a result of mastery of knowledge about text structure in the higher grades because similar to meyer et al. (1980) about 50% of the 9th-graders gave evidence for good use of the problem-and-solution text structure. age may also predict readers’ knowledge of other structural properties of expository texts. both younger and older readers have knowledge of how texts should be organized, but older readers may have a more sophisticated and developed sense of organization. danner (1976) found that even with prompting few 2nd-graders could attribute differences between ill-structured and well-structured passages to the text structure, while with prompting 4thand 6th-graders could describe the organizational differences. similarly, ohlhausen and roller (1988) found that 9th-graders were more likely than 5th-graders to provide structural reasons for the selection of main ideas in an underlining task. in their study of 3rd-, 5th-, and 7th-grader’s knowledge of structure, garner et al., (1986) asked readers to complete a paragraph recognition task (identify a paragraph) and a paragraph construction task in which readers were given a set of related and unrelated sentences. although all readers could identify segments of text as paragraphs, construct paragraphs which included topically related sentences, and place topic sentences appropriately, there were age differences in some areas of paragraph construction. seventh grade readers were better at excluding unrelated sentences and establishing cohesion by organizing related sentence pairs to be adjacent. in a similar study, garner and gillingham (1987) asked 5th and 7th graders to compose a good and bad paragraph using a set of given sentences and to provide a verbal report of their decisions. these sentences included intrusions (sentences unrelated to the topic). seventh graders scored higher on verbal reports of topic relatedness and superordination (placement of topic sentence) (garner & gillingham, 1987). however, age differences in performance on topic relatedness and superordination measures for good paragraphs showed no significant differences between the groups (garner & gillingham, 1987). although younger readers used superordination and topic grouping to organize paragraphs, they were less likely to report it. individual differences in children’s knowledge / ray & meyer 75 age and text structures in addition to awareness of the hierarchical relationships between ideas, knowledge of various types of text structures may increase with age. englert and hiebert (1984) found an interaction between grade and text structure on distractor items (ability to determine that a sentence did not fit the structure of the stimulus sentences). for 6th-graders there was no difference in performance across text structures, while 3rd-graders had lower ratings of distractors for sequence and description in relation to comparison (englert & hiebert, 1984). for 3rd-grade readers, their ability to detect that a sentence did not belong varied as function of the text structure, but for 6th graders, it did not. smith and hahn (1989) compared readers in grades 4, 6, and 8 on oral recalls of enumeration, comparison, and, sequence and found that all readers used the enumeration and description structures to organize their oral recalls. in contrast, few 4th and 6th graders used the comparison text structure after reading comparison texts, while many 8th-grade readers did (smith & hahn, 1989). overall, previous research suggests that as readers age, their knowledge moves across the continuum that meyer and freedle (1984) proposed with students gaining knowledge of more organized structures like comparison. these more organized structures may provide more knowledge hooks to improve processing and memory for readers who can use the structures (meyer & freedle). age and comprehension skill structural awareness appears to increase as children age; however, previous research also indicates that the development of structural awareness may be constrained by the overall comprehension ability of the reader. taylor (1980) examined the recalls of 6th-grade skilled and less skilled readers as well as 4th graders. at immediate recall, 6th-grade readers recalled more ideas than 4th graders, with no difference between reading skill groups. however, at delayed recall, skilled 6th-grade readers recalled more ideas than less skilled readers and 4thgrade readers, with no significant difference between the latter groups (taylor, 1980). readers performed similarly on measures of organization of recall. at immediate post-test there was no difference between grades, while at delayed recall, more skilled 6th-grade readers organized their recalls according to the structure of the text in comparison to the other groups, with no significant difference between 4th graders and less skilled 6th graders (taylor, 1980). at delayed recall only those 6th-grade readers classified as skilled demonstrated higher levels of structural awareness in comparison to the 4th-grade group. research on the development of text processing, has also revealed that increases with age may be related to the overall comprehension skill of the reader. in a longitudinal study, vauras, kinnunen, & kuusela (1994) compared changes in text processing strategies in grades 3 through 5 for readers classified as high, middle, and low. the researchers found that all readers improved in their organization of recalls according to the text structure, but for low readers these differences were not statistically significant (vauras et al., 1994). high and middle groups of readers improved in both local (e.g., integrating ideas found in adjacent sentences) and global coherence (e.g., integrating across paragraphs), but low readers significantly improved only in local coherence (varuas et al., 1984). moreover, in comparing high and low readers, vauras et al. found that high readers made larger gains in structural processing than low readers. taken together, these findings suggest that development of readers’ structural awareness is related to their overall reading ability, while skilled readers acquire the ability to approach text with a structure strategy, less skilled readers may continue to approach text with a default/list strategy. international electronic journal of elementary education 76 other research suggests that while reading skill may predict differences within grades, it does not predict difference across grades. mcgee (1982) compared recalls of skilled and less skilled 5thand skilled 3rd-graders. mcgee found that both high and low skilled 5th-grade readers recalled more ideas than 3rd-grade readers. the structure of recalls also improved across groups, with skilled 5th-graders demonstrating use of the authors’ structure, less skilled readers showing partial use, and 3rdgraders showing no use of the author’s structure (mcgee, 1982). similarly, englert & hiebert (1984) and ray (2011) failed to find a significant interaction between reading ability and age. however, ray (2011) did find that when the overall comprehension ability of the grades compared, in this case 6thand 7th-grade, were not significantly different, no significant difference in performance on structural awareness measures was found, suggesting that increases in structural knowledge are related to the overall comprehension skill of the reader. nevertheless, these findings suggest that structural awareness increases with age for both skilled and less skilled readers. findings from the longitudinal study by vauras et al. (1994) suggest the need for more longitudinal studies focusing on growth in structural awareness across elementary and middle school years comparing below grade-level vs. grade level and above readers. prior knowledge in comparison to reading ability and age, relatively few studies have investigated the influence on use of text structure of younger readers’ prior knowledge about the topic domain of a text. readers’ prior knowledge of text content may influence the extent to which readers use text structure and benefit from indicators of text structure. research with adults suggests that prior knowledge influences processing of the hierarchical structure of expository texts (birkmire, 1985) and the relative benefits of text signaling (lorch & lorch, 1996). in relation to text signaling, lorch and lorch (1996) found that signaling was less effective when text topics were familiar to readers, than when they were unfamiliar. this finding suggests that readers who have prior knowledge may be less reliant on cues to the text structure. in their study of prior knowledge and text structure, wylie and mcguinness (2004) found that both low and high prior knowledge readers benefited from reading texts with more structure (comparison, sequence, classification, and enumeration) than for the least structured texts (generalization). when comparing relative performance of these more structured texts, wylie and mcguinness found that high prior knowledge readers recalled more ideas with comparison, but there were no differences in recall across these four text structures for low prior knowledge readers. although both groups used structural knowledge, when prior domain knowledge was low readers may have been less successful in applying structural knowledge about texts. overall research of adults’ text processing suggests that readers’ application of structural knowledge is related to their prior content knowledge. research findings regarding the relationship between children’s prior domain knowledge and text structure have been mixed. armand (2001) examined the relationship between french 6th-grade readers’ prior topic knowledge and text structure in their influence on text comprehension. the author compared the performance of readers with high and low prior knowledge on the comprehension of texts with a combined text structure of causation plus comparison (causative agent acid rain affecting numerous attributes [i.e., bark] of two kinds of trees discussed attribute by attribute with comparative signaling comparing each tree for each attribute) versus causation plus collection (causative agent acid rain affecting two types of trees with the trees discussed one at a time going through each tree’s attributes before discussing the second tree and its attributes). the second presentation method was thought to be less complex and less demanding on processing. the findings were complex because they varied with the classification of questions (e.g, recall [open ended] vs. recognition individual differences in children’s knowledge / ray & meyer 77 [closed multiple choice question]). the open-ended questions showed that for high knowledge students, the causation plus comparison yielded better results than causation plus collection, while the opposite was true for low knowledge students. armand also found a significant interaction between prior knowledge and text structure for the multiple-choice questions. however, for high knowledge readers there was no difference in performance across text structures, whereas low prior knowledge readers had better performance with the causation plus collection structure. these findings suggest that prior knowledge mitigates the demands of reading texts containing two, challenging yet more organized, structures for younger readers. yekovich, walker, ogle, and thompson (1990) found that high domain knowledge (i.e., football) enabled low aptitude students to handle causal relationships and other types of inferences above what was expected based on their aptitude test scores. in the armand study high prior knowledge enabled 6th graders to process ideas from text in the more complex causal and comparative text, while those with low prior knowledge found the complex causal and comparative text too difficult. students with low prior knowledge could better understand the less processing intense causation plus collection with its attributes grouped in a collection related to each tree. similar to wylie and mcguinness (2004), armand’s finding also suggests that low prior knowledge readers may be less able apply structural knowledge when the content demands are high. the causal structure had been found to be particularly difficult with 6th graders (richgels et al., 1987). in the armand study with high prior knowledge, text structure differences did not matter for recognition, but only with the deeper processing needed for recall questions. some text structures may be unfamiliar, or too difficult for a certain grade levels or knowledge or skill levels within a grade level. both the armand as well as the wylie and mcguinness (2004) study discussed above suggest that with high domain knowledge, the comparison structure can be helpful for deeper understanding. unlike the more complex texts studied by armand (2001), yochum (1991) examined the effect of attribution (collection of descriptions) vs. comparison structures on 5th-graders’ recall and performance on comprehension questions. yochum failed to find an interaction between text structure and prior knowledge. both prior knowledge and text structure predicted the number of ideas recalled, while only prior knowledge predicted comprehension test scores. neither prior knowledge nor text structure was associated with the structure of recalls. this finding suggests that while both prior knowledge and text structure influence comprehension, prior knowledge does not moderate the effect of text structure. however, one should be cautious about drawing this conclusion because the structure of the text did not influence organization of recall. previous studies have indicated that readers’ of this age have varying levels of sensitivity to these structures (e.g. richgels, et al. 1987). few studies and inconsistencies in findings make it difficult to draw substantive conclusions about the relationship between prior knowledge about text content and/or text structure and measures of structural awareness. this probably relates to the complexity of the interactions between the reader variables of domain knowledge, structure strategy knowledge with different types structures, general reading skills, and the particular text to be read as well as the task demands. nevertheless prior knowledge may impact readers’ need to avail themselves to the affordances of text structures. additionally, the ability to apply knowledge of text structure may vary with the type of text structure used in a text (e.g., causal vs. comparative structures). summary of research findings international electronic journal of elementary education 78 previous research indicates that the interaction between the knowledge and skills of the reader and the structural characteristics of the text influences readers’ structural awareness. older readers possess knowledge of a larger set of text structures in comparison to younger readers. in developing awareness of expository text structures, readers’ move from a list like/default strategy to a “structure strategy” as proposed by meyer et al. (1980). this development of structural knowledge reflects the continuum of structural organization proposed by meyer and freedle (1984). less organized structures are likely acquired before more organized and more organized structures. previous research indicates that by 6th-grade readers demonstrate awareness of a variety of text structures (richgels et al., 1987, englert & hiebert, 1984). development of structural awareness may differ for less skilled readers, as lower overall comprehension skills may hinder these readers’ abilities to acquire greater levels of structural awareness. however, additional research is needed to further explore the influence comprehension skill has on the development of structural awareness. moreover, even if readers have structural knowledge, the content knowledge demands of a text may inhibit their ability to apply structural knowledge. alternatively, highly familiar domains for readers may make the affordances of text structure unnecessary. ultimately readers’ ability to perceive and use the authors’ structure is the result of their ability to meet both the structural and the content demands of a text. although many studies have examined sources of individual differences, few studies have examined multiple sources simultaneously (see table 1). in particular, few studies have examined the influence of comprehension skill on multiple text structure structures (englert & hiebert, 1984; meyer et al., 1980). although several studies examined age differences, few studies have examined the relationship between age and comprehension skill (e.g. taylor, 1980; mcgee, 1982; vauras et al., 1994). only one study in this review examined the influence of age and comprehension ability on awareness of several text structures (englert & hiebert, 1984). moreover, few studies have examined the influence of younger readers’ prior knowledge on the use of expository texts structures (armand, 2001; yochum, 1991). even though several studies have examined multiple text structures, few of these included the causation and problem-and-solution structures. none of the studies reviewed have examined the relationship between all of the sources of individual differences (age, reading ability, and prior knowledge). additional research is needed to explore these complex relationships, using multiple text structures, particularly more organized text structures like causation and problem-and-solution as these may be more sensitive to changes in reader characteristics. a closer examination of how sources of individual differences vary across reading situations or time in longitudinal designs will provide a clearer picture of students’ needs. instructional implications most elementary school readers would likely benefit from explicit instruction in the use of expository text structures. all students, but particularly younger readers, need instruction in more organized text structures like comparison, causation, and problem-and-solution. not only do these structures provide maximal benefits for memory of expository texts, these structures may also pose the greatest challenge to younger readers. despite age increases in structural awareness, previous research indicates that later elementary and middle school students continue to find these structures difficult (richgels et al., 1987; meyer et al., 1980). students in elementary grades may also benefit from instruction in the hierarchical structure of expository texts. although this review did not include writing research, it appears from research which has used text construction (garner et al., 1986; garner & gillingham, 1987), that younger readers may need more practice in writing expository texts. individual differences in children’s knowledge / ray & meyer 79 in addition, less skilled readers are likely in the greatest need of intensive and explicit instruction in text structure, including instruction in text signaling. in an intervention study, meyer et al. (2010) found that an interaction between type of structure strategy instruction (elaborated feedback with scaffolding vs. just giving information about correctness of answers) and reading ability (below grade level vs. grade level or above) predicted 31% of the variance in who would make large gains in competency using the problem-and-solution structure before and after instruction. below-grade-level readers were more likely to jump from no awareness of the problem-and-solution structure to competency using the problem-and-solution structure if they received elaborated feedback with modeled responses. however, the feedback condition did not make a difference for better readers. poorer readers improved only with elaborated feedback, while better readers improved with structure strategy instruction with both types of feedback. in general, previous research which has examined development of structural awareness, suggests that these students may fail to develop competency in structural awareness without explicit instruction. these readers may also benefit from instruction in text signaling as they may be less sensitive to these devices. teachers need to carefully select texts when teaching readers how recognize and use expository text structure. texts which are well organized and clearly reflect the structure being taught will help readers to apply structural knowledge. teachers should also consider readers’ prior knowledge of the text topics. texts which contain difficult and unfamiliar material may pose a challenge to readers’ ability to apply structural knowledge. however, once readers gain greater levels of structural awareness, they should also be provided opportunities to read a variety of texts. finally, because readers vary in their knowledge and ability to use expository text structures, it is important for teachers to assess reader’s levels of structural awareness prior to instruction. awareness studies have relied heavily on recalls, usually written. while these are easy measures to administer they may be challenging for teachers to analyze and interpret. additional research is needed on classroom-based assessments of children’s structural awareness. conclusion knowledge of text structure may help younger reader to overcome the difficulties of reading expository texts. however, in order to gain competency in recognizing and using expository text structure, many students will likely require explicit instruction. in designing effective, text structure instruction, it is important to understand the needs of individual readers. unfortunately, the individual needs of a particular reader may not always be clear given the complex nature of the relationship between reader and text. additional research which examines the simultaneous contributions of a variety of influences may be able to provide much needed insights into meeting the needs of diverse groups of learners. • • • melissa n. ray is a doctoral candidate in educational psychology at the pennsylvania state university. her primary research interests are reading comprehension, literacy development, and literacy instruction. bonnie j. f. meyer, professor of educational psychology at the pennsylvania state university, studies reading comprehension, decision-making, text structure, and the structure strategy. she has over 50 publications about the structure strategy and its effects on understanding and applications for reading or writing. her current research, funded by the institute for education sciences of the u.s. international electronic journal of elementary education 80 department of education, is focused on structure strategy instruction for students in grades 4-8. she has served on editorial boards for discourse processes, the journal of educational psychology, the reading research quarterly, cognition and instruction, the educational psychologist, and the journal of literacy research and is a fellow in the american psychological association, american educational research association, and american psychological society. references alvermann, d. e., & hague, s. a. (1989). comprehension of counterintuitive science text: effects of prior knowledge and text structure. the journal of educational research, 82, 197-202. armand, f. (2001). learning from expository texts: effects of the interaction of prior knowledge and text structure on responses to different question types. european journal of psychology of education, 16, 67-86. armbruster, b. b., anderson, t. h., & ostertag, j. (1987). does text structure/summarization instruction facilitate learning from expository text? reading research quarterly, 22, 331-346. best, r. m., floyd, r. g., & mcnamara, d. s. (2008). differential competencies contributing to children's comprehension of narrative and expository texts. reading psychology, 29, 137-164. birkmire, d. p. (1985). text processing: the influence of text structure, background knowledge, and purpose. reading research quarterly, 20, 314-326. britton, b. k., meyer, b. j., hodge, m. h., & glynn, s. m. (1980). effects of the organization of text on memory: tests of retrieval and response criterion hypotheses. journal of experimental psychology: human learning and memory, 6, 620-629. coté, n., goldman, s. r., & saul, e. u. (1998). students making sense of informational text: relations between processing and representation. discourse processes, 25, 1-53. danner, f.w. (1976). children's understanding of intersentence organization in the recall of short descriptive passages. journal of educational psychology, 68, 174-183. duke, n. k. (2000). 3.6 minutes per day: the scarcity of informational texts in first grade. reading research quarterly, 35, 202-224. englert, c. s., & hiebert, e. h. (1984). children's developing awareness of text structures in expository materials. journal of educational psychology, 76, 65-74. englert, c. s., & thomas, c. c. (1987). sensitivity to text structure in reading and writing: a comparison between learning disabled and non-learning disabled students. learning disability quarterly, 10, 93-105. frederiksen, c. h. (1975). representing logical and semantic structures of knowledge acquired from discourse. cognitive psychology, 7, 371-458 garner, r., & gillingham, m. g. (1987). students' knowledge of text structure. journal of reading behavior, 19, 247-259. garner, r., alexander, p., slater, w., hare, v. c., smith, t., reis, r. (1986). children's knowledge of structural properties of expository text. journal of educational psychology, 78, 411-416. goldman, s. r., rakestraw j. a. (2000). structural aspects of constructing meaning from text. in m. l. kamil, p. b. mosenthal, p. d. pearson & r. barr (eds.), handbook of reading research: volume iii (pp. 311-335). mahwah, new jersey: lawrence erlbaum associates. hiebert, e. h., englert, c. s., & brennan, s. (1983). awareness of text structure in recognition and production of expository discourse. journal of reading behavior, 15, 63-79. kardash, c. m., & noel, l.k. (2000). how organizational signals, need for cognition, and verbal ability affect text recall and recognition. contemporary educational psychology, 25, 317-331. kintsch, w. (2004). the construction-integration model of text comprehension and its implications for instruction. in r. b. ruddell, & u. j. norman (eds.), theoretical models and processes of reading (5th ed., pp. 1270-1327). newark, de: international reading association. kintsch, w., & yarbrough, j. c. (1982). role of rhetorical structure in text comprehension. journal of educational psychology, 74, 828-834. loman, n. l., & mayer, r. e. (1983). signaling techniques that increase the understandability of expository prose. journal of educational psychology, 75, 402412. individual differences in children’s knowledge / ray & meyer 81 lorch, r.f. (1989). text-signaling devices and their effects reading and memory processes. educational psychology review, 1, 209-234. lorch, r. f., & lorch, e. p. (1985). topic structure representation and text recall. journal of educational psychology, 77, 137-148. lorch, r. f., lorch, e. p., & inman, w. e. (1993). effects of signaling topic structure on text recall. journal of educational psychology, 85, 281-290. lorch, r.f., & lorch, e.p. (1996). effects of headings on text recall and summarization. contemporary educational psychology, 21, 261-178. mcgee, l. m. (1982). awareness of text structure: effects on children's recall of expository text. reading research quarterly, 17, 581-590. mcnamara, d.s., kintsch, e. songer, n.b., & kintsch, w. (1996). are good texts always better? interactions of text coherence, background knowledge, and levels of understanding in learning from text. cognition and instruction, 14, 1-43. meyer, b. j. f. (1975). the organization of prose and its effects on memory. amsterdam: north holland. meyer, b. j. f. (1985a). prose analysis: purposes, procedures, and problems. in b. k. britton, & j. black (eds.), analyzing and understanding expository text (pp. 11-64, 269 304). hillsdale, nj: erlbaum. meyer, b. j. f. (1985b). signaling the structure of text. in d.h. jonassen (ed.), the technology of text (pp. 64–89). englewood cliffs, nj: educational technology. meyer, b. j. f. (2003). text coherence and readability. topics in language disorders, 23, 204-224. meyer, b. j. f., brandt, b. j., & bluth, g. j. (1980). use of top-level structure in text: key for reading comprehension of ninth-grade students. reading research quarterly, 16, 72-103. meyer, b. j., & freedle, r. o. (1984). effects of discourse type on recall. american educational research journal, 21, 121-143. meyer, b. j. f., middlemiss, w., theodorou, e., brezinski, k. l., mcdougall, j., & bartlett, b. j. (2002). effects of structure strategy instruction delivered to fifth grade children using the internet with and without the aid of older adult tutors. journal of educational psychology, 94, 486-519. meyer, b. j. f., & rice, g. e. (1982). the interaction of reader strategies and the organization of text. text, interdisciplinary journal for the study of discourse, 2, 155-192. meyer, b. j. f., & rice, g. e. (1989). prose processing in adulthood: the text, the reader, and the task. in l. w. poon, d. c. rubin, & b. a. wilson (eds.), everyday cognition in adulthood and later life. (pp. 157-194). new york: cambridge university press. meyer, b. j. f., wijekumar, k., middlemiss, w., higley, k., lei, p., meier, c., & spielvogel, j. (2010). webbased tutoring of the structure strategy with or without elaborated feedback or choice for fifthand seventh-grade readers. reading research quarterly, 45, 62-92. meyer, b. j. f., wijekumar, k., & lin, y. -c. (2011). does individualization of web-based structure strategy instruction increase fifth graders’ reading comprehension. journal of educational psychology, 103, 140-168 ohlhausen, m. m., & roller, c. m. (1988). the operation of text structure and content schemata in isolation and in interaction. reading research quarterly, 23, 70-88. ozuru, y., dempsey, k., & mcnamara, d. s. (2009). prior knowledge, reading skill, and text cohesion in the comprehension of science texts. learning and instruction, 19, 228-242. rand reading study group. (2002). reading for understanding: toward an r&d program in reading comprehension. santa monica, ca: rand. retrieved april 27, 2009 from http://www.rand.org/multi/achievementforall/reading/readreport.html ray, m.n. (2011). the development of structural awareness: the relationship between age, skill, and text (unpublished master’s thesis). the pennsylvania state university, university park. richgels, d. j., mcgee, l. m., lomax, r. g., & sheard, c. (1987). awareness of four text structures: effects on recall of expository text. reading research quarterly, 22, 177-196. ritchey, k., schuster, j., & allen, j. (2008). how the relationship between text and headings influences readers’ memory. contemporary educational psychology, 33, 859–874. roller, c. m. (1990). the interaction between knowledge and structure variables in the processing of expository prose. reading research quarterly, 25, 79-89. rossi, j. p. (1990). the function of frame in the comprehension of scientific text. journal of educational psychology, 82, 727-732. international electronic journal of elementary education 82 sanders, t. j. m., & noordman, l. g. m. (2000). the role of coherence relations and their linguistic markers in text processing. discourse processes, 29, 37-60. smith, t. f., & hahn, a. l. (1989). intermediate-grade students' sensitivity to macrostructure intrusions. journal of reading behavior, 21, 167-180. spooren, w., mulder, m., & hoeken, h. (1998). the role of interest and text structure in professional reading. journal of research in reading, 21, 109-120. taylor, b.m. (1980). children's memory for expository text after reading. reading research quarterly, 15, 399-411. taylor, b. m., & samuels, s. j. (1983). children's use of text structure in the recall of expository material. american educational research journal, 20, 517-528. vauras, m., kinnunen, r., & kuusela, l. (1994). development of text-processing skills in high-, average-, and low-achieving primary school children. journal of reading behavior, 26, 361-389. van dijk, t. a. & kintsch, w. (1983). strategies of discourse comprehension. new york: academic press. van den broek, p., young, m., tzeng, y., & linderholm, t. (2004). the landscape model of reading: inferences and the online construction of a memory representation. in r. b. ruddell, & u. j. norman (eds.), theoretical models and processes of reading (5th ed., pp. 1249-1269). newark, de: international reading association. voss, j. f., & silfies, l. n. (1996). learning from history text: the interaction of knowledge and comprehension skill with text structure. cognition and instruction, 14, 45-68. weaver, c. a. i., & kintsch, w. (1991). expository text. in r. barr, m. l. kamil, p. b. mosenthal & p. d. pearson (eds.), handbook of reading research volume ii (pp. 230-245). mahwah, nj: lawrence erlbaum associates inc. williams, j. p., hall, k. m., lauer, k. d., stafford, k. b., desisto, l. a., & decani, j. s. (2005). expository text comprehension in the primary grade classroom. journal of educational psychology, 97, 538550. williams, j. p., nubla-kung, a. m., pollini, s., stafford, k. b., garcia, a., & snyder, a. e. (2007). teaching cause-effect text structure through social studies content to at-risk second graders. journal of learning disabilities. special issue: teaching history--in all its splendid messiness--to students with ld: contemporary research, 40, 111-120. wylie, j., & mcguinness, c. (2004). the interactive effects of prior knowledge and text structure on memory for cognitive psychology texts. british journal of educational psychology, 74, 497-514. yekovich, f. r., walker, c. h., ogle, l. t., & thompson, m. a. (1990). the influence of domain knowledge on inferencing in low-aptitude individuals. the psychology of learning and motivation, 25, 259278. yochum, n. (1991). children's learning from informational text: the relationship between prior knowledge and text structure. journal of reading behavior, 23, 87-108. students’ performance calibration in a basketball dibbling task in elementary physical education international electronic journal of elementary education, 2013, 5(2) issn:1307-9298 copyright © iejee www.iejee.com dear iejee readers, one of the contemporary educational issues in our time is to highlight the different aspects of assesment for learning, assesment of learning, high-stakes assessments, and the role of the different rating and assessment strategies that the teachers use. practicing teachers and educational administrators need research based knowledge about the relationship between these educational activities and the students’ learning autcomes. in the first article dr. gallant presents interesting results from a large-scale research in us context. her conclusion is that developmentally appropriate curriculum and curriculum-embedded performance assesment have positive impact on primary students’ academic achievement. in the second article dr. berridge and dr. thomas examined the 79 fiction selections of a reading list for k-2 grades which was sponsored by the educational authorities of the state of indiana, usa. the result of their critical study of indiana reading list is informative and hopefully will have impact on the state’s future actions. the good news for indiana is that “…gender bies was minimal and no presence of racial bias” was observed. the challanging issue however is that “the sponsored reading list is out of date, need to contain more ward winning selections.” furthermore indiana’s sponsored reading list “fails to represent a balance of women authors and selections containing female protogonists.”. the article is both critical and constructive in the sense that dr. berridge and dr. thomas on the one hand pinpoint the proplematic sides of the reading list and on the other hand they show the way how the disfunctional aspects of a such policy can be eliminated. in 2011, iejee devoted a special issue on reading comprehension edited by dr. karen m. zabrucky, georgia state university, usa. we received an overwhelming feedback on that special issue. in this number, dr. mokhtari and dr. niederhauser present a fascinating paper on the role of vocabulary knowledge and syntactic awareness on reading comprehension. the authors present the findings in reading research over the last seven decades concerning the important contribution that vocabulary knowledge has to the students’ reading comprehension. at the same time, they drow our attention on the importance of syntactic awareness for reading comprehension. based on their own research on thirty-two 5th grade students, they found that “…when both vocabulary and syntax were used as predictors, about 68% of the variance in reading comprehension could be predicted.” in their discussions of the litterature and own findings they add the following “these findings indicate that reading requires fundamental knowledge of the meaning of words and fundamental knowledge of the internal structure of sentences.” it will not be a surprise for me if the readers consider this article as one of the best contributions to the field of reading comprehension. the fourth article in this issue is about a country which came into focus of educationists and educational policy makers when pisa (program for international student assessment; an international assessment that measures 15-year-old students' reading, mathematics, and science literacy) results were released for about a decate ago. we are talking about finland, a nordic country with 5.5 million inhabitants. since the year of 2000, the finnish school system became a hot topic. in their informative article dr. uusiautti, dr. paksuniemi and dr. määttä give the reader a historical overview of the development of the finnish elementary education, http://www.iejee.com/ editorial / özerk v changing perceptions on child rearing, parents-school relationship and the teachers and teacher education. as a part of their discussion in the paper they underline this “in finland, the science of education as a key component of teacher education was and is a sustained and consistent tradition”. i think there is an important lesson to be taken from this point when teacher education is on the agenda in many countries. multicultural education in a broad sense is an important contemporary topic in an increasing number of coutries. recently i arranged a seminar on ‘diversity and citisenship education in global times’ at the university of oslo, department of education. it was a great preasure for me to meeting dr. cherry banks and dr. james banks, who gave a wonderful speech about the topic. i had the same feelings when i read dr. iwai’s article. in several circles the concept og multicultural education is not understood or misunderstood. in her well written article she drows on important perspectives originated from wellknown researchers gay, banks & banks and nieto, regarding multicultural education which at a filosophical level is “an idea or concept, an educational reform movement”. multicaltural education is at the same time “ a prosess” which at an operational level is an educational strategy “…to augment diverse students’ academic achievements and assist other students in becoming meaningful members of the global society”. multicultural education “includes the assumption of providing all students with equal opportunities to learn in school”. she presents important findings in related research and her own research, particulary based on cultural diversity awareness inventory, developed and/or revised by several prominent researchers, and findings with regard to teacher candidates’ cultural and diversity awareness. in her discussion of the results of three research questions, she has an curicial massage: it’s important and possible to achieve a better multicultural awareness and attitude toward cultural diversity and multicultural education in teacher candidates and practicing teachers. the sixth article is an experimental study in which gifted education teacher zousel, dr. rule and dr. logan compare concept acquisition and enjoyment of learning about perfectionism under two condition: a) bibliotherapy and b) analysis and construction of cartoons. i have to admit that this article is one the untraditional and at the same time one of the creative article i have ever read. thanks to the authors and the reviewers for giving me the opportunity to be confronted with a such challanging, interesting, innovative and well written article on an important topic. i consciously decided not to make any more comments on the article, than telling the readers about the thoughts that i just got in my mind: cartoons are something to be taken seriously. the creative cartoonist and gifted teacher miranda zousel and professors rule and lohan have more to tell about the cartoons in their article. the last paper in this number of iejee is about a pilot study on the beneficial impacts of psysical activities on children’s academic perpormance autcomes. since the year of 1992, i spent more than three years as visiting schoolar at claremont graduate university, california usa. during my stay, i got the opportunity to visit several schools in different stats in the country. one of the things that stuck me was that many schools did not have physical education as a subject at the school. i was disappointed. different countries have different educational systems and devote different empthasis on pe education. in norway the coutry where i live, the basic school students (6 to 16 years of age) have compulsory 706 hours pe during their 10 years of schooling. in addition to that it’s mandatory that 5th, 6th and 7th grade students are entitled to regular physical activity in addition to the hours allocated to pe each week. the main objective of this psysical activity policy is to facilitate a more varied and active day of school for students. in the upper secondary schools (16 to 19 years of age) the students, regardless program, have about 90 minutes mandatory pe as a part of their education. all the international electronic journal of elementary education vol.5, issue 2, 2013 vi students get grades in pe starting from the year of 13. with this background i found the well written article by dr. erwin, dr. fedewa and dr. ahn very informative and theoretically and empirically well presented. i wish to express my gratitudes to all authors for their scientific contributions. i also want to thank to our reviewers. last but not least, i want to say thanks to dr. turan temur and dr. gökhan özsoy for all the academic advising, professional coordination, and the tireless technical contribution they have done for the publication of this issue of iejee. dr. kamil özerk editor in chief microsoft word iejee_5_1_varela international electronic journal of elementary education, 2012, 5(1), 5-26. issn:1307-9298 copyright © iejee www.iejee.com the reflective experimental construction of meanings about the shape of the earth and the alternation of day and night∗∗∗∗ paulo varela ∗∗∗∗∗∗∗∗ ciec-institute of education, university of minho, portugal received: 2 september 2012 / revised: 5 november 2012 / accepted: 11 november 2012 abstract the purpose of this paper is to describe and analyze the process of construction of meaning about the shape of the earth and the alternation of day and night, which is inherent to the practice of experimental science teaching. this teaching practice was gradually done by the researcher in a 1 st grade class of a portuguese primary school. the class was composed of 18 students, ten girls and eight boys, with ages ranging from six to seven years old. the analysis of the meaning construction process focused on the class diary prepared by the researcher, based on the field notes and audio recordings made during the participant observation in the classroom. the goals of the interpretive analysis of the diary were as follows: a) identifying the students' initial ideas expressed during class about the shape of the earth, b) characterizing the processes that promote the construction of knowledge about the topics under study; c) and presenting the learning that takes place during class. these instances of learning described in the class diary, combined with the results of a true or false questionnaire, suggest that most students developed a good understanding about the shape of the earth and the alternation of day and night. keywords: conceptual development, evolution understanding, parent-child conversation, informal learning environments, science education. introduction the identification of the children’s intuitive ideas about various science topics was, over the last three decades, a powerful research guideline in the field of cognitive science and science education. several studies have demonstrated that children construct, from an early age, intuitive mental models about the shape of the earth and the alternation of day and night that diverge from the scientific model (nussbaum, 1985; vosniadou & brewer, 1992, 1994; fleer, 1997; siegal, et. al, 2004; blown & bryce, 2007; özsoy, 2012). as an example, vosniadou and brewer (1992) identified, in primary school children in the u.s., five alternative mental models of the earth: the ∗ this manuscript and the case study about the shape of the earth and the alternation of day and night are an integral part of the author’s doctoral thesis. ∗∗ paulo varela, centro de investigação em estudos da criança, instituto de educação, universidade do minho, 4710-057 braga, portugal, phone: +351 253601221. e-mail: pibvarela@ie.uminho.pt international electronic journal of elementary education, 2012, vol.5, issue 1, 5-26 6 rectangular earth, the disc earth, the dual earth, the hollow sphere, and the flattened sphere. nevertheless, the vast knowledge produced by such lines of research, especially within science education, has garnered criticism from some constructivists as regards its diminished influence on the improvement of the children’s learning and teaching. matthews (2000) claims that this theoretical knowledge “offers very little guidance for teachers who are in the classroom trying to teach science contents” (2000, p.270). in the same sense, white states that “...although the research on alternative conceptions has sparked interest in the content, it has not yielded clear advice about how to teach different topics” (1994, p.255). in many countries, the primary school science syllabuses, while they recommend activities based on inquiry methods, hands-on activities, dialogues, discussion and collaborative working, do not include any information on how teachers should implement such activities in their approach to the various teaching contents (eurydice, 2011). despite all efforts, in many countries these curricular guidelines still do not bear the necessary influence to change the pedagogical practices of teachers. for example, martínez and díaz (2005), when referring to the spanish educational context, mention that the curricular guidelines for primary education have been advising on more innovative science teaching, based on an active and constructive role for the student. however, the reality in that country's classrooms is quite different, as the authors point out: “studies and polls carried out show that science teaching as a transmission is still predominant, and it is based on blackboard master-classes, on the school textbook and the solving of closed problems related to the studied themes” (2005, p. 243). the promotion of inquiry-based science teaching is a highly demanding challenge, which calls for major changes in the teaching practice (harlen, 2010; brand & moore 2011). indeed, most primary school teachers have insufficient scientific knowledge and are not familiar with these teaching strategies, thus depriving their students from the opportunity to engage in relevant and meaningful learning (harlen, 1999; lee, et. al, 2004; appleton, 2003; brand & moore, 2011). harlen (1999), in analyzing several studies about the teachers’ understanding of science conducted during the 1990s in countries such as the usa, england and scotland, concludes that primary school teachers have a low level of confidence about teaching science and understanding science concepts. these insufficiencies have implications for the students’ learning opportunities, and are commonly associated “with restricting classroom activities to following instructions and inhibiting creativity and questioning” (harlen, 1999, p.81). faced with those limitations, appleton (2003), in a study performed with australian preservice teachers, states that some teachers avoid teaching science or rely on activities with little scientific content, which are usually conducted as a demonstration. other teachers, however, find support in science textbooks and worksheets, whose hands-on activities are typically presented with step-by-step instructions (huber & moore, 2001). the way these activities are generally introduced in textbooks and developed by the teachers does not foster intellectual engagement on the part of the students. instead, it leads to much physical action and little mental activity (harlen, 2007), and it tends to give students a distorted and fallacious view of the nature of science (huber & moore, 2001; levinson, 2002). teachers should build learning opportunities and encourage students to engage in genuine inquiry activities or, as mentioned by jorgenson (2005) in “hands-on minds-on activities”. in portugal, the experience of more than three decades has shown that the introduction of science themes and new approaches to school knowledge construction in the primary education programs has failed to produce any significant effects in the renovation of pedagogical practices and subsequent improvement of the quality of the reflective experimental construction of meanings about the shape of the earth and the alternation of day and night / varela student learning (sá, 2002a). children in primary schools still do not have opportunities to develop the “experimental attitude”, highly praised in the science component of the environmental studies curricular area (ministry of education, 2004), which leads to the neglect of important domains of knowledge construction and skill development that are necessary all across the different curricular areas (varela, 2012). memorization and repetition activities are overrated, and the students keep performing stereotyped, meaningless tasks (roldão, 2003). in this learning environment, the student takes on a passive role, fundamentally limited to the accumulation of knowledge. learning loses relevance, and its personal and social use becomes ineffective. the contact with schools through interventions that we have been conducting in primary school classrooms (sá & varela, 2004, 2007) has allowed us to verify that children do not usually have opportunities to conduct practical or experimental science activities which: potentiate their intellectual, personal and social development; stimulate thinking and conceptual understanding (zohar, 2006); promote language use as a tool for constructing and sharing knowledge (aleixandre, 2003; rivard, 2004; ibáñez & alemany, 2005); stimulate discussion and argumentation around the students' ideas and the experimental evidence they produce (naylor et. al, 2007); promote in students an active, autonomous regulator and reflective role on their own learning (cleary & zimmerman, 2004); and develop positive attitudes towards science learning in children (harlen, 2007). this reality is perceived also by costa, who referring to practical and experimental science activities states the following: “(…) the way that they have been used has contributed nothing at all to the learning of scientific concepts by students, or to the understanding of the processes used by science or even to the acquisition of transversal skills. (…) they are rarely used in ways that develop skills of observation, inference, communication, interpretation and planning. instead, they are more often used as a treat for students "if there is time left" (which rarely happens), or, at best, as an attempt to engage the less motivated students” (2006, p.33). thus, primary school students are rarely involved in a genuine process of scientific meaning construction and development of cognitive resources, which are based on a direct relation to concrete objects, manipulating, feeling, experiencing them and reflecting on the observations they make and the actions they carry out with them. without this knowledge and resources, the students will lack the foundations on which to build essential skills for new forms of learning, which are necessary all throughout the different curricular areas and that will ensure them a participating and informed citizenship in the future (sá & varela, 2007). in this context, we have been developing for over a decade and a half a research and intervention work in the classroom, geared towards an experimental reflective approach of sciences in the early years of schooling (sá, 2002a; sá & varela, 2004, 2007). this paper is situated in the continuity and deepening of this perspective of teaching and research, and its content is a small part of a study conducted by the author (varela, 2012). objectives in this paper the process of meaning construction is described and interpreted through the study of "the shape of the earth and alternation of day and night", aiming at specific purposes: (a) identifying the students' initial ideas about the shape of the earth, (b) identifying and characterizing processes that stimulate classroom construction of meanings about the topic under study, (c) and presenting the learning that takes place during class. international electronic journal of elementary education, 2012, vol.5, issue 1, 5-26 8 reflective experimental science teaching – r est rest places great emphasis on the stimulation of the student's reflective thinking skills, integrating and intensifying, in an interdependent manner, the development of cognitive processes and conceptual comprehension (miras, 2001; sá 2002a; zohar, 2006; harlen, 2007). it is an approach to science teaching in which: “(...) experimental activities are not simple manipulations executed mechanically by imitation, or following instructions provided by the teacher or described in a textbook. on the contrary, they are actions with strong intentionality, closely associated with the student's mental processes. it is this combination of thought and action that leads to higher quality learning” (sá, 2002a, p.47). learning takes on a dynamic and evolving nature of (re)construction of socially constructed meanings, which depart from the ideas that students construct in their personal and sociocultural experiences. when explained in the social context of the classroom, these are subject to a generative and reconstructive process of new meanings with greater power to explain physical and natural phenomena (sá, 2002a; harlen & qualter, 2005; harlen, 2007). learning starts from: “relevant problems and personal ideas that describe and interpret them, in order to gradually construct, through a process of critical contrast with other ideas and with reality phenomena, a school knowledge that is socialized and shared by means of processes of conceptual change and evolution” (porlán, 1998, p.101). in the teaching and learning process, students confront their ideas and expectations with the experimental evidence produced (harlen & qualter, 2005; harlen, 2007) in a methodical, organized and intentional way. the student thus becomes gradually skilled in the process of coordinating personal theories with evidence (kuhn, et. al, 1988), aiming for a progressive harmonization and conformity of the new theories with the physico-natural world. however, the perspective of conformity between theories and experimental evidence is different for each subject, i.e. “the same experience or the same observation are experienced, seen and understood very differently by different children” (charpack, 2005, p.29). for this reason, the meanings constructed by way of physical interaction with materials and objects are the subject of discussion and reflection in small and large groups, so that the critical selection and negotiation leads to higher-level meanings, shared by a growing number of students (naylor et. al, 2007; domínguez & stipcich, 2009). it is in the process of social interaction that the different interpretations of physical experience are confronted, negotiated and reconstructed and it is in that interactive process that the different meanings are defined and refined (candela, 1999). in rest, the creation of collaborative contexts has a particular importance, as they facilitate the emergence and exchange of different meanings and explanatory interpretations for the various learning situations (larkin, 2006) and stimulate the joint construction of scientific meanings (palincsar & herrenkohl, 2002). we thus recognize the importance of promoting spaces for collaborative mediation and negotiation of meanings, which stimulate students to share opinions among them and with the teacher, to defend their points of view and to justify and/or refute the arguments presented (henao & stipcich, 2008). the discussion generated in the classroom provides children with the awareness of their own ideas, the different ideas and ways of thinking that exist in the group (larkin, 2006; harlen, 2007) and the need to review and/or restructure their ideas, in face of other more plausible and consensual ones that appear in the social context of the class (varela, 2012). through this intense collaborative activity, children also learn, by the action of others and the teacher, to monitor and auto-regulate their own thought and gain access to a wider range of problem-solving strategies (mercer & littleton, 2007). the reflective experimental construction of meanings about the shape of the earth and the alternation of day and night / varela thus, the teaching and learning process is aimed at encouraging students to reach the highest limit of their potential, i.e. their “zone of proximal development”, proposed by vygotsky, allowing for the awakening of “a variety of internal development processes that are able to operate only when the child is interacting with people in his environment and in cooperation with his peers” (vygotsky, 1978, p.90). from sociocognitive activity, which takes place alternately in small and large groups, emerges the need for more refined observations of the evidence, as well as for the repetition of experimental procedures, which are accompanied by a more reflective attitude from the student. this attitude brings forth new ideas, propelling the discussion to higher quality thinking levels, inducing metacognitive and learning self-regulation skills in students (larkin, 2006), while also favoring a high degree of transference of the learning acquired to new contexts (georghiades, 2006) and the autonomy of students (gonzález, & escudero, 2007). individual meanings, when explained, reflected upon, contradicted and negotiated will result in a smaller number of meanings, now enriched and shared by a large group of students (sá & varela, 2004). rest lends special importance to the role of oral language as an instrument of communication and construction of scientific meanings (català & vilà 2002; aleixandre, 2003; maloney & simon, 2006). students often resort to written language, which requires greater awareness of the mental operation one executes, developing a process of inner speech within the subject himself (vygstsky, 1987). writing implies thinking about whatever is the object of the writing, organizing ideas, establishing connections between them, selecting the best words and articulating them correctly. indeed, when we encourage students to develop the regular habit of writing about the experimental activities, we are simultaneously giving continuity to the reflective process, promoting the highest level of learning within their reach. rest implies renewed roles for students and teachers. in this teaching practice, students: • explain their ideas and ways of thinking about questions, problems and phenomena; • argue and counterargue among themselves and with the teacher regarding the validity of their ideas and strategies; • mentally construct simplified research plans with their peers; • carry out the resolution plans and strategies for the problem situations they are confronted with; • submit personal ideas and theories to the critical confrontation of their peers and to the test of evidence by resorting to the scientific processes; • keep written records of their observations and evidence data, as an integral part of the exploration of practical and experimental situations; • critically assess the conformity level of their theories, expectations and predictions with the ideas of others and with the experimental evidence they produce; • negotiate different personal perspectives about evidence, questions or problems, aiming for the construction of enriched and socially shared meanings (sá & varela, 2004). the teachers, for their part, take on a role of high activity, reflection and strong pedagogic intentionality: a) in the interpretation of the actions carried out by the students and in the meanings that are generated and reconstructed in the classroom, in order to regulate and re-feed the students’ mental constructive activity; b) in the stimulation and mediation of the students’ interactions with the experimental evidence they produce with their peers; c) in the promotion of an active participation by the international electronic journal of elementary education, 2012, vol.5, issue 1, 5-26 10 students, providing them with the necessary stimulus for verbalization, action and reflection; d) in the valorization and regulation of the discussions that arise around the students' interventions; e) in the creation of an environment of collaboration, accountability and freedom of communication; f) through continuous and recurrent reflective questioning, which stimulates the students' thoughts and actions (varela, 2012). this questioning will provide, at each moment, adequate help to the needs expressed by students in order to escalate to progressively higher levels of thought and learning (rojas-drummond & mercer, 2003; chin, 2006; molenaar, et. al, 2011). method the study is developed according to an action research approach within the theoretical framework of interpretive research, applied to the study of teaching and learning processes in a classroom context (erickson, 1986; guba & lincoln, 2000). a 1st grade class from a portuguese primary school located in the outskirts of the city of braga, composed of ten girls and eight boys (n=18) with an average age of 6.25 years, was subjected to a process of rest. distributed over one school year, 20 lessons were taught addressing various science topics within the curricular area of environmental studies, amounting to a total of 40 hours of intervention in the classroom. each lesson, which corresponds to one action research cycle, begins with a teaching and learning plan that takes the form of a starting “curricular hypothesis” (porlán, 1998) to be implemented flexibly, according to the teaching and learning processes that are generated and promoted in the class reality. the teaching and learning plan pertaining to the curricular topic on the shape of the earth and the alternation of day and night was prepared according to the didactic sequence suggested by vosniadou (1991) and vosniadou et. al, (2004)1. according to these authors, in learning basic concepts of astronomy there should be an interrelation between the understanding of the spherical shape of the earth, the rotation of the earth with regard to the apparent movement of the sun and the explanation of the alternation of days and nights. at the time of the pedagogic intervention on "the shape of the earth and the alternation of day and night", the students had already benefited from the cumulative effect of 32 hours of rest. the classes were taught by the researcher, who, in collaboration with the class teacher, played the role of both a researcher and the teacher. thus, there was an attempt to capture and understand the processes of generating and (re)constructing scientific meanings promoted in the classroom, in a social learning context. the researcher-teacher's attention was especially focused on the interpretation of the meanings manifested by students in the moments of communication, action and interaction with their peers and the researcher, and on how these meanings were being reconstructed and negotiated within the class. the data generated in the action were collected using two complementary methods, namely the fieldnotes made by the researcher and the audio recordings of the lesson. this raw data were later materialized in the form of detailed narratives of the most relevant events that occurred in the classroom – the class diary. these constituted the principal method of recording data and, simultaneously, a strategy of reflection and modeling of the teaching and learning process (sa, 2002b; zabalza, 2004). the lesson, as a referential unit of analysis represented in the class diary, is therefore composed of a sequence of learning moments that correspond to more particular units of analysis. each unit of analysis is the bearer of a specific sense that distinguishes it 1 see the full teaching and learning plan, in portuguese, in sá and varela (2007) or part of the plan, in english, in varela (2012). the reflective experimental construction of meanings about the shape of the earth and the alternation of day and night / varela from other units – “unit of meaning” (ratner, 2002), in the evolving and interactive process of constructing scientific meanings. in each diary we began by identifying the sequence of units of meaning. we then carried out the interpretive analysis of the meaning of the data concerning each identified unit and the definition of its central theme, based on that analysis. the data contained in the diary represent a sample of the diversity of meanings that the students construct in the classroom, while interacting with their peers and the teacher, as well as during the activities conducted. a true or false questionnaire about the “shape of the earth and the alternation of day and night” was therefore applied in order to attain a more reliable perception concerning the level of individual learning achieved by the students. in view of the dynamic character of the constructive process of meanings the range and depth of learning only occurs after some time (coll & martín, 2001) -, the questionnaire was applied at two different moments, i.e., immediately after the lesson and after three weeks. the interpretive content analysis of this class diary is the starting point to interpret the process of teaching and learning promoted in the classroom about “the shape of the earth and the alternation of day and night”. interpretive quality criteria adopted in this study the regular presence of the researcher with the children in the classroom for various periods of time, in an accumulated total of 32 hours until the lesson about “the shape of the earth and the alternation of day and night”, ensures a “prolonged involvement“ (guba & lincoln, 2000). the researcher's prolonged presence affords a progressive construction of a relationship of empathy and openness with the students. this relationship is essential so that the research subjects may share their views with the researcher. in these circumstances, the researcher can access multiple perspectives of meanings from the students’ perspective, and can linger on them, analyzing them in depth and detail in order to derive a better clarification and understanding in the context in which they occur (pérez gómez, 2005). “listening to participants”, proposed by guba and lincoln (1989), can hardly be sustained in this study. the subjects in this study are 1st grade students. from our point of view, it is not feasible to ask six and seven-year-old children to audit the representations that the researcher generates from the meanings they constructed in the teaching and learning process. thus, while maintaining a certain parallelism to “listening to participants”, a process of recurrent "validation” was adopted in the identification of the students’ constructions. this consisted of ascertaining the meaning of what children say and do, at the time and in the context, through a systematic interaction with them. thus, the researcher may put his/her inferences to the test, in a close and situated manner, because he/she is an active subject in the observation context (erickson, 1986). the use of audio recording is a procedure that lends more credibility and veracity to the qualitative data collected. audio recordings afford the researcher greater availability to reflect and interact with the subjects in order to ensure that the meanings referred by them are correctly interpreted and represented (maclean, et. al, 2004). moreover, in situations of ambiguity or inconsistency (uncertainty) regarding the meanings inferred in the classroom, the recourse to a later hearing of the recording can clarify the intended meaning from the original source (fasick, 2001). from our point of view, revisiting the original data, by hearing the recordings at a later time, allows for a novel outlook, distanced from the data itself and the interpretations made in the course of the international electronic journal of elementary education, 2012, vol.5, issue 1, 5-26 12 participant observation, in order to construct a representation of the studied reality as accurate as possible. the combination of the preceding techniques allows for the collection of “abundant information” (carrasco & hernández, 2000). after class, to best take the advantage of fresh memory, the class diary was written on the basis of the audio recordings and the field notes. the diary includes verbatim transcripts of what the children say in meaningful moments of interaction, events of non-verbal nature and emotional aspects not captured by audio recording. it represents the events generated in the classroom by means of a thick description (denzin, 1989). a rich and detailed narrative of the observations increases the credibility and plausibility to an external reader regarding the data and the inferences made. furthermore, the act of creating a narrative increases the confirmability of the study, since it allows us to distance ourselves from our judgments, premature interpretations and provides an opportunity to open our work to the inspection of others (newman, 2000). the analysis of the class diary includes segments of raw data so that an external reader can judge the credibility and neutrality of the inferences made from their meanings (lincoln & guba, 1989; ratner, 2002). this constitutes a relevant factor for the confirmability criterion, i.e. to verify that we are not in the presence of arbitrary constructions imagined by the researcher. results interpretive content analysis of the class diary about the shape of the earth and the alternation of day and night the teaching and learning activities begin by identifying what students think about the shape of the earth. students are asked to draw the shape of the earth. what ideas do children present about the shape of the earth? the interpretation of the meaning of the drawings takes place in two moments: i) in class, through observation, communication and discussion generated around the intended meaning of the drawings; ii) after class, through a more detailed analysis of the content of the drawings and the arguments presented to the class by the students. we identified three categories, whose content represents qualitatively different ideas about the shape of the earth: a. most students' drawings evidence the idea that the earth is flat. at the bottom of the drawing, the earth extends down and to the sides. on the surface, the students drew houses, trees, people, etc. above that, there is the sky and/or space, with some birds, clouds, stars and the sun. this is the most primitive conception of the shape of the earth identified in the class (11/18; 61.1%). b. a second category of drawings apparently considers the idea that the earth is round. however, what is relevant about the sphere on the drawing is what is inside it: i) a well-defined area at the bottom, which is round on the bottom and flat at the surface. that is where the trees, houses and people are; ii) the top part corresponds to the sky and/or space, where some birds and flying insects, the stars and the sun are drawn. although these students claim that the earth is round, it seems very plausible that this model results from the incorporation of the scientific information figure 1. gabriel; 6.5 years figure 2. francisca; 6.4 years the reflective experimental construction of meanings about the shape of the earth and the alternation of day and night / varela space earth space skyspace earth air notion1 notion2 notion3 figure 3. sara; 7.1 years regarding the sphericity of the planet into the previous model. the earth itself would be the bottom part, with the flat surface, whereas the top part would correspond to the sky and/or space. phrases like "our country is inside the earth" suggest that the word "earth" can either take on the meaning of the cosmic body we inhabit, or that of a cosmic entity that contains the earth and the space inside it. reinforcing this interpretive hypothesis is the fact that the expression "inside the earth" does not, in any way, mean "below the earth's crust", but rather "inside" the sphere on the drawing (3/18; 16.7%). this interpretation is further validated by other authors who have identified the same concept in children belonging to the same age group (nussbaum, 1985; vosniadou, et. al, 2004). vosniadou et. al, (2004) call this model "hollow earth", as it is a synthetic model derived from the children's attempts to incorporate the scientific information that says the earth is a sphere into the initial concept that the earth is a supported and stable plane. c. in a third category of drawings, the earth appears as a spherical body surrounded by space, where the stars and the sun are drawn. on its surface there are countries, continents and oceans. for these students, people live on the surface and not "inside" the earth: "it's on the outside"; "people walk up here on the land"; "they also ride boats on the sea"; "and swim and ride water scooters" (4/18; 22.2%). the meaning of the drawing is communicated to the class as follows: "i made the earth round, seen from mars. in space i drew the sun and the stars, and here (on earth) i drew the islands, the seas and people's lands (countries)". such ideas are in line with those identified by nussbaum (1985) in israeli students, aged between 8 and 14 years, about the earth concept: the shape of the earth, space and gravity. the author identified five notions that, from 1 to 5, correspond to a conceptual progress, from the most egocentric and primitive vision to the most decentered and scientific one. notions 1, 2 and 3 consider only the shape of the earth and the nature of the sky/space: our drawing categories (a, b, c) suggest a high level of parallelism with notions 1, 2 and 3 identified by nussbaum (1985). the author found that approximately 80% of children aged eight years, in the 2nd grade, are distributed among notions 1 and 2. figure 4. notions about the shape of the earth and the nature of the sky/space – adapted from nussbaum (1985). international electronic journal of elementary education, 2012, vol.5, issue 1, 5-26 14 a. development of ideas about the shape of the earth: from the flat model to the spherical model. from the previous ideas that emerged in class it is intended that students, in small and large group, reflect on them and submit them, by way of discussion, to critical confrontation before colleagues and before the scientific model (photographs of earth taken from space and earth globe), in order to develop a better knowledge and understanding of the shape of the earth. a1. communication and discussion of the meanings expressed in the drawings. passage from class diary: “i did the earth, which is round” francisca (6.2 years). "francisca put things inside it" (lionel; 6.9 years). "she put butterflies, the sun and the clouds inside it" (júlia; 6.9 years). francisca's drawing falls into category b. mafalda (6.4 years) clearly expresses the meaning of her drawing (category c): "i made the earth round, seen from mars. in space i drew the sun and the stars, and here (on earth) i drew the islands, the seas and people's lands (countries)". gabriel (6.3 years) drew a flat earth (category a): "i did the clouds, a boy, the sun..." he also says: "the earth is under the boy." i ask him if the shape of the earth in his drawing is flat. with a sad look, he says “yes”, but he recognizes that it is not flat: "no. i made it flat like this, but now i think it's round". others also become aware that their drawings were not in agreement with the idea that the earth is round: "luís said that the earth was round, but he didn't make it round" (pedro; 6.3 years); "i didn't either, i thought it was flat" (susana; 6.6 years), "mine is not round" (lionel). oral communication is aimed at sharing and confronting the meanings expressed in the three previous models, so as to subject them to critical review and promote their reelaboration by others. in this process we observe the following: i) the model of pseudo-sphericity of the earth is strongly criticized with a hint of irony: "oh look, she put things inside it; she put in butterflies, the sun and clouds"; ii) the flat earth model, confronted with the spherical model, generates some dissatisfaction and a critical attitude towards their own drawings and those of others, now rendering the notion of sphericity of the earth far more plausible: “i made it flat like this, but now i think it's round"; "luís said the earth is round, but he didn't make it round". a2. the class faced with the spherical model of the earth. the students are unanimous in admitting that the spherical model (category c) is the one that best represents the shape of the earth. however, conflicting thoughts emerge between the most primitive meaning (flat earth model) and the socially accepted meaning (spherical model), which demonstrate the difficulty in reconciling sphericity with the perception of the earth's flat surface resulting from direct observation. b. the earth: development of a more comprehensive and richer meaning. the students' initial ideas and ways of thinking are now subject to confrontation with empirical evidence: a photograph of the earth taken from space and the earth globe. b1. the photograph of the earth and the earth globe. the comments about the observation of the photo begin by focusing on the earth's spherical shape. however, color is a piece of information that stands out in their comments. the shade of blue is identified as the "seas" and, surprisingly, the white spots are associated with clouds and the ice at the poles. this inference requires a certain level of abstraction: in everyday life, students see the clouds from the earth, but the reflective experimental construction of meanings about the shape of the earth and the alternation of day and night / varela now they are identified from another perspective – that of someone who sees the earth from a given location in space. when the students' attention is focused on the globe, most of them know its name and prove to understand that the earth globe represents a miniature of the earth. b2. what are the similarities between the photograph of the earth and the globe? passage from the diary: children state without hesitation: "the shape is the same" (júlia; 6.8 years), "they are both round" (sara; 6.9 years); "it's the shape"; "it's the same" (other children). some also recognize a few differences: "it's just that, in the picture, the blue is darker and there (globe) it's lighter" (gabriel; 6.3 years). "this was taken from very far away" – argues lionel (6.9 years), referring to the different shades of blue. when asked about the differences, they realize the following details: "the lands here (the globe) are neater and we can see them better" (sara); "and here it seems they are more spread out, here (the photo) we can see the clouds" (lionel); "it's as if it were space" – adds sara. the groups easily identify the similarity between the spherical shape of the earth on the photo and on the globe. in that comparison, they also identify some differences between the reality of the earth in the photograph and its representation on the globe model. in the photograph, the earth is distinguished by: i) the darker shade of blue; ii) the lack of identification and contouring of the "lands", i.e. the continents and the countries contained therein; iii) the presence of clouds and the cosmic space around the earth seen in the photograph. b3. a renewed outlook on the drawings of the shape of the earth. passage from the diary: the children's attention is again focused on their drawings. those who drew a flat earth recognize once again that the earth is round: "mine is not round" (lionel; 6.9 years); "the earth here is round (photo) and here it isn't" – says gabriel, pointing at his drawing. i ask the class what they now know about the shape of the earth. they assertively answer that "it is round" and gabriel states that it looks like a ball. "it looks like a ball, but it's always spinning" says júlia. the idea of the earth's sphericity is very mature in the class. for some students, the contrast between that knowledge and the less evolved ideas expressed in the drawings promotes greater awareness of their own learning. only a few verbalize that increased awareness, but by doing it in a social context they are not only consolidating their ideas, but also promoting the intra-personal processes of assimilation of that learning in the other children. verbalization favors the construction of more elaborate formulations of those same ideas, as in júlia's case: "it looks like a ball, but it's always spinning". c. day and night c1. what is day? within the small groups, students are encouraged to think about what day is2. there is a reference to morning as being daytime, an idea that can stem from the portuguese morning greeting "bom dia" (good day). it is by opposition to that idea that they realize that the concept of day includes morning, noon and afternoon. after being questioned again, they now acknowledge that the elements sun and light are subsumed in the 2 it is intended that children develop the notion of day as the period of time during which a given location on earth is illuminated by the sun, i.e., the period between sunrise and sunset (natural day). that notion and the previously acquired meaning of the sphericity of the earth will support the later development of the comprehension of the day-night alternation, as a result of the earth's rotation movement in the presence of the sun. international electronic journal of elementary education, 2012, vol.5, issue 1, 5-26 16 definition of day. it is said that the day begins with the sunrise. some answers seem to contemplate the idea that the sun is always visible during the day. this idea is subject to discussion. students demonstrate an understanding of day as corresponding to the period of time during which the sun illuminates a location on earth, even if it is covered by clouds for whole days. c2. what is night? in answers to questions about what night is, the following ideas emerged: i) they begin by making reference to darkness; ii) the darkness is a consequence of the absence of the sun; iii) the absence of the sun during the night is explained by some children with ideas of an animist nature – the need for the sun to "go away to rest or sleep"3; and iv) in contrast, others claim that the sun stays in space, in a different relative position, illuminating other parts of the planet. this is quite an evolved idea: it acknowledges the simultaneity of day and night in different locations, as a result of the sun's relative position to those places. d. day and night in the earth model – the sun, without the earth's rotation. d1. identification by analogy of what the globe and the flashlight represent. passage from the diary: the children's attention is again focused on the globe on the desk. they have no difficulty in recognizing that "it's the earth in miniature". i show them a flashlight and some immediately associate it with the sun: "it’s the sun" (pedro; 6.2 years); "if you turn it on, it seems like the sun" (lionel; 6.9 years). others also refer to the sun and sara adds: "it will give light to the earth”. the students identify by analogy what each of the objects represents: the globe – "it's the earth in miniature" – and the flashlight – "it's the sun"; "if you turn it on, it seems like the sun"; "it will give light to the earth". d2. elaboration of records. the students proceed to the individual recording of what each object drawn on their record sheet intends to represent, i.e., the earth and the sun. � i write in the boxes what each object represents. (earth) (sun)(earth) (sun) francisca (6.2 years) figure 5. learning record: the globe-flashlight versus earth-sun analogy. 3 it is interesting to verify that this idea has also been identified by other authors. for instance, fleer (1997), in a study with australian aboriginal children aged four to eight years, found a similar conception when, during the interview, she posed the following question to the children: why is it dark at night? some answers explained the occurrence of night with the fact that the sun went away or went to sleep, also relating night to the appearance of the stars and the moon. however, in most of the answers given by children, according to the author, there is an animistic view of the sun. according to the author, this conception tends to reflect expressions that children hear in everyday life (“the sun has already gone away”), or the perception that the sun descends over the horizon as night falls. the reflective experimental construction of meanings about the shape of the earth and the alternation of day and night / varela d3. it is daytime on the part of the earth that is lit by the sun and nighttime on the part of the earth that is not illuminated by the sun. passage from the diary: "imagine the earth is in the dark, what must we do to have daytime on the earth?" – i ask. everyone agrees that we have to turn on the flashlight and some state: "now it's daytime". "in what part of the earth is it daytime?" – i ask. children answer that it is in the part of the globe that is facing the flashlight: "on the part that is lit" (bruno; 6.9 years); "it's the one that has the light" (several). "and on the other side of the earth, what will it be?" – i ask. without hesitation some answer: it is nighttime. mafalda looks pleased to see her previously expressed idea confirmed and states: "on the sun's side it is daytime and on the other it's nighttime, the sun does not get there and it goes dark". d4. collective construction of a sentence about day and one about night. in the discussion, the students show a good understanding of the notion that it is daytime on the part of the earth facing the sun, hence receiving light; and that it is nighttime on the part of the earth that is opposite the sun, which is in the dark. in class discussion, the following sentences are agreed upon about day and night, and are then written on the individual records: "it's daytime on the part of the earth that is facing the sun"; "it's nighttime on the part of the earth that is not facing the sun". e. earth globe: from day to night in portugal. our country is situated in the part of the globe illuminated by the flashlight (sun) and therefore it is day in portugal. e1.what must we do to have nighttime in portugal? passage from the class diary: the children answer: "we must try to turn the sun to the other part" (mafalda; 6.4 years); "turn the sun" (emanuel; 7.2 years); "the sun has to go to the other side" (júlia; 6.8 years); "we must turn the sun" (rui; 6.6 years); "we have to turn off the flashlight and put it on the other part" (joana; 6.7 years); "when it is too sunny, it tilts a little" (joão; 6.8 years). among these answers, gabriel and bruno are the only students to contemplate the possibility of rotating the earth: "we must rotate the earth, the earth is always spinning" (gabriel; 6.3 years); "we must turn the earth" (bruno; 6.9 years). mafalda argues: "we must put the sun on the other part so that night comes over here" – the side where it was daytime. gabriel does not accept mafalda's or the other classmates' ideas and insists: "it's spinning the earth, it's always spinning". i point out that there are two different ideas in the classroom: some say that the sun (flashlight) must move around the earth (globe) and others say that the sun stands still and the earth must spin around itself. gabriel defends his idea before the class once more. for a few moments they remain in silence. "what do the others think?" – i ask. bruno insists: "it's the earth". other children begin to support that idea: "it's the earth that rotates, it's like a ball. sometimes we kick it and it starts to spin" (júlia; 6.8 years). but others again state that it is the sun: "i think that it's the sun that goes around the earth" (sara; 6.9 years); "me too" – says rui. the students are divided. asked to raise their fingers, ten children think that it is the sun that must rotate around the earth and eight think that it is the earth that rotates around itself. the answers are mostly supportive of moving the flashlight (the sun) around the globe (earth) so as to turn day into night in portugal. this idea is spontaneously constructed by the students as a result of the observation of the changing position of the sun throughout the day in relation to where they are – the apparent motion of the sun. only two children say that it is necessary to rotate the globe around itself – the earth's rotation movement. these differing opinions generate intense discussion and promote participation by other children, who explicitly favor the notion of the earth's rotation international electronic journal of elementary education, 2012, vol.5, issue 1, 5-26 18 movement, as in júlia's case. after the discussion, the class is divided (earth's rotation: 10;55.6% vs. movement of the sun: 8; 44.4%). f. apparent movement of the sun. f1. the illusion of movement of a static body when carried by another body in motion. the students show an understanding of the illusion of backward movement of the "trees", "ground" and "houses" in relation to the automobile they are riding in. these family situations, evoked and recreated in class, enhance the comprehension of what is apparent movement. but will students be able to mobilize that knowledge and apply it to the apparent motion of the sun? the answer appears in the following pedagogical approach. f2. the illusion of movement of the sun as we are carried by the moving earth. passage from the diary: the children are encouraged to think about the following question: "so, is the sun moving?" – i ask. again, no one supports the idea of the sun moving around the earth. however, some evolve into a mixed idea: "it's the sun and the earth" (pedro; 6.2 years), "i think the earth moves, but the sun moves everywhere" (joão; 6.8 years). others are now beginning to support the idea of the earth's rotation: "it's the earth" (several) "it's the earth that is always spinning" (francisca; 6.2 years); "the earth is a ball and it's always spinning and it seems like it's the sun that is moving" (mafalda; 6.4 years). sara intervenes and states: "as the earth moves slowly and we are here in our place, it seems like the sun is moving. but it's not, it's the earth". sara is very excited at this point and continues to explain her idea: "because if the earth did not spin, we would always be the same". the idea that the movement of the sun is only apparent gains momentum. in the process of social interaction, students evolve to meanings that reveal different levels of conceptual development: − some evolve into a construction that combines the spontaneous idea of the sun moving around the earth with the idea of the earth's rotation, conveyed in the social context of the class: "it's the sun and the earth"; "i think the earth moves, but the sun moves everywhere". − others, who were previously partial to the movement of the sun, now show an understanding of the earth's rotation as the cause of the apparent motion of the sun, as in the cases of sara and mafalda: "it's the earth"; "it's the earth that is always spinning"; "the earth is a ball and it's always spinning and it seems like it's the sun that moves". − there are also those who develop a conceptual formulation of a higher level than the previous, with the generalization of the earth's rotation movement as a cause of the day and night alternation: "as the earth moves slowly and we are here in our place, it seems that the sun is moving. but it's not; it's the earth (...) because if the earth did not spin, we would always be the same". g. the earth's rotation movement in the presence of the sun: the day and night alternation g1. the day and night alternation in portugal in the earth– sun model. passage from the diary: "if it's the earth that rotates, what must we do for it to be nighttime in portugal?" – i ask. without hesitation, the children recognize the need to rotate the earth globe: "we have to rotate the earth" – some say; "we turn the earth" – others; "we must rotate the earth" (rui; 6.6 years). i slowly rotate the globe and ask them what had happened. they say that now it is nighttime in portugal. "and on the other side of the reflective experimental construction of meanings about the shape of the earth and the alternation of day and night / varela the earth, what is it now?" – i ask. "it's daytime" – the children answer. when asked about what they must do for it to be daytime in portugal again, the children answer that "they must rotate the earth". when applied to the earth – sun (globe – flashlight) model, the comprehension that the earth revolves around itself promotes the acknowledgement that the alternation between day and night is a consequence of the earth's rotation movement. g2. being daytime in portugal, could there be night in our country if the earth stopped spinning? the question raises the level of reflection in the class, translating into more elaborate answers. the students understand that, for it to be night, the globe, i.e. the earth, must keep spinning until portugal is again on the non-illuminated part: "it cannot be. if the sun were always in portugal and the earth didn't spin, it would always be daytime and there would be no night. if it were nighttime and it wasn't spinning, it would always be nighttime" (sara); "there couldn't, it would always be daytime" (gabriel); "it would always be the same" (mafalda). g3. generalization of the day and night alternation. passage from the diary: "so, why is there day and night?" i ask. gabriel states: "because the earth is always spinning. it never stops". other children intervene: "because the earth is always spinning" (sara); "if it didn't spin, there would only be day" (lionel); "it's the earth that rotates, and then there is day and night" (bruno); "if it were daytime and the earth stopped, there would never be night again. if it were night, when the earth stopped, there would be no more day" (mafalda). rui adds: "only if the earth moved again". the children who did not answer agreed with those answers and apparently understood that the succession of day and night was a result of the earth's rotation. the students' thoughts about the succession of days and nights have focused on the globe, based on the concrete situation of portugal. when they are asked why there is day and night, the answers point towards the generalization of the idea of the alternation of day and night as a consequence of the earth's rotation movement: "because the earth is always spinning. it never stops"; "it's the earth that rotates, and then there is day and night"; "if it were daytime and the earth stopped, there would never be night again. if it were night, when the earth stopped, there would be no more day". h. alternation of day and night: the sun as a cause vs. consequence of the earth's rotation movement h1. confrontation between the intuitive ideas and the newly acquired learning. passage from the diary: at the beginning of the class the idea sprang up that, in the evening, the sun would go "away" or that it "had gone to sleep". i remind them of those ideas and ask them what they have to say now. sara begins by saying: "the sun never sleeps, it never goes out, it never travels. it is always still, in one place." other interventions follow: "it didn’t go away" (filipa; 6.2 years), "it's like a statue" (gabriel; 6.3 years), "it never does anything, it is always still" (lionel; 6.9 years); "it's as if it were glued to a wall" (mafalda; 6.4 years). "if it didn't go away, then why don't we see it during the night?" – i ask. sara answers again: "because the earth is always rotating and the sun stayed in the part where it was. it's on the other part" – says mafalda, in the meantime. júlia adds: "we are not the only ones who need the sun, other people also need it. they also have plants to grow and they need the sun"; "the earth spins and then the other part gets the sun and the part that had the sun gets night" – says joana (6.7 years). international electronic journal of elementary education, 2012, vol.5, issue 1, 5-26 20 this confrontation reveals remarkable progress in the comprehension of the apparent motion of the sun. the conceptual level of development achieved by the students allows them a critical look at those ideas, and therefore the absence of motion of the sun now makes more sense to them. the child who previously claimed that the sun went to sleep at night is now the first to answer, in a critical tone: "the sun never sleeps, it never goes out, it never travels. it is always still, in the same place". other children now present solid arguments for the fact that we do not see the sun during the night which rely on the earth's rotation movement. analysis of the assessment results of the acquired learning at the end of the class, students answered individually to a questionnaire with true or false items about the alternation of day and night. after three weeks, the students answered the same questionnaire. the following table shows the results obtained at the two moments. table 1. results obtained in the two moments of assessment of the student learning items correct answers m1 (%) after class m2 (%) after 3 weeks 1. the earth is round like a ball 16 (88.9%) 18 (100%) 2. the sun stops shining during the night. 9 (50%) 10 (55.6%) 3. it is daytime in the part of the earth that is facing the sun. 14 (77.8%) 16 (88.9%) 4. when it is nighttime in portugal, it is also nighttime in the whole world. 11 (61.1%) 14 (77.8%) 5. the earth never stops spinning. 14 (77.8%) 14 (77.8%) 6. when it is daytime in portugal, it is nighttime in other countries. 14 (77.8%) 16 (88.9%) 7. there is day and night because the earth is always rotating. 12 (66.6%) 13 (72.2%) these results suggest that the learning acquired by the students was meaningful because it is long-lasting, as opposed to memorized learning, which is soon forgotten (coll & martín, 2001). discussion this study did not aim to assess the impact of reflective experimental science teaching (rest) on improving the understanding of the topics under study. however, the combination of the students’ learning described in the class diary with the results obtained from the questionnaire suggests that the process of teaching and learning occurred in the classroom by means of the practice of rest may have had a highly positive effect on the students’ learning. thus, the combination of such data is indicative that the majority of students have developed a good learning about the earth's shape and alternation day and night, as a result of the rotation of the earth. the construction of this learning started from the students' initial ideas, whose identification is an integral part of the teaching and learning process. despite differences in method, it can be verified that ideas about the shape of the earth are convergent with some ideas identified by other authors (nussbaum, 1985; vosniadou & bewer, 1992) in children from other countries, cultures and similar age groups. through the interpretative content analysis of the class diary it is also possible to identify and characterize some of the processes that promote the quality of the students’ thought and learning. the following stand out: • the communication of ideas and ways of thinking to the class allows the students the reflective experimental construction of meanings about the shape of the earth and the alternation of day and night / varela to contrast their own ways of thinking with the thoughts of others. in this process of verbalization, the students become more aware of their own ideas and the ideas of others. this increased awareness promotes, in some children, the need to restructure their ideas when confronted with other more plausible and consensual ones that appear in the social context of the class. take, as an example, the communication to the class of the meanings implied in the drawings of the earth; • the discursive activity generated around the ideas that spring up in the classroom through the conjoint influence of their peers and the teacher's action improves the quality of those ideas, allows for the participation of other students and favors the development of more elaborate meanings; • the students' more evolved meanings and the teacher's action direct and support the conjoint cognitive activity, allowing the slower students to elaborate new reconstructions and approximations to those meanings, which, after being verbalized in the social context of the class, are then shared by a growing number of students; • the students' sociocognitive activity generated around experimental evidence introduces a considerable increase in the development of scientific meanings – the shape of the earth; what is day and what is night; the alternation of day and night as a consequence of the earth's rotation movement; • the teacher's action, through continuous and recurrent reflective questioning (scaffolding), not only helps students to become aware and regulate their cognitive activity, but it also promotes their ability to escalate to progressively higher levels of cognition and learning; • the introduction of significant and familiar analogies related to their day-to-day contexts facilitates the comprehension of particularly difficult situations, as was the case of the apparent motion of the sun; • the contrast between the learning acquired and their less evolved initial ideas triggers in the students a heightened awareness of their own learning – metacognitive knowledge; the promotion of an experimental science teaching practice in primary school has proved a difficult and complex task, as it requires that teachers assimilate and develop not only scientific knowledge, but also specific didactic knowledge about how to teach the subjects of specific curricular areas. the processes of teacher training, in our perspective, should be shaped by the practical and theoretical knowledge emerging from the holistic understanding of the teaching and learning processes, promoted and experienced in the classroom context. thus, the present paper may prove a valuable resource for the initial and continuous teacher training process in order to endow these professionals with a specific knowledge on how to elicit and promote, within the classroom context, identical processes in approaching the curricular topic on the shape of the earth and the alternation of day and night. finally, it can be argued that children are able to overcome complex cognitive challenges when they are approached in a collaborative context of stimulation and freedom of expression of their thoughts. thus, the interaction with other more developed children, or with the teacher, and the domain of language promote higher levels of learning, which is an important factor for the development of thought (vygotsky, 1978). international electronic journal of elementary education, 2012, vol.5, issue 1, 5-26 22 • • • paulo varela received his ph.d. in child studies, in the field of science education, from the university of minho in 2010. he is an assistant professor at the institute of education of the department of integrated studies of literacy, didactics and supervision at the same university since 1998. his research interests focus on the study of in-class experimental science teaching and learning processes with children. references aleixandre, j. m. p. (2003). comunicación y lenguaje. in j. aleixandre (ed.), enseñar ciencias (pp. 55-71). barcelona: editorial graó. appleton, k. (2003). how do beginning primary school teachers cope with science? toward an understanding of science teaching practice. research in science education, 33, 1-25. blown, e. j. & bryce, t. g. k. (2007). knowledge restructuring in the development of children’s cosmologies. international journal of science education, 28(12), 1411-1462. brand, b. r. & moore, s. j. (2011). enhancing teachers’ application of inquiry_based strategies using a constructivist sociocultural professional development model. international journal of science education, 33(7), 889-913 candela, a. (1999). ciencia en el aula. los alumnos entre la argumentación y el consenso. méxico: editorial paidós. carrasco, j. b. & hernández, j. f. c. (2000). aprendo a investigar en educación. madrid: ediciones rialp. català, m. & vilà, n. (2002). las funciones lingüísticas en el proceso de adquisición de los conocimientos científicos. in m. catalá, et. al, (eds.). las ciencias en la escuela. teorías y prácticas (pp. 89-103). barcelona: editorial gráo. charpack, g. (2005). as ciências na escola primária: uma proposta de acção. mem martins: editorial inquérito. chin, c. (2006). classroom interaction in science: teacher questioning and feedback to students’ responses. international journal of science education, 28(11), 1315-1346. cleary, t. j. & zimmerman, b. j. (2004). self-regulation empowerment program: a schoolbased program to enhance self-regulated and self-motivated cycles of student learning. psychology in the schools, 41(5), 537-550. coll, c. & martín, e., (2001). a avaliação da aprendizagem no currículo escolar: uma perspectiva construtivista. in c. coll, (eds.), o construtivismo na sala de aula. novas perspectivas para a acção pedagógica (pp. 196-221). porto: edições asa. costa, f. (2006). as actividades práticas na educação em ciência: uma oportunidade perdida? in g. hamido; h. luís; m. c. roldão & r. marques (eds.), transversalidade em educação e em saúde (pp. 31-38). porto: porto editora. denzin, n. k. (1989). interpretive interactionism. newbury park, ca: sage. domínguez, m. a. & stipcich, m. s. (2009). buscando indicadores de la negociación de significados en clases de ciencias naturales. revista electrónica de enseñanza de las ciencias, 8(2), 539-551. retrieved from http://www.saum.uvigo.es/reec/ erickson, f. (1986). qualitative methods in research on teaching. in m. c. wittrock (ed.), handbook of research on teaching (pp. 119-161). new york, ny: macmillan. eurydice (2011). science education in europe: national policies, practices and research. brussels. education, audiovisual and culture executive agency p9 eurydice fasick, f. a. (2001). some uses of untranscribed tape recordings in survey research. public opinion quarterly, 41, 549-552. the reflective experimental construction of meanings about the shape of the earth and the alternation of day and night / varela fleer, m. (1997). a cross-cultural study of rural australian aboriginal children's understandings of night and day. research in science education, 27 (1), 101-116. georghiades, p. (2006). the role of metacognitive activities in the contextual use of primary pupils’ conceptions of science. research in science education, 36, 29-49. gonzález, s. & escudero, c. (2007). en busca de la autonomía a través de las actividades de cognición y de metacognición en ciencias. revista electrónica de enseñanza de las ciencias, 6(2), 310-330. retrieved from http://www.saum.uvigo.es/reec/ guba, e. g. & lincoln, y. s. (1989). fourth generation evaluation. thousand oaks, ca: sage publications. guba, e. g. & lincoln, y. s. (2000). competing paradigms in qualitative research. in n. denzin, & y. lincoln (eds.), handbook of qualitative research (pp. 105-117). thousand oaks, ca: sage. harlen, w. & qualter, a. (2005). the teaching of science in primary schools. london: david fulton publishers. harlen, w. (1999). effective teaching of science. a review of research. edinburgh: the scottish council for research in education (scre). harlen, w. (2007). enseñanza y aprendizaje de las ciencias. (2ª ed. atualizada). madrid: ediciones morata. harlen, w. (2010). implementing inquiry-based learning in science education (ibse). fibonacci project presentation on september 22 nd 2010. retrieved from http://fibonacci.unibayreuth.de/index.php?id=203 henao, b. l. & stipcich, m. s. (2008). educación en ciencias y argumentación: la perspectiva de toulmin como posible respuesta a las demandas y desafíos contemporáneos para la enseñanza de las ciencias experimentales. revista electrónica de enseñanza de las ciencias, 7(1). retrieved from http://www.saum.uvigo.es/reec/ huber, r. a. & moore, c. j. (2001).a model for extending hands-on science to be inquirybased. school science and mathematics, 101(1), 32-42. ibáñez, v. e. & alemany, i. g. (2005). la interacción y la regulación de los procesos de enseñanza-aprendizaje en la clase de ciencias: análisis de una experiencia. enseñanza de las ciencias, 23(1), 97-110. jorgenson, o. (2005). what k-8 principals should know about hands-on science. principal effective interventionspecial section, 85(2), 49-52. kuhn, d., e. amsel & m. o'loughlin (1988). the development of scientific thinking skills. orlando, fl: academic press. larkin, s. (2006). collaborative group work and individual development of metacognition in the early years. research in science education, 36, 7-27. lee, o.; hart, j. e.; cuevas, p.; enders, c. (2004). professional development in inquiry-based science for elementary teachers of diverse student groups. journal of research in science teaching, 41(10), 1021-1043. levinson, r. (2002). o professor, o aluno e o livro didático oficial de ciências: será que deveríamos renunciar ao livro escolar? brasília: m. a. marfan. retrieved from http://portal.mec.gov.br/seb/arquivos/pdf/vol2c.pdf maclean, l. m., meyer, m., & estable, a. (2004). improving accuracy of transcripts in qualitative research. qualitative health research, 14(1), 113-123. maloney, j. & simon, s. (2006). mapping children’s discussions of evidence in science to assess collaboration and argumentation. international journal of science education, 28(15), 1817-1841. international electronic journal of elementary education, 2012, vol.5, issue 1, 5-26 24 martínez, j. m. o. & díaz, j. a. a. (2005) la enseñaza de las ciencias en primaria y secundaria hoy. algunas propuestas de futuro. revista eureka sobre enseñanza y divulgación de las ciencias, 2(2), 241-250. matthews, m. r. (2000). construtivismo e o ensino de ciências: uma avaliação. cad. cat. ens. fís., 17(3), 270-294. mercer, n. & littleton, k. (2007). dialogue and the development of children's thinking: a sociocultural approach. london: taylor and francis. ministry of education (2004). organização curricular e programas ensino básico – 1º ciclo. mem martins. departamento de educação básica. miras, m. (2001). um ponto de partida para a aprendizagem de novos conteúdos: os conhecimentos prévios. in c. coll; e. martin; t. mauri; m. miras; j. onrubia; i. solé & a. zabala (eds.). o construtivismo na sala de aula. novas perspectivas para a acção pedagógica (pp. 54-73). porto: edições asa. molenaar, i.; van boxtel, c. a. & sleegers, p. j. c. (2011). metacognitive scaffolding in an innovative learning arrangement. instructional science, 39, 785-803. naylor, s.; keogh, b.; downing, b. (2007). argumentation and primary science. research in science education, 37, 17-39. newman, j. m. (2000). action research: a brief overview. forum qualitative sozialforschung / forum: qualitative social research, 1(1). retrieved from http://qualitative-research.net/fqs. nussbaum, j. (1985). the earth as a cosmic body. in r. driver, e. guesne & a. tiberghien (eds.), children´s ideas in science (pp. 170-192). philadelphia: open university press. özsoy, s. (2012). is the earth flat or round? primary school children’s understandings of the planet earth: the case of turkish children. international electronic journal of elementary education, 4(2), 407-415. palincsar, a. s. & herrenkohl, l. r. (2002). designing collaborative learning contexts. theory into practice, 41, 26-32. pérez gómez, a. i. (2005). comprender y transformar la enseñanza (pp.17-137). madrid: ediciones morata,s. l.. porlán, r. (1998). constructivismo y escuela: hacia un modelo de enseñanza-aprendizaje basado en la investigación. sevilha: díada editora s. l.. ratner, c. (2002). subjectivity and objectivity in qualitative methodology. forum qualitative sozialforschung / forum: qualitative social research, 3 (3). retrieved from http://www.qualitative-research.net/fqs/fqs-eng.htm rivard, l. p. (2004). are language-based activities in science effective for all students, including low achievers? science education, 88, 420-442. rojas-drummond, s. & mercer, n. (2003). scaffolding the development of effective collaboration and learning. international journal of educacional research, 39, 99-111. roldão, m. c. (2003). gestão do currículo e avaliação de competências. as questões dos professores. barcarena: editorial presença. sá, j. & varela, p. (2004). crianças aprendem a pensar ciências: uma abordagem interdisciplinar. porto: porto editora. sá, j. & varela, p. (2007). das ciências experimentais à literacia: uma proposta didática para o 1º ciclo. porto: porto editora. sá, j. (2002a). renovar as práticos no 1º ciclo pela via das ciências da natureza. porto: porto editora. sá, j. (2002b). diary writing: an interpretative research method of teaching and learning. educational research and evaluation, 8(2), 149-168. the reflective experimental construction of meanings about the shape of the earth and the alternation of day and night / varela siegal, m., butterworth, g., & newcombe, p. (2004). culture and children’s cosmology. developmental science, 7(3) 308-324 varela, p. (2012). experimental science teaching in primary school: reflective construction of meanings and promotion of transversal skills. saarbrücken, germany: lap lambert academic publishing. vosniadou, s. & brewer, w. f. (1992). mental models of the earth: a study of conceptual change in childhood. cognitive psychology, 24, 535-585. vosniadou, s. (1991). designing curricula for conceptual restructuring: lessons from the study of knowledge acquisition in astronomy. journal curriculum studies, 3(23), 219-237. vosniadou, s., & brewer, w. f. (1994). mental mod s of the day/night cycle. cognitive science, 18, 123-183. vosniadou, s.; skopeliti, i.; ikospentaki, k. (2004). modes of knowing and ways of reasoning in elementary astronomy. cognitive development, 19, 203-222. vygotsky, l. s. (1978). mind in society: the development of higher psychological processes. london: havard university press. vygotsky, l. s. (1987). pensamento e linguagem. são paulo: martins fontes editora. white, r. t. (1994). dimensions of content. in p. fensham, r. gunstone & r. white (eds.), the content of science. a construtctivist approach to its teaching and learning (pp. 255262). london: falmer press. zabalza, m. a. (2004). diarios de clase: un instrumento de investigación. madrid: narcea. zohar, a. (2006).el pensamiento de orden superior en las clases de ciencias: objetives, medios e resultados de investigación. enseñanza de las ciencias, 24(2), 157-172. international electronic journal of elementary education, 2012, vol.5, issue 1, 5-26 26 www.iejee.com this page is intentionally left blank students’ performance calibration in a basketball dibbling task in elementary physical education international electronic journal of elementary education, 2015, 8(1) issn:1307-9298 copyright © iejee www.iejee.com dear iejee reader, we are excited by presenting you a new issue of international electronic journal of elementary education (iejee). on behalf of our editorial board, as an editor-in-chief, i would like to share a message i received from one of our researchers who published a paper in iejee. his paper was downloaded by 64 persons within a week. we also noticed that iejee is reaching more and more researchers, teachers, students and policy makers, particularly after being indexed in eric education resources information center, usa. as its previous issues, this issue of iejee also includes papers addressing several important topics. ersoy & bozkurt addresses the changes, challenges and opportunities that the new technologies represent for elementary school teachers in their work. the authors use the metaphor of journey from sand table to interactive whiteboard for the changes that elementary teachers are going through in our time. nurlu addresses the importance and the consequences of the teachers’ self-efficacy in mathematics teaching. her findings suggest that teachers with high self-efficacy demonstrate higher level of effort and persistence with students, are more open to new ideas and new methods. they believe in students’ achievements and take responsibility for students’ success. these teachers use to place more importance on building a warm relationship with their students rather than with the parents. bedi̇r investigated perception of teaching efficacy by primary and secondary school teachers. she found significant differences among teachers’ perception concerning teaching efficacy across different variables such as the faculty they graduated, gender, course match, in-service training, branch, and seniority. an interesting finding seems to be the high level of self-efficacy among teachers with regard to class management and low level of self-efficacy with regard to choosing instructional methods/strategies. özyürek & aydin investigated the impact of computer-animated concept cartoons and outdoor science activities on creating awareness among seventh graders about light pollution. the findings show that both strategies can be effective in creating awareness about light pollution. çakir investigated the instructional materials commonly employed by foreign language teachers at elementary schools. on the basis of his study çakir argue for utilization of variety of instructional materials to motivate learners. at the same time he wish to see instructional materials that can help teachers to create an interactive foreign language teaching atmosphere. özerk & özerk has conducted a single-case study with a bilingual student with autism spectrum disorders (asd). the aim was to teach the student social-communications skills through video modeling. their study reveals that children with asd can learn desirable behavioral skills as by-products. video modeling can also contribute positively to the social inclusion of bilingual children with asd in school settings. dağdelen & köstereli̇oğlu investigated the effect of conceptual change texts in overcoming misconceptions in “people and management” unit of a social studies course. their quasi-experiment study reveals that using conceptual change texts can be a helpful http://www.iejee.com/ international electronic journal of elementary education vol.5, issue 2, 2013 vi strategy for prevent and overcoming misconceptions related to several topics in social studies. özsoy, kuruyer & çakiroğlu looked at the relationship between a small group of students’ mathematical problem solving skills in relation to their reading levels. the researchers utilized several methodological approaches. their findings show that there is a close relationship between the students’ problem solving skills and their reading skills. luo presents a case study examining 3rd, 4th and 5th graders’ design fixation and cooperative learning in an engineering design project. a mixed methods instrument, the cooperative learning observation protocol (clop), was adapted to record frequency and class observation on cooperative learning engagement through detailed field notes. she identifies three major themes for design fixation and the challenges the students encountered in cooperative learning process. graham, graham & west take up an important issue: vocabulary development. they designed a study to demonstrate the effect of implementing multi-component vocabulary strategy instruction in fourth grade social studies. their study will without any doubt be an important contribution to our knowledge base with regard to teaching strategies for vocabulary development. kašparová addresses the legal and principal aspects of home education. on the basis of a longitudinal ethnographic study, she discusses how the homeschoolers combat the structural discrimination embodied in their lawful protection in the czech republic. her main argument is the importance of the right of homeschoolers to be included. in her analytical discussions, she includes pierre bourdieu’s theoretical perspectives regarding various forms of capital. i would like to thank all of the authors, peer reviewers and iejee’s staff for their contributions. dr. turan temur editor-in-chief microsoft word iejee_5_3_libarkin_schneps international electronic journal of elementary education, 2012, 5(1), 47-62. issn:1307-9298 copyright © iejee www.iejee.com elementary children’s retrodictive reasoning about earth science julie c. libarkin ∗∗∗∗ michigan state university, east lansing mi, usa matthew h. schneps harvard-smithsonian center for astrophysics, cambridge ma, usa received: 2 september 2012 / revised: 10 november 2012 / accepted: 10 november 2012 abstract we report on interviews conducted with twenty-one elementary school children (grades 1-5) about a number of earth science concepts. these interviews were undertaken as part of a teacher training video series designed specifically to assist elementary teachers in learning essential ideas in earth science. as such, children were interviewed about a wide array of earth science concepts, from rock formation to the earth’s interior. we analyzed interview data primarily to determine whether or not young children are capable of inferring understanding of the past based on present-day observation (retrodictive reasoning) in the context of earth science. this work provides a basis from which curricula for teaching earth and environmental sciences can emerge, and suggests that new studies into the retrodictive reasoning abilities of young children are needed, including curricula that encourage inference of the past from modern observations. keywords: earth science, reasoning, retrodiction. introduction this paper discusses the nature of children’s reasoning about earth phenomena and processes, and specifically the extent to which retrodictive reasoning is evident in their discourse. we utilize a set of twenty-one interviews with elementary-aged children as the data set from which evidence of retrodictive reasoning emerged. we also documented the presence of alternative conceptions about the earth and considered the extent to which these alternative conceptions interfered with reasoning. retrodictive reasoning in earth science retrodiction, the interpretation of present-day evidence to infer ancient processes, lies at the heart of much of earth science (e.g., ault, 1998). while prediction has a role in earth science (e.g., for forecasting natural hazards or extrapolating the impact of human actions on natural ∗ center for integrative studies in general sciences & dept. of geological sciences, michigan state university, 288 farm lane, 206 natural science, east lansing, mi usa 48824, phone: 011+1+517-3558369, e-mail: libarkin@msu.edu international electronic journal of elementary education vol.5, issue 1, 47-61, 2012 48 systems), retrodiction lies at the heart of all fields associated with paleo-processes, including geology, evolutionary biology, and cosmology. interestingly, the concept of retrodiction is not commonly found in discourse about scientific reasoning that emerges from the science education community (sibley, 2009), perhaps because predictive domains of science dominate the field. at the same time, retrodictive reasoning is of vital importance because of the role it plays in public debate about topics such as evolution, the creation of the universe, and the age of the earth. what are the unique characteristics of retrodictive reasoning? retrodictive reasoning requires the understanding that patterns present in the modern world are the imprints of processes that have already occurred. as a consequence of this recognition, retrodictive reasoners must be able to extrapolate possible causes for these patterns, balance the probability of one specific cause against the likelihood of another, and rationalize a preference for one particular event resulting in an observed pattern. in this way, retrodictive reasoners recognize the role of causation in the production of patterns; interestingly, not all people are able to link processes and patterns together (libarkin & kurdziel, 2006). within this ontology also lies the need for using narrative to explain phenomena (norris et al., 2005) as well as reasoning about time. retrodictive reasoning about earth systems is inherently connected to systems thinking (kali et al., 2003; lawton, 2001). systems thinkers must recognize that processes, the events that result in observable patterns, often interact to produce surprising results. recognizing not only singular events but also the confluence of events is the hallmark of an effective systems thinker. systems thinking also requires an understanding that processes, particularly within complex system like the earth, do not always interact in linear ways. non-linear processes, including negative and positive feedback loops, are important components of earth systems thinking for both modern and ancient earth. retrodictive reasoning is inherently different from predictive reasoning. both a process and its result are observable when they are occurring in real time, thus allowing a prediction to be tested. a process that has already occurred is not observable; one can only engage in experiments, in the lab or through observable natural systems, which replicate the process and look for results consistent with the original observation. in retrodiction, one can never actually observe the original process in action. this results in interesting problems from a scientific perspective; one can never completely disprove a hypothesis about a process that has long since occurred. one can only engage in a “more likely than not”, or vice versa, standard. curricula that explicitly address retrodiction, even in its simplest forms, are surprisingly uncommon in the earth sciences especially for young children, although inquiry in earth science education requires attention to retrodiction (pyle, 2008). existing studies demonstrate that middle and high school students can engage in retrodictive reasoning about possible evolutionary pathways through inquiry with hominid skulls and radiometric data (thomson & chapman beall, 2008). similarly, geology majors in a capstone course specifically oriented towards retrodiction in global systems reported better understanding of earth’s spheres after engaging in the course as well as more confidence in their ability to retrodict patterns based on observable modern processes (sunderlin, 2009). alternative conceptions in earth science although alternative conceptions are not the focus of this study, the students in our interviews present a number of non-scientific ideas that warrant discussion of alternative conceptions here. a growing body of literature has documented the alternative conceptions about earth’s systems held by elementary, secondary, and advanced students (see reviews of cheek, 2010; dove, 1998; king, 2008). these conceptions provide a window into the elementary children’s retrodictive reasoning about earth science / libarkin & schneps 49 reasoning that might be occurring as students interact with earth science concepts and phenomena, as well as insight into the potential difficulties students may face in the classroom. an understanding of alternative conceptions is vital for teachers interested in aligning curriculum with student needs, and exposure to the ideas of others can provide students themselves with a gateway into learning complex material. alternative conceptions about earth science have been documented across the earth system and across age groups. for the purposes of this paper, we focus on alternative conceptions related to the topics covered in our study, including geologic time, particularly as it relates to the timing and rate of earth processes, rock and soil formation, and deep earth processes related to, for example, plate tectonics and magma formation. studies of student ideas about earth science are much rarer than in other disciplines. where possible, we report on studies of young children, and include studies of older students (high school, college) where relevant studies of young children are not available. student conceptions about earth’s surface processes are often related to their personal observations of the natural world. for example, alternative conceptions about rock formation mechanisms can be understood in the context of observable surficial processes. college students in two studies (kortz & murray, 2009; kusnick, 2002) articulated the idea that rocks form when water dries up or when water deposits material into piles; the simple acts of drying and depositing generate aggregate rocks. younger students also describe rocks as growing from smaller objects or pebbles (ault, 1984; blake, 2005; dal, 2007), an idea that may also be present in older students (kusnick, 2002). the relationship between rocks and soils is also sometimes misunderstood, with some teachers believing that soils are deposited as rock layers (gosselin & macklem-hurst, 2002). happs (1984) noted particularly the importance of geologic time in understanding soil formation, and many of the aforementioned studies note difficulty students have in conceptualizing deep time. conceptions about deep earth processes may more often be driven by instruction, rather than personal experience. phenomena that are not directly tangible but are rather recognized by their effects, such as plate tectonics, geomagnetism, and gravity, can be particularly difficult for students in earth science courses to understand (libarkin & kurdziel, 2006). this can result in confusion at basic levels, such as with simple terminology used to explain deep earth processes (e.g., libarkin et al., 2005), and at more conceptual levels. for example, students may believe that earthquakes push tectonic plates (barrow & haskins, 1996; ross & shuell, 1993), that mountains simply grow (e.g., muthukrishna et al., 1993; trend et al., 2000), or that volcanic magma originates at the earth’s core (nelson et al., 1992). students also draw a surprising array of models of the earth’s interior when asked to imagine cutting the earth in half, including an earth containing flat or no layers (e.g., blake, 2005; delaughte et al., 1998; libarkin et al., 2005; lillo, 1994). the physical location of tectonic plates and the physical state of earth’s interior are also areas of significant confusion. student ideas about geologic time have focused on understanding of the order of events as well as the long timescales (‘deep time’) inherent to many earth processes. ault (1982) recognized that young children are able to reason about relative time, an ability that is found across age ranges (e.g., dahl, anderson, & libarkin, 2005; libarkin et al., 2005; trend, 1998, 2000, 2001). a number of alternative conceptions about the relative ordering of events have been identified, including that man and classical dinosaurs co-existed (schoon, 1995), life and supercontinents existed when earth first formed (libarkin et al., 2005), and similar ideas related to the misunderstanding of the order and scale of geologic events. the idea that temporal reasoning is unique from other abilities (montangero, 1996) has also been applied to studies of high school students, with the conclusion that difficulties in reasoning about international electronic journal of elementary education vol.5, issue 1, 47-61, 2012 50 deep time may be related to deficiencies in diachronic thinking (dodick & orion, 2003). similarly, trend (1998) suggests that mathematical difficulties associated with understanding large numbers may inhibit understanding of deep time. aim of the study the earth sciences have generally not received as much attention within science education as biology, chemistry, and physics have, although a growing recognition of the importance of earth science in schooling and global discourse is increasing the attention paid to it and related disciplines by science educators (lewis & baker, 2009). our goal in undertaking this study is to further the understanding that teachers and researchers have about the ways in which children reason about earth processes. to further this goal, we analyzed student ability to engage in simple retrodictive reasoning, as reflected in their discourse about earth processes, and utilized this opportunity to also document the presence of alternative conceptions. based on existing work that clearly shows predictive reasoning ability among children (i.e., zimmerman, 2000), we hypothesized that elementary students would be capable of engaging in the aspects of simple retrodictive reasoning needed to reason in earth science. we present evidence of retrodictive reasoning as suggested by student discourse, and considered the role that age might play in student reasoning ability. secondarily, we hypothesized that, given the limited research into children’s alternative conceptions about earth science, confirmation of existing studies and new alternative conceptions about earth processes and phenomena would be evident in student responses. methods context interviews with twenty-one elementary students were analyzed for this study. these data originated from a series of interviews conducted during the creation of a video series created to support teacher-education, published in 2004 (argow, reilly, & schneps, 2004). this video series, containing edited components of the interviews analyzed here, is accessible online (http://www.learner.org/resources/series195.html) and was designed to help elementarygrade teachers develop deep understanding of the science concepts needed to effectively address standards in earth and space science. interviews with children were used to engage teachers about the prevalence of student misconceptions, and were coupled with in situ interviews with real geologists, explanatory simulations, and online activities. students were interviewed singly and in pairs, and some students were interviewed more than once. participants (and guardians) provided written informed consent for this study. nevertheless, we delayed publication of these data expressly because some of the children discussed here are shown in the video series – we wanted enough time to pass between publication of the series and analysis of these data to ensure that neither we nor readers would be able to identify specific participants. that is, given the nine-year delay between these interviews and this analysis, we are now unable to link any de-identified interview transcripts with specific children. finally, while we present general demographic information below, we have explicitly limited the information provided to ensure interviewee anonymity. procedures recruitment. participants were twenty-one elementary-aged children recruited from classrooms in a large, north american city. given that the interviews were conducted to provide source material for teacher education, participants were selected on the basis of their interests in science and their willingness to appear on camera, as gauged by their elementary children’s retrodictive reasoning about earth science / libarkin & schneps 51 teachers. ethnicities represented a broad diversity as expected for a large city; we do not provide specific ethnicity information. children were enrolled in first through fifth grade and were 62% (n=13) female (table 1). table 1. grade and gender distribution of interviewees grade gender* general topics covered 1st 1 f, 1 m soil, earth’s interior, air 2nd 2 f, 1 m rock formation, soil, volcanoes, earth’s interior, plate tectonics 3rd 5 f, 2 m rock formation, soil, volcanoes, earth’s interior, plate tectonics 4th 3 f, 3 m rock formation, volcanoes, earth’s interior, erosion 5th 2 f, 1 m rock formation, soil, volcanoes, earth’s interior * f=female, m=male. semi-structured interviews. all interviews were conducted in the same room. interviews were audioand video-taped, with up to three video technicians present to manage multiple cameras and the audio recording. the first author conducted all but three of the interviews, and the second author observed all of the interviews. during most of the interviews, the second author was able to speak to the interviewer through an earpiece; this allowed the second author to make suggestions for the interviews without interfering with the interview process directly. care was taken to ensure the interviewer did not to introduce verbal or nonverbal cues that would direct or lead the participant’s response. in particular, a set of predetermined interview prompts was used with all subjects and any interview probes were derived from the interviewees’ own language during the interviews. each interviewee was given a unique identifying code prior to analysis; the second letter and the number indicate gender and grade level, respectively, while the first letter makes each code unique. most of the children were interviewed alone; eight children were interviewed in pairs and two participants were interviewed more than once. interviews started with general topics related to rocks, soil, mountains, and water, although interviewees ultimately discussed a variety of other topics with interviewees (table 1). each interview started with a visual or drawing prompt. at the start of each interview, children were asked to draw pictures and/or were shown rocks (e.g., fig. 1), a bucket of soil, a rain stick, or photos to prompt their thinking. these probes were used as needed throughout the interview, and interviewees were encouraged to draw out their ideas. the interview protocol was semi-structured. the interviewer began each interview with a few scripted questions, and these questions were used throughout the interview to redirect the discussion, as well. as the interview progressed, probes were generated in response to interviewee discourse and drawings. this resulted in a wide variety of topics being covered across the entire sample. coding. transcript analysis focused on retrodictive reasoning patterns, with a secondary purpose of documenting alternative conceptions. first, we analyzed transcripts for the presence or absence of basic retrodictive reasoning. this coding scheme does not represent retrodictive reasoning in its entirety, but rather models the most simple aspects of retrodictive reasoning that are necessary for effective reasoning about the earth: 1) links are made between observations and processes as evidenced by an understanding that patterns present in the modern world are the imprints of processes that have already occurred. 2) multiple working hypotheses are raised (multiple possible causes for these patterns are held simultaneously). 3) preference for one hypothesis over others are rationally explained. 4) reasoning references time beyond human timescales. international electronic journal of elementary education vol.5, issue 1, 47-61, 2012 52 reliability. inter-rater reliability for codes was established through concurrent coding of ~10% of the responses by the first author and a colleague (a geologist and former high school teacher). the raters calculated the single measures intraclass correlation across all retrodictive codes evaluated by both authors. the average measures intraclass correlation was 0.92 (min.=0.82 and max.=0.96). an intraclass correlation of 1.0 implies perfect reliability; a correlation of 0.92 indicates that both raters were in strong agreement. disagreement related to the similarity in two codes, one that related to interacting events and the other associated with feedback; this disagreement was clarified through recognition that neither was fundamental to retrodictive reasoning and both were removed. this high reliability for the coded subsample allowed one author to complete the coding with a reasonable assumption of reliability for all analyses. figure 1. examples of rocks used to engage students during interviews. a) fossil-rich limestone. b) fine-grained sandstone with ripple marks. results and discussion the structure of students’ retrodictive reasoning patterns is intertwined with the presence of alternative conceptions about earth phenomena. as such, we discuss retrodictive reasoning patterns in significant detail, and note where alternative conceptions are also evident in the student discourse. we particularly focus on evidence that highlights the presence of retrodictive reasoning in students. the elementary students interviewed in this study exhibited varying degrees of retrodictive reasoning. nearly every interviewee recognized that earth materials and patterns present today are the result of prior events. this reasoning was evident most commonly when students were presented with tangible materials, such as dirt or rocks, or when students were drawing pictures of structures they had seen or learned about, such as volcanoes or the earth’s interior. most students were also able to articulate a possible process that could have created a specific observed pattern, although not all students were able to articulate nonanthropogenic processes. that is, only a subset of students recognized that processes occur on earth that are independent of human action. in addition, a few students articulated more than one possible process, demonstrating that young children are able to generate multiple working hypotheses about earth processes. links between observations and processes all interviewees (n=21) recognized that processes leave traces, and that these traces can be used to reason about processes themselves. this is the most fundamental component of retrodictive reasoning and was evident in student discourse about both simple and complex phenomena. in addition, students were generally able to articulate reasoning for why a a b elementary children’s retrodictive reasoning about earth science / libarkin & schneps 53 process would create a specific outcome. to demonstrate this, we provide examples related to students’ ideas about rock formation as well as more complex ideas about why some rocks are found in seemingly unlikely places. students most often called upon processes that were familiar from everyday life, and less commonly called upon processes that had clearly not been experienced directly. many of the children interviewed about earth’s interior held the previously documented idea that the earth contains a layer of magma, often with that layer at the earth’s center, and that this layer is the source of lava that is extruded by volcanoes. interviewees who discussed the earth’s interior all expressed an idea that the center of the earth is very hot. one fourth grader (jf4) explained that, in addition to other reasons, the inside of the earth is hot because neither water nor air could get inside the earth to cool it off. she compared a hot, stuffy room to the earth’s interior, explaining that: jf4: if no air can be in [a] room, it gets horribly hot. q: why does it get really, really hot in that room? jf4: because no fresh air from the outside can get in here and make us feel nice and cool. so that’s why. this student is clearly calling upon familiar experiences with fire and stuffy rooms in building her explanations about earth phenomena, and may related to the alternative conception that insulation by itself is a source of heat (e.g., wiser & amin, 2001). another example of everyday explanations is found in a student’s model for the formation of dirt and rocks included in a model in which the rotation of the earth resulted in the mixing of materials that form dirt. for example, a 1st grade female indicated that dirt forms because “the earth is spinning so fast, that kind of mixes everything together” (ef1). this student used the mixing that occurs when she makes cookies as an explanation for why this mixing would result in a solid rock. although not based on everyday experience, a fourth grade female presented a disconnected set of ideas for how shells might form a rock (fig. 1a) that included earth’s rotation: “the weather or the sun or you know how the earth sort of rotates around, maybe they got hardened and that’s how this thing was made” (zf4). experience with heat and the sun’s impact on materials at earth’s surface likely also influenced the processes students called upon to explain rock formation. one third-grade student believed that rocks would harden because “the sun probably was looking at it for a long time, like a lot of heat was on it for a lot of years and then it finally just hardened” (ef3). a fourth grader (mf4) expressed the partially correct idea about sedimentary rock formation, stating that, “sand rocks, they're made from sand crushed under the ground for many years”; it is unclear in this case whether ‘crushing’ refers simply to pressure pushing sand together (a scientific idea) or whether it indicates the formation of sand from rocks underground (an alternative conception). beyond simple explanations of rock formation, students were asked to reason about seemingly contradictory information. a rock containing shells (fig. 1a) was presented to some students as having been discovered beneath the ground within the interior of the usa. the students were generally adamant that the rock formed near a beach, and also recognized that the interior usa is far from beaches. this cognitive dissonance provided an opportunity for students to reason retrodictively. for example, a third-grade student (of3) provided an explanation for the contradiction of a shell-containing rock far from a beach that relied upon her prior knowledge: q: …how could this [rock in fig. 1a], that you said came from a beach, end up in the middle of kansas? international electronic journal of elementary education vol.5, issue 1, 47-61, 2012 54 of3: maybe a long time ago…maybe that was formed when the dinosaurs, where the united states wasn’t all together…it was probably next to the ocean once, but then it got in while the states formed together to make united states of america. probably just stayed there until you guys dug it up. with prompting, of3 provided a drawing of the process that would result in the “states form[ing] together to make united states of america”, with each state representing a unique tectonic plate (fig. 2). she clearly articulated that the rock needed to form near the ocean, and recognized that while an ocean did not currently exist in the region an ocean may have existed in that location in the past. several researchers have noted that students often mistake continental boundaries for plate tectonic boundaries (e.g., (marques & thompson, 1997). the above-mentioned student as well as others in this study similarly suggested that tectonic plates are delineated by non-related external boundaries. in this case, of3 confounded geopolitical boundaries associated with the borders of states within the usa with physical boundaries separating tectonic plates. despite this confounding, of3’s hypothesis of tectonic plate movement is a good explanation for the presence of shells at great heights such as in the alps, although a rock in the usa’s interior likely resulted from a different mechanism (an intracratonic ocean). figure 2. drawing of the “state tectonics” process made by a third-grade student (of3). each landmass, outlined in brown, was drawn as a different state (e.g., kansas, minnesota). the brown dot represents the location in the interior of the usa where the interviewer indicated the rock in fig. 1a was discovered. blue represents ocean locations during the time of the rock’s formation. a second third-grader (tf3) provided an alternative, human-based cause for the rock’s location in the continent’s interior. she also recognized the necessity that the ocean must have existed in the rock’s place of origin in the past. her model appears to be based on prior exposure to beach replenishment strategies undertaken by municipalities to combat beach erosion. q: …how could this [rock in fig. 1a], that you said came from a beach, end up in the middle of kansas? elementary children’s retrodictive reasoning about earth science / libarkin & schneps 55 tf3: …i've heard that they took dirt or sand or stuff and dumped it into the ocean, so the ocean got smaller and smaller and smaller, like somewhere over there, and then it became land. so maybe…they put dirt and sand in the water to make land, like extra land. q: so who did that? tf3: people that were there…not necessarily people today, because they would be long gone by now, but like people that existed a long time ago. in this exchange, the student still recognizes that the process must have been an ancient one. this requires her to call upon peoples who are “long gone by now”, rather than requiring the process to be modern. both of these exchanges, that of of3 and tf3, suggest that these students are using retrodictive reasoning to combine disparate pieces of information (i.e., state boundaries, ancient peoples, beach replenishment) into a single explanatory model that still allows for earth processes to have occurred in the past. presence of multiple working hypotheses in most cases, interviewees provided only one explanation for their observations or ideas; that is, most of the interviewed students did not offer more than one process that would result in an observed material or pattern. however, a subset of students (n=8) did exhibit the ability to consider more than one explanation for their observations. most often, these ideas were presented in response to interviewer prompts over time, rather than as a set of possible mechanisms presented in tandem. this is an important distinction in that we cannot always know if these children abandoned one idea before presenting a new idea, or if ideas were truly held as multiple working hypotheses simultaneously. returning to the idea of rock formation, a fifth grade girl (of5) provided two possible mechanisms for the formation of a shell-filled rock (fig. 1a). her first explanation revolved around lava mixing with seashells: q: you said that the seashells were in the rock; how did the seashells get in the rock? of5: well, maybe if a volcano erupted or something, maybe the lava mixed in the sea shells and became a rock. the interviewer prompted the student to provide other explanations in an attempt to identify multiple hypotheses. although many students articulated just one mechanism, of5 provided a second explanation similar to ideas other students held about the sun baking rocks: of5: maybe it’s clay and seashells hardened. q: how would that happen? of5: maybe it was just left out in the sun with sea shells and it just became really hard and you couldn’t get the sea shells out of it anymore. other students provided multiple explanations without significant prompting. a third-grade boy (jm3) presented a suite of possible mechanisms for the formation of mountains that included meteor impacts, earthquakes, pressure from underground, volcanoes, and wind deposition. this represents many of the common alternative conceptions about mountain formation documented in the literature (e.g., muthukrishna et al., 1993). in the case of jm3, he articulates quite clearly that different mountains are formed in different ways, clearly demonstrating his ability to work with multiple hypotheses at once. in the following international electronic journal of elementary education vol.5, issue 1, 47-61, 2012 56 exchange, jm3 is explaining how mountains can form from meteor impacts, volcanoes pushing up from underground, or from the remnants of old volcanoes: q: you said earlier you think mount everest formed when a crater hit the earth; can you tell me more about that? jm3: if it were formed by a crater, it must have been a whole bunch of craters… a crater is a giant hole in the ground that was formed by a meteorite …[the mountains are] in between most of the craters. … q: do you think all mountains form that way? a: no, i don’t. q: how can you form other mountains? a: …something happening underground, so it pushes all the layers of land up so it forms a mountain. q: what’s happening to push that land up underground? a: maybe it was when a volcano was erupting, or many at the same time. or maybe it was just an old volcano that was made in the sea and then after years, it wasn’t a volcano anymore, because the rocks change over time, so it becomes a mountain instead of a volcano. the student did not change his mind when subsequently probed, but rather stuck with the idea that multiple processes can produce the same effect. this is indicative of reasoning with multiple hypotheses. preference for one hypothesis over others as explained above, all students possessed the ability to identify processes that must have occurred to produce modern observations, and some students were able to generate and hold at once multiple explanatory processes. however, students generally did not reason about which possible cause was the most likely cause, an important aspect of retrodictive reasoning, and perhaps the most difficult. students were comfortable providing explanations for their own ideas, and were willing to argue with others. however, we were unable to identify any discourse in which these children rationalized a preference for one idea over another. this may suggest that elementary-aged children see no need to choose one hypothesis over another or may simply be an artifact of the type of interviews conducted here. reasoning references time beyond human timescales almost every student (n=19) explicitly discussed the importance of time in their explanations. students generally recognized that geologic processes take a long time, although the exact nature of “long” was unclear. because elementary children are unlikely to understand the meaning of specific large numbers, we chose to focus on relative temporal descriptions (i.e., long, short) rather than on absolute ages. in essence, we believe that the absolute numbers stated by interviewees were generally meaningless and should not be over interpreted. as an example, we consider the discussion of how long students thought it would take for dirt to form. this question was posed after a discussion of the formation of dirt from solid rock, initiated either by the interviewee or as an interviewer prompt. not surprisingly, students’ perspective about how long this process might take ranged from a few days to thousands of years. interestingly, students’ ideas were generally well aligned elementary children’s retrodictive reasoning about earth science / libarkin & schneps 57 with the actual amount of time it takes for soils to form. we suggest that this does not indicate strong conceptual understanding of the temporal nature of soil formation, but rather that soil happens to form over timespans that align quite well with the numbers the interviewed students happened to know. although humans can impact the earth system, as evidenced by modern climate change, most earth processes occur without human intervention and certainly the vast majority of earth history passed before humans evolved. as noted above and by other researchers (e.g., blake, 2005), some students were unable to recognize that humans play almost no role in most earth processes. a particularly good example of this is reflected in a discussion between the interviewer and a first-grade student (lm1). in this exchange, the student has explained that a rock looks like it has been “knocked around”: q: how would a stone like this get knocked around? lm1: humans kicking it, maybe. it came out of a volcano, or something, and it’ll hit and crash apart because it came down with such force. q: is there any other way it can get knocked around? lm1: yeah, by humans kicking it. this student is clearly most comfortable with a human cause for the rock’s movement, although a geologic event (volcano) is mentioned in passing. this is similar to tf3 explaining shells found in the interior usa through humans filling up the ocean with sand. the idea that humans cause some geologic events is not limited to the youngest children. for example, a fifth grade student (tf5) is discussing dirt as an unchanging material: q: is the dirt in the dinosaur period the same dirt that we find today? tf5: yes, i think so. it’s possible for dirt to live that long, because…like people, they will get killed and stuff, and they might just die. and plants, anybody could smoosh it, or a tree, they could cut down. but dirt’s so small, nobody would really want to do anything to it. and if they do anything, nothing would really happen. you can’t kill dirt. it’s just there forever. this student views at least some aspects of the earth as being static and unchanging. this discourse also reflects a notion that humans must be involved in changing or destroying dirt. generally, those students who called upon human activities to explain changes, or lack of changes, to the earth were least likely to recognize the importance and scale of geologic time. however, most of the young children interviewed here recognized the importance of time in creating earth phenomena, as evidenced by the data presented in preceding sections, suggesting that deep time in an abstract sense is not outside the reach of young children. conclusion the children interviewed here showed themselves to be remarkably capable of some aspects of retrodictive reasoning, despite the presence of a number of interesting alternative conceptions about earth processes. all interviewed students present evidence of an understanding that patterns present in the modern world are the imprints of processes that have already occurred, and most also explicitly reasoned about geologic time. this reasoning was more nuanced than the simple idea that processes take time to occur. rather, these young students recognized that geologic processes resulting in modern features generally occur in the past and often before humans, or at least modern humans, were living in the same areas as modern features. this understanding of the relationship between modern international electronic journal of elementary education vol.5, issue 1, 47-61, 2012 58 observations and past events is the first prerequisite for retrodictive reasoning, and its presence indicates that young children are capable of making this type of inference about the past from modern evidence. children in this study were often able to describe specific events that might result in modern features and demonstrated the ability to use a variety of knowledge types in the context of their retrodictive reasoning. in some cases, children were drawing analogies from their own observations. this is particularly well exemplified by the young student who reasoned that the hot center of the earth might result from a lack of air, as might occur in a stuffy room on a hot day. similarly, an analogy of ingredients mixed together with a beater to form cookie dough was used to support the idea that the earth’s rotation results in materials mixing together to form rocks. these analogies clearly result from everyday experience. everyday experiences often aligned with domain-general types of information, such as the idea that rocks will harden in the sun; we suggest that this notion comes from physical experience with materials, such as mud, drying and hardening, rather than from instruction. in other cases, such as with the model of states as tectonic plates or people infilling the ocean, the child was clearly pulling from an idea they had been taught in school or heard from the media, an authority, or friend. most of these ideas were quite domain-specific. for example, students had very specific ideas about the ways in which mountains form that were unlikely to derive from everyday observations, such as the idea that meteors generate craters that are mountains. while children might experiment in the classroom with craters in simple experiments using for example flour and golfballs, meteor impacts are not likely a phenomenon directly observed by these students. the role of everyday experience and learned ideas in influencing a child’s ability to reason retrodictively warrants further research. most importantly, the ways in which everyday experience could be used to encourage retrodictive reasoning should be explored. the presence of alternative conceptions about earth systems, of which there were many documented in even this relatively small number of interviews, did not seem to interfere with the ability to reason retrodictively. certainly, alternative conceptions will interfere with the ability to ascertain an accurate reason for earth features and processes. however, the ability to reason despite significant alternative conceptions suggests that retrodictive reasoning in young students can be fostered even when children are cognitively unable to grasp some of the more complicated principles underlying strong scientific literacy. gaining deep understanding of the fundamental laws and principles that govern the earth system is vital for the reasoning, both retrodictive and predictive, that is needed for decision-making about human impacts on earth. • • • acknowledgements we thank the entire science media group team (including, notably, alex griswold) as well as all teachers and students who participated in this study. we especially thank our interviewees for sharing their ideas and passion for learning. an anonymous reviewer also provided helpful suggestions that significantly improved our discussion. this work was supported in part by the annenberg foundation through annenberg media. julie libarkin is an associate professor of geological sciences at michigan state university and director of the geocognition research lab. she specializes in assessment of student ideas and communication of science, with a specific focus on earth science. elementary children’s retrodictive reasoning about earth science / libarkin & schneps 59 matthew schneps is the director of the laboratory for visual learning at the harvard-smithsonian center for astrophysics. his interests include science education, student ideas, educational media, learning differences, and cognitive psychology, with a focus on non-verbal learning. references argow, b., reilly, j., & schneps, m. (2004). essential science for teachers: earth and space science. harvardsmithsonian center for astrophysics. science media group. ault, c. r. (1982). time in geological explanations as perceived by elementary-school students. journal of geological education, 30(5), 304–09. ault, c. r. (1984). the everyday perspective and exceedingly unobvious meaning. journal of geological education, 32(2), 89–91. ault, c. r. (1998). criteria of excellence for geological inquiry: the necessity of ambiguity. journal of research in science teaching, 35(2), 189–212. barrow, l., & haskins, s. (1996). earthquake knowledge and experiences of introductory geology students. journal of college science teaching, 26(2), 143–46. blake, a. (2005). do young children’s ideas about the earth’s structure and processes reveal underlying patterns of descriptive and causal understanding in earth science? research in science & technological education, 23(1), 59–74. 0 cheek, k. a. (2010). commentary: a summary and analysis of twenty-seven years of geoscience conceptions research. journal of geoscience education, 58(3), 122–134. dahl, j., anderson, s. w., & libarkin, j. c. (2005). digging into earth science: alternative conceptions held by k-12 teachers. journal of science education, 12, 65–68. dal, b. (2007). how do we help students build beliefs that allow them to avoid critical learning barriers and develop a deep understanding of geology? eurasia journal of mathematics, science and technology education, 3(4), 251–269. delaughter, j. e., stein, s., stein, c. a., & bain, k. r. (1998). preconceptions abound among students in an introductory earth science course. eos, transactions american geophysical union, 79(36), 429– 432. dodick, j., & orion, n. (2003). cognitive factors affecting student understanding of geologic time. journal of research in science teaching, 40(4), 415–442. dove, j. e. (1998). students’ alternative conceptions in earth science: a review of research and implications for teaching and learning. research papers in education, 13(2), 183–201. gosselin, d. c., & macklem-hurst, j. l. (2002). pre-/post-knowledge assessment of an earth science course for elementary/middle school education majors. journal of geoscience education, papers in natural resources, 50(2), 169–175. happs, j. c. (1984). soil genesis and development: views held by new zealand students. journal of geography, 83(4), 177–180. kali, y., orion, n., & eylon, b.-s. (2003). effect of knowledge integration activities on students’ perception of the earth’s crust as a cyclic system. journal of research in science teaching, 40(6), 545– 565. king, c. (2008). geoscience education: an overview. studies in science education, 44(2), 187–222. kortz, k. m., & murray, d. p. (2009). barriers to college students learning how rocks form. journal of geoscience education, 57(4), 300–315. international electronic journal of elementary education vol.5, issue 1, 47-61, 2012 60 kusnick, j. (2002). growing pebbles and conceptual prisms understanding the source of student misconceptions about rock formation. journal of geoscience education, 50(1), 31–39. lawton, j. (2001). earth system science. science, 292(5524), 1965–1965. doi:10.1126/science.292.5524.1965 lewis, e. b., & baker, d. r. (2009). a call for a new geoscience education research agenda. journal of research in science teaching, 47(2), 121-129. libarkin, j., anderson, s., dahl, j., beilfuss, m., boone, w., & kurdziel, j. (2005). college students’ ideas about geologic time, earth’s interior, and earth’s crust. journal of geoscience education, 53(1), 17– 26. libarkin, j. c., & kurdziel, j. p. (2006). ontology and the teaching of earth system science. journal of geoscience education, 54(3), 408. lillo, j. (1994). an analysis of the annotated drawings of the internal structure of the earth made by students aged 10–15 from primary and secondary schools in spain. teaching earth science, 19, 83– 87. marques, l., & thompson, d. (1997). misconceptions and conceptual changes concerning continental drift and plate tectonics among portuguese students aged 16‐17. research in science & technological education, 15(2), 195–222. montangero, j. (1996). understanding changes in time: the development of diachronic thinking in 7-12 year old children. taylor & francis. muthukrishna, n., carnine, d., grossen, b., & miller, s. (1993). children’s alternative frameworks: should they be directly-addressed in science instruction? journal of research in science teaching, 30(3), 233–248. nelson, b. d., aron, r. h., & francek, m. a. (1992). clarification of selected misconceptions in physical geography. journal of geography, 91(2), 76–80. norris, s. p., guilbert, s. m., smith, m. l., hakimelahi, s., & phillips, l. m. (2005). a theoretical framework for narrative explanation in science. science education, 89(4), 535–563. pyle, e. j. (2008). a model of inquiry for teaching earth science. electronic journal of science education, 12(2). retrieved from http://ejse.southwestern.edu/article/view/7770 ross, k. e. k., & shuell, t. j. (1993). children’s beliefs about earthquakes. science education, 77(2), 191– 205. schoon, k. (1995). the origin and extent of alternative conceptions in the earth and space sciences: a survey of pre-service elementary teachers. journal of elementary science education, 7(2), 27–46. sunderlin, d. (2009). integrative mapping of global-scale processes and patterns on “imaginary earth ” continental geometries: a teaching tool in an earth history course. journal of geoscience education, 57(1), 73–81. thomson, n., & chapman beall, s. (2008). an inquiry safari: what can we learn from skulls? evolution: education and outreach, 1(2), 196–203. trend, r. (1998). an investigation into understanding of geological time among 10‐ and 11‐year‐old children. international journal of science education, 20(8), 973–988. trend, r. (2000). conceptions of geological time among primary teacher trainees, with reference to their engagement with geoscience, history, and science. international journal of science education, 22(5), 539–555. trend, r. d. (2001). deep time framework: a preliminary study of u.k. primary teachers’ conceptions of geological time and perceptions of geoscience. journal of research in science teaching, 38(2), 191– 221. elementary children’s retrodictive reasoning about earth science / libarkin & schneps 61 trend, r., everett, l., & dove, j. (2000). interpreting primary children’s representations of mountains and mountainous landscapes and environments. research in science & technological education, 18(1), 85–112. wiser, m., & amin, t. (2001). “is heat hot?” inducing conceptual change by integrating everyday and scientific perspectives on thermal phenomena. learning and instruction, 11(4–5), 331–355. zimmerman, c. (2000). the development of scientific reasoning skills. developmental review, 20(1), march 2000, 99-149. international electronic journal of elementary education vol.5, issue 1, 47-61, 2012 62 www.iejee.com this page is intentionally left blank international electronic journal of elementary education, 2015, 7(3), 333-354. issn:1307-9298 copyright © iejee www.iejee.com prospective elementary school teachers’ views about socioscientific issues: a concurrent parallel design study muhammet özden dumlupınar university, turkey received: 9 february 2015 / revised: 1 april 2015 / accepted: 11 may 2015 abstract the purpose of this research is to examine the prospective elementary school teachers’ perceptions on socioscientific issues. the research was conducted on prospective elementary school teachers studying at a university located in western turkey. the researcher first taught the subjects of global warming and nuclear power plants from a perspective of socioscientific issues in the science and technology education course and then conducted the research. concurrent parallel design, one of the mixed-method research approaches, was used to conduct the research. in this context, semistructured interviews were conducted with eight teachers in the qualitative strand of the study to explore the phenomenon. the data obtained from the interviews were analyzed using thematic analysis. during the quantitative strand of the research, 113 prospective teachers were administered a questionnaire form. the results of the study revealed that none of the participating prospective teachers mentioned about the religious and cultural characteristics of socioscientific issues, and they need training about how to use socioscientific issues in teaching. keywords: science education, scientific literacy, socioscientific issues, mixed methods, concurrent parallel design introduction the main goal of science education is to enhance scientific literacy (american association for the advancement of science [aaas], 2009; ministry of national education [mone], 2013; national research council [nrc], 1996) and scholars argued that scientific literacy can be achieved by integrating socioscientific issues (ssi) into science education (ekborg, ottander, silfver, & simon, 2013; kolstø, 2001; sadler & zeidler, 2005a, 2005b; zeidler & nichols, 2009). ssi are contemporary controversial issues with no established consensus on, which arise from advances in science and technology and have individual, social, political, economic, ethical and moral aspects (ozden, 2011). these issues can alternatively be defined as the issues which are complex, open-ended, have no definite solutions and  muhammet özden, dumlupinar university, faculty of education, department of elementary education, kutahya, turkey phone: +90 (274) 422-4612 e-mail: muhammetozden@gmail.com http://www.iejee.com/ international electronic journal of elementary education vol.7, issue 3, 333-354, 2015 334 emerge in the form of controversial dilemmas (sadler, 2004), that people face in their daily lives (kolstø, 2001), that focus on scientific content and the social dimension of the scientific content (topcu, 2010). the definitions suggest that in an educational approach based on ssi, students are faced with issues incompatible with their own belief systems or containing different scientific, social, and moral perspectives (zeidler, sadler, applebaum, & callahan, 2009). ssi are generally related with advances in biotechnology and environmental problems (sadler & zeidler, 2005a). for example, deforestation, genetically modified foods (foong & daniel, 2013), climate change (morris, 2014), cloning, nuclear energy, depletion of the ozone layer, and epidemics can be specified as ssi (pedretti, 2003). in addition, some controversial issues such as embryo selection, stem cell, tissue or organ transplantation between two distinct species are acknowledged as ssi (levinson, 2006). these issues are employed by science educators as current and interesting contexts, as well as being considered as significant social problems (topcu, yilmaz-tuzun, & sadler, 2011). it can be asserted that with the introduction of 3rd-8th science teaching curriculum (mone, 2013) in 2013, turkey had an opportunity for employing ssi in teaching. there are certain reasons for employing ssi in science education. first of all, ssi are a means of improving scientific literacy (sadler, 2009). ssi involve political, personal, and moral issues, as well as scientific claims and arguments. however, for many ssi, basic scientific claims are controversial. therefore, when making decisions about these issues individuals should consider two main aspects, one being political/ethical and the other scientific (kolstø et al., 2006). for example, it may be political decision when it comes to permitting to trade genetically modified food. on the other hand, whether genetically modified foods are a threat to human health is a scientific question, which receive different scientific explanations. allowing the students to evaluate and construct their thought on the scientific descriptions, views, and arguments brought about the issues can be an example to the development and utilization of scientific literacy skills. secondly, ssi help students understand the social, moral, political and economic effects of science (dawson, 2001) by providing a context for a better understanding of both the epistemological beliefs and science (zeidler, herman, ruzek, linder, & lin, 2013). thus, it becomes easier for the students to understand the nature of science (jones et al., 2011). students will realize that they use personal beliefs and values as well as scientific knowledge, while they are interpreting and evaluating evidence related to ssi, and offering solutions to these problems. in a curriculum based on ssi, for the students to use scientific knowledge together with their personal beliefs may help them realize the procedures and processes of science. thus, it becomes easier to teach the nature of science within the context of ssi. thirdly, ssi help enhancing the students’ abilities to make decisions based on evidence, to make argumentation, and to debate (ideland, malmberg, & winberg, 2011), thus improving their analytical thinking skills. since ssi are complex, open-ended, controversial problems with no definite answers, the possible solutions to the emerging dilemmas can be discovered if only multiple perspectives are employed. on the other hand, when limited and controversial sources of information are taken into consideration, students and ordinary citizens can develop their own cognitive constructs and produce explanations in response to the controversial scientific problems if they can develop informal reasoning skills (sadler, 2004). finally, ssi make contributions not only to students’ cognitive development but also to their emotional and social development (topcu, 2010; topcu, sadler, & yilmaz-tuzun, 2010). science teaching based on ssi supports the character development (zeidler et al., prospective elementary school teachers’ views about socioscientific issues / özden 335 2009) and citizenship skills (barrue & albe, 2013; lee et al., 2013) of individuals by focusing on the discourse and regarding the moral and ethical issues. thus it is apparent that use of ssi in science education has four main goals: to improve scientific literacy, to provide an understanding about nature of science, to enhance higher order thinking skills by promoting cognitive development, and finally, to ensure emotional and social development. the potential of ssi to perform multiple goals simultaneously, to offer students interesting and authentic learning experiences has led an increasing interest among science educators into this subject and facilitated its inclusion in the curriculum. the movement of ssi has emerged in the united states (saunders & rennie, 2013). however there have been an increasing interest at the international level and many research carried out. among these researches, the effect of ssi on scientific literacy (kolstø et al., 2006; ritchie, thomas, & tones, 2011) and learning the nature of science (albe, 2008, eastwood et al., 2012; khishfe 2012, 2014; sadler, chambers, & zeidler, 2004) have become the two important research topics. another important field of researches included the attempts to understand the relationship between ssi and cognitive skills. in this context, some commonly studied topics included argumentation in ssi (dawson & venville, 2013), the transfer of argumentation skills (foong & daniel, 2013), decision making (greschner, hasselhorn, & bögeholz 2013; zeidler et al., 2009), epistemological (zeidler et al, 2013), moral (sadler & zeidler, 2004) and informal reasoning patterns (topcu et. al, 2010; topcu et al., 2011). similarly, the importance of content knowledge in terms of informal reasoning and argumentation skills has been studied (sadler & donnelly, 2006; sadler & zeidler, 2005b). fewer studies investigated the relationship between ssi and learning outcomes as another component of the cognitive skills. in this respect, researches have focused on the effect of ssi in facilitating learning (rudsberg, öhman, & östman, 2013) and on the learning outcomes (ottander & ekborg, 2012). previous research investigated ssi relationship with affective variables, in addition to the cognitive ones. in this respect, researchers have investigated the impact of the ssi students interest in and attitudes towards science lessons (albe, 2008; ottander & ekborg, 2012; thomas, ritchie, & tones, 2011) and prospective teachers’ perceived competencies on ssi (kara, 2012; kilinc et al. 2013; lee, abd-el khalick, & choi, 2006). in a research study, an attitude scale towards ssi was developed (topcu, 2010). other research studies focused on difficulties teachers faced in classroom discussions (day & bryce, 2011), teachers' views on ssi (ekborg et al., 2013), the role of ssi in citizenship education (barrue & albe, 2013; lee et al., 2013), how ssi are used in classes with students representing different socioeconomic status and ethnicities (ideland et al., 2011). one study evaluated how ssi are handled in textbooks (morris, 2014). researches in the literature can be grouped under two categories according to the use of ssi: using socioeconomic issues as the goal and using socioeconomic issues as an instrument (topcu, mugaloglu, & guven, 2014). in a more detailed analysis, the focus of the studies on ssi can be categorized as (i) the nature of science and scientific literacy, (ii) argumentation, reasoning and decision-making processes, (iii) content knowledge, (iv) views and sense of efficacy in using ssi in teaching, (vi) interest in and attitudes towards in science lessons. in this contexts, data were obtained from teachers (day & bryce, 2011; lee et al., 2006), prospective teachers (kara, 2012; topcu et al., 2010), secondary (ideland et al., 2011; khishfe, 2014) and high school (eastwood et al., 2012; thomas et al., 2011) students. however, there is no study which focuses mainly on elementary school teachers regarding the ssi. nevertheless, alacam-aksit (2011) conducted a research to detect the prospective elementary school teachers' on teaching of ssi. many research studies about ssi have not been directly associated elementary school teachers or prospective elementary school teachers. this implies that while the rapidly international electronic journal of elementary education vol.7, issue 3, 333-354, 2015 336 growing literature on ssi puts forward new implications for science education, the roles and functions of elementary school teachers have not been discovered yet. because of the reasons specified above, new researches should be conducted to determine and improve the views prospective elementary teachers who will be responsible for guiding the science lessons in the future. determining the views of prospective elementary school teachers may help evaluating the problems and views to affect their instructional practices. moreover, such an evaluation may contribute to take necessary measures in the relevant field and to promote the quality of teaching activities to be planned for the students of prospective elementary teachers in the future. therefore, determining the prospective elementary school teachers’ views on ssi, their perceptions about the characteristics of ssi, and their beliefs about their roles as teachers will form the basis for an effective science teaching. in this respect, the aim of the present research is to examine the perceptions prospective elementary school teachers on ssi. this research study seeks answers to the following questions:  what are the perceptions of prospective elementary school teachers about ssi?  what are the views of prospective elementary school teachers about the use of ssi in science teaching at elementary school?  do prospective elementary school teachers' views on the use of ssi in science education differ significantly by gender to academic success? method design present study was conducted based on mixed methods design (tashakkori & teddlie, 2010). mixed methods research merges qualitative and quantitative data to answer the research question (creswell, 2014). there are other terms used to refer to mixed methods such as integration, synthesis, qualitative and quantitative methods, multiple methods, and mixed methodology (byrman, 2006; tashakkori & teddlie, 2010). in the present study, mixed methods was used to overcome the restrictions of using either of the qualitative or quantitative approaches alone, and to find a comprehensive answer to the research question. more specifically, the convergent parallel design, one of the mixed methods (creswell & plano clark, 2011) was used in the study. in this design, qualitative and quantitative data were collected in a parallel manner, but analyzed independently. next, qualitative and quantitative results were mixed to make an overall interpretation about the research question (creswell & plano clark, 2011; creswell, 2014). there are some reasons for using the convergent parallel design. first reason is the need for different but complementary data regarding the research question, which is believed to lead to obtain a more effective answer to the research question. second reason is to overcome the limitations to emerge when qualitative and quantitative would be used alone. third reason is that this method allows comparing the qualitative and quantitative data in order to increase the internal validity of the study. in this respect, thanks to the qualitative data participants were able to comment on and explain the research topic in a detailed manner with their own words, and quantitative data made it possible to understand the perceptions of a larger group on ssi in general. in the present study, both qualitative and quantitative methods were given the equal priority (creswell & plano clark, 2011). that is, qualitative and quantitative procedures of the study had equal responsibility in answering the research questions. the symbolic representation of the design is qual+quan (morse, 1991), which is displayed in figure 1: prospective elementary school teachers’ views about socioscientific issues / özden 337 figure 1. symbolic representation of research design (creswell, 2015). qualitative strand context and participants present research was conducted with the participation of prospective elementary school teachers studying at a state university in western turkey. the researcher has taught science and technology teaching course in the program mentioned above. ssi and their use in education was one of the topics involved in the course content. a two-week period was allocated for ssi in the science and technology teaching course program. the researcher as the instructor discussed the topic of global warming during the first week and nuclear energy during the second week with the students. these rather current and interesting topics were selected because recently they have been discussed in the society broadly with their political, economic, ecological and scientific aspects. participants were asked to find scientific articles offering different arguments about both topics, to read this article critically, and to use these articles while forming and defending their own ideas. students participated into discussions directed by researcher after making these preparation before coming to the class. at the end of the second week of the discussions, researcher informed the participants that the topics discussed are named ssi in the relevant literature. next, the participants were asked to reflect on the characteristics of ssi, the ssi that can be handled at elementary school level, the roles that teachers and students should have while addressing ssi based on the classroom discussions and to form their own opinions. at every stage of the research, the researcher refrained from disclosing his own ideas or giving information about ssi, but directed the participants to express their opinions based on political, social, economic, and moral aspects. during the week after the classroom discussions were completed, the researcher announced the participants that he would like to conduct semi-structured interviews with to examine the educational characteristics of ssi, and he asked the volunteering participants to give feedback about their intent to take part in the study by sending an email, visiting the researcher’s office, or just calling. since the research was conducted with the natural members of the researcher’s class, convenience sampling method was used in the research (yildirim & simsek, 2013). after the announcement, 8 of the prospective teachers informed the researcher about their voluntary participation to the research either by visiting the researcher’s office or sending a message via social network (facebook) although it was not an announced way of feedback. next, a timetable was arranged with the participants according to their convenient days and times, and semistructured interviews were conducted according to this meeting schedule. among the participants, five were women and three were men. in terms of their academic, while the 7 qualitative data collection and analysis quantitative data collection and analysis compare or relate interpretation international electronic journal of elementary education vol.7, issue 3, 333-354, 2015 338 participants had average grade points of 3.00 or more, only one had an average grade below 2.99. data collection and analysis the qualitative data of the study was collected through interviews (spradley, 1979). interviews are effective data collection tools enabling to obtain and record the individuals’ or groups’ views, feelings, ideas, values, attitudes and beliefs about their experiences and social worlds in in their own words (saldaña, 2011). it is known that there are different approaches about classifying the interviews (patton, 2001; ten have, 2004; spradley, 1979; yildirim & simsek, 2013). in the present study semi-structured interviews were used (yildirim & simsek, 2013). open-ended questions are used in semi-structured. the main responsibility of the interviewer is to explore the participants’ responses to openended questions and to build the research on the basis of their responses (seidman, 2006). the semi-structured interview form consisted of four open-ended questions. these questions are: 1) how do you describe in your own words the concept of ssi? 2) can you give examples of ssi that can be used in science and technology courses? 3) what can be the contribution of involving ssi into science and technology course? 4) what should be the roles of teachers in teaching ssi? interview questions were derived from notes the researcher took during class discussions and the relevant literature. all semi-structured interviews were conducted face to face with each participant individually. data were analyzed using thematic analysis (gibson & brown, 2009; yildirim & simsek, 2013). thematic analysis requires the analysis of the data according to common features, relationships, and differences in the dataset (gibson & brown, 2009). thematic analysis is a descriptive strategy which facilitates the search of patterns of experiences present in the qualitative dataset. therefore, the outcome of the thematic analysis is a structure which enables the identification and integration of existing patterns (ayres, 2008). in the thematic analysis, themes do not involve a process of simply counting the words (firmin, 2008), but that of examining the structures both hidden and apparent in the data (vaismoradi, turunen, & bondas, 2013). the following sequential steps were followed during the thematic analysis: 1) identification of the data by the researcher, 2) the creation of basic codes, 3) establishment of leading themes, 4) revising themes 5) identifying and naming themes, and 6) writing the research report (braun & clarke, 2006). in this respect, clusters of related themes were examined within the data set and two major themes were produced as the end of the data analysis: "the nature of socioscientific issues" and "educational use of socioscientific issues". quantitative strand samples sometimes mixed methods researchers work on completely different samples in qualitative or quantitative strands of their research. however, a good mixed methods research is carried out on different samples selected from within the same population at every stage. at this point, researchers should be careful not to involve the same individuals into both samples (creswell, 2014). in this respect, no sampling strategy was used and all prospective teachers other than the ones participating in the qualitative strand of the study were invited to participate in the quantitative strand of the study. a total of 113 prospective teachers other than those participated in semi-structured interviews agreed to participate in the study. demographics of prospective teachers participating in the quantitative strand are presented in table 1. prospective elementary school teachers’ views about socioscientific issues / özden 339 table 1. demographics of the participants attending the quantitative strand variables f % gender woman 68 60.2 man 44 38.9 missing data 1 .9 grade average 2.99 and below 68 60.2 between 3.00-4.00 44 38.9 missing data 1 .9 total 113 100 among the participants 60.2% were female and 38.9% were men. on the other hand, 60.2% of them had 2.99 or lower gpas and 38.9% had a gpa between 3:00 and 4:00. one participant did not answer questions about gender and gpa. collection and analysis of data quantitative data were collected using “socioscientific issues in science course questionnaire", which was developed by the researcher. to develop the questionnaire items, firs the literature was examined. in this context, an item pool was formed using the questionnaire forms used in lee et al. (2006) and kara (2012). relevant items were evaluated by the researcher in terms of content and those items which are not compatible with the research questions, not clearly understood, not specific to the topic, and contain multiple statements, were discarded. the draft questionnaire form was consulted to an expert panel to check its content validity and necessary corrections were made in accordance with the feedback received. to test the intelligibility of the questionnaire form, a pilot study was conducted with 52 students in the elementary science education program and after necessary modifications were made questionnaire preparation process was finalized. the questionnaire was used as a structured written interview form to obtain participants’ views about ssi. in this respect, since it is not proper to refer to any internal reliability or construct validity to estimate a total score as in the scales (erkus, 2011), no reliability coefficient estimation or factor analysis were done on the questionnaire items. the questionnaire was composed of three parts. in the first part, there were two questions asking for the prospective teachers demographics. in the second section, there was a supplementary knowledge which describes the characteristics of ssi with examples. the third part consists of 13 items asking for prospective teachers’ views about ssi. prospective teachers were asked to select one of the responses including "strongly disagree," "disagree," "undecided", "agree" or "strongly agree". participants completed the questionnaires during their regular classes. the data obtained from the quantitative strand of the research was analyzed using frequency, percentage, and mean scores. chi-squared test was used in order to test weather prospective teachers’ views on ssi differ by gender and academic achievement scores (buyukozturk, 2005; tabachnick & fidell, 2000). international electronic journal of elementary education vol.7, issue 3, 333-354, 2015 340 findings findings were presented below under two sections as required by the mixed methods design. the findings of the qualitative strand of the research the nature of socioscientific issues the participants described the ssi as the current events which affect individuals, have no consensus on, include understanding the risks and probabilities, are structured in the form of open-ended dilemmas, necessitate moral and ethical choices to be made, have more than one alternative solutions (having no definite solution, however). sophia described ssi as issues, which emerge as a result of scientific developments and affect individuals in a society. sophia puts her thought in more detailed way as follows: "ssi are the ones with scientific basis existing in a society. they are the issues directly or indirectly affect the society.” jackson regards ssi as the issues with no consensus on. to him, the ssi are related with understanding certain risks and possibilities. he comments on the issue as: “ssi are the issues whose pros and cons have been discussed for some time, and hardly any conclusion was made upon". emma also referred to the aspects of ssi in terms of understanding the risks and possibilities, stating that "ssi are the ones about which everybody has some knowledge, but no consensus has been established about the benefits and costs." emma did not mention about the controversial nature of ssi in terms of understanding the risks and possibilities alone. in addition, it is remarkable that she put that individuals in the society are aware of these issues and are informed, through limited, about these issues. olivia also stated supporting ideas. she stated that ssi are “the ones on which everyone have some idea, about which one can talk in a classroom or community. generally it is a current issue”. olivia also mentioned that everybody knows about the ssi, as suggested in the previous thought. on the other hand, she also recognized the social impact of these problems. this is because ssi are not only a tool to be used in instructional educational environment, but they are also important in everyday social relations of individuals in a society. another important emphasis was on the actuality of the ssi. participants were observed to refer frequently to their in-class experiences while voicing their views. the fact that participants voice similar views may suggest that they gain similar learning outcomes from in-class practices. ava noticed that ssi involve certain uncertainties and thus they have no definite solutions. similarly, ava argued that ssi often arise in the form of media news, stating that "i believe that they are the issues that media publicizes to some extent and on which we cannot make to a definite conclusion." similarly, isabella mentioned that ssi emerge in the form of media news, stressing that individuals are informed about them via internet and social networks. unlike other participants jackson was no mention that requires ethical choices of ssi. to him, contemporary developments in science and technology are threatening the future of the humankind, because human life is entirely built on mobile phones, computers and other smart systems and they are likely to threaten the future of human existence. he explains that "google has purchased a robot company... for example, some think that eventually the future will turn out to be a land of robots and robot fights. above all, if you make robots become completely human-like, thinking and acting like humans, they could become a threat to the human race in the future." therefore, he believes that integrating ssi into science lessons would enable students contemplate on the ethical consequences of the scientific and technological application. prospective elementary school teachers’ views about socioscientific issues / özden 341 two of the participants, sean and connor, defined the ssi as scientific events. sean stated that "... a socioscientific issue is a scientific event concerning normal people. scientific event which interests people." connor on the other hand explained a socioscientific issue as "a scientific event which affects our lives, our being, that is our social life, and the world universally". ssi are the dilemmas concerning economy, environment, politics, moral and ethical subjects, and bears in conflicts at least in one of these fields. as a matter of fact, while scientific developments emerge as the activities of scientists, ssi are the problems are outside the scope of the world of science and they have been debated for long and affecting the daily life of an ordinary individual. in this sense, it can be asserted that sean and connor fall into a misconception in defining ssi as "scientific events". educational use of socioscientific issues participants expressed their views about the benefits of using ssi in science teaching and teacher roles under the theme of educational use of ssi. participants believed that the benefits of ssi are closely related with providing students with higher order thinking skills. in this respects, they stated that ssi can have students gain such higher order thinking skills as argumentation, opinion development, scientific process skills and creativity. likewise, they thought that, though limited, integrating ssi into science teaching can help students think on their citizenship responsibilities. sean expresses his point about the positive contribution of use of ssi on students’ argumentation ability as such: "for example, i think differently about nuclear energy. i believe they should be built. however, if one of my friends who opposes nuclear power plants can make a pretty good argumentation, i can be convinced (...) i may change my mind finding his arguments wise." to sean, discussing the ssi in class necessitates the students to use information resources to create the necessary basis for their opinion. thus, evidence-based discussions by the students becomes a means of analyzing different views, and evaluating and developing opinions. similarly, olivia associated the use of ssi in science lessons with the creation of argumentation and development of opinions. she believes that in order to engage in class discussions and create a foundation for the defended opinion, students would read scientific articles, and be able to disprove each other’s thesis during the class discussions and sometimes the processes may end up with the development of the initial opinion, i.e. adoption of the opposing opinion. olivia puts in her thoughts as follows: "we read an article before coming to the class. i was indecisive about whether nuclear power plants should be founded or not, but after i read the article i dominantly got idea that they should be built. while listening to the opponents’ ideas, you may learn something new or the opposite party can disprove your thesis (...) you can adopt opposite views. there may be such changes in your opinion." connor claimed that ssi cannot be taught directly saying “we are not going to tell these directly. we have to provide students with perspectives, scientific perspectives, about ssi. hence, it can be asserted that connor accepts ssi not as an educational goal, but as a context to be used to achieve a goal. emma believes that ssi help students gain scientific thinking skills, explaining that "scientific process skills can be improved, and children's thinking skills can be improved." similarly, sophia believes that ssi should be used to develop reasoning ability among students and to help them notice different viewpoints. jackson and sean highlighted that integrating ssi into lessons can work in maximizing the imagination and creativity of the students. ava believes that ssi may help the students contemplate on their individual responsibilities as good citizens. ava pointed that ssi such as genetically modified foods, organ transplantation, global warming, nuclear power plants can be used in science international electronic journal of elementary education vol.7, issue 3, 333-354, 2015 342 education and these issues can lead the learners inquiry the answers of such questions as "what is my responsibility in this issue? what would be my responsibility? what can i do myself in this matter?" participants also stated that teachers should have certain roles in the course of using ssi in science education. according to the participants, teachers should have content knowledge, not impose their views on students, guide students, lead the discussions, select challenging problems for the students, get prepared before the lessons, ask intriguing and thought-provoking questions. so, according to the views of the participants, teachers should have content knowledge about ssi, as well as the pedagogical competence that accompanies this content knowledge. emma points out that a teacher needs to have content knowledge, if she is to integrate ssi into science lessons. to her, a teacher's role as a guide requires making necessary explanations, offering resources for the students to acquire knowledge and being impartiality. she expresses her views on this issue as follows: "i think teachers should play the role of guide, just like you. she should not disclose her opinion first, but listen to students. (...) the teacher should give information on some issues as you do. (...) i would give resources about the topics in advance, and ask students to explore the topic in advance. " sean also believes teachers should tell their views while discussing the socioscientific discussing issues. he justifies himself stating that “because every student imitate their teachers." also sean emphasized the importance of traditional role of teachers as “the transferor of knowledge”. however, he stressed the importance of the information given to the students should not be in the form of a detailed presentation, but the students should discover the details. following is sean’s other views on the use of ssi: "while selecting the topics, everything should be considering including students’ age, level and context. if we bring in a big socioscientific problem for student discussion, let alone improving students’ problem solving abilities, they cannot even speak as they are shocked. this is because they cannot find any views." olivia emphasizes the need that teachers should select topics suitable for learners’ level, while isabella mentioned that the language used should be appropriate for student's level. while ava points out that teachers should arrange their questions very well before the lesson, sophia associated teacher roles with the characteristics of ssi, explaining that "since these subjects are open-ended, teachers should set a framework. teachers should guide students well. teachers should be guiding their students, but should not express a definite opinion. students should ask the students to freely defend and express their opinions.” as it is understood, almost all of the participants emphasized the guidance role of the teachers, and stated that especially in classes with young students, teachers have the responsibilities to access to resources, to lead to class discussions, to redirect the discussion when students deviate from the subject and to make theoretical explanations to some extent. the findings relating to the quantitative strand of the research participants views about the use of socioscientific issue in primary science education participants commented on the use of ssi in primary science education by responding to the questionnaire items. the participants' responses for each item are presented in table 3. prospective elementary school teachers’ views about socioscientific issues / özden 343 table 3. participants’ views on the characteristics of ssi s tr o n g ly d is a g re e d is a g re e in d e ci si v e a g re e s tr o n g ly a g re e m 1) successful students would be more interested in ssi in science lessons. f 13 39 16 40 3 2.82 % 11.7 35.1 14.4 36.0 2.7 2) elementary students are not mature enough to be interested in ssi. f 6 39 31 28 8 2.93 % 5.4 34.8 27.7 25.0 7.1 3) science lessons are more suitable for ssi than other lessons. f 4 9 19 58 19 3.72 % 3.7 8.3 17.4 53.2 17.4 4) integrating ssi into science lessons is not compatible with the essence of science course. f 28 63 11 6 3 2.03 % 25.2 56.8 9.9 5.4 2.7 5) teachers are not competent in integrating ssi in science lessons. f 2 18 38 46 8 3.35 % 1.8 16.1 33.9 41.1 7.1 6) it is hard for primary students to understand ssi. f 4 30 17 37 3 3.05 % 4.4 33.0 18.7 40.7 3.3 7) integrating ssi would increase the primary students’ interest in science lessons. f 2 11 19 63 18 3.74 % 1.8 9.7 16.8 55.8 15.9 8) teachers can answer easily the student questions about, ssi. f 2 23 36 45 6 3.26 % 1.8 20.5 32.1 40.2 5.4 9) prospective teachers should be trained about ssi. f 4 3 3 54 43 4.15 % 3.6 2.7 7.1 48.2 38.4 10) integrating ssi into science education would increase scientific literacy. f 6 9 9 56 32 3.88 % 5.4 8.0 8.0 50 28.6 11) integrating ssi into science education means simplifying science education. f 18 37 28 25 5 2.66 % 15.9 32.7 24.8 22.1 4.4 12) i think primary school students can learn science better by discussing ssi. f 3 10 14 66 19 3.78 % 2.7 8.9 12.5 58.9 17.0 13) ssi should definitely be involved in science lessons. f 3 5 13 55 37 4.04 % 2.7 4.4 11.5 48.7 32.7 an analysis of the table 3 reveals prospective teachers have positive views on the use of ssi in science education and tend to strongly agree to the relevant items. the item that participants agreed the most was "prospective teachers should be trained about ssi" (m = 4.15). accordingly, it can be understood that prospective teachers have highly in need of being trained about ssi. despite this educational need, prospective teachers seem to believe in the importance of integrating ssi in science lessons. this judgement is supported by the following findings: "ssi should definitely be involved in science lessons." (m = 4:04), "integrating ssi into science education would increase scientific literacy." (m = 3.88), "i think primary school students can learn science better by discussing ssi." (m = 3.78) and "integrating ssi would increase the primary students’ interest in science lessons." (m=3.74). according to these findings, it can be stated that prospective teachers think that integrating ssi into science education would increase students’ interest in learning science, facilitate learning science, and improve scientific literacy. on the contrary, it was seen that prospective teachers agreed less with the negative statement about the integration of ssi into science education. among them the item international electronic journal of elementary education vol.7, issue 3, 333-354, 2015 344 participants agreed relatively the least was “integrating ssi into science lessons is not compatible with the essence of science course.” (m = 2.03). in other words, it can be said that participants do not find it incompatible with the nature of science education to integrate ssi into science education. another item that prospective teachers agreed rather at a low level was "integrating ssi into science education means simplifying science education." (m = 2.66). however, it is a remarkable finding that while participants generally agreed at a low level to this item, about one third of them agreed that “integrating ssi into science education simplifies science education” and a good number of them (24.8%) were indecisive about this issue. other items which the participants had rather low levels of agreement included "successful students would be more interested in ssi in science lessons.” (m = 2.82), "elementary students are not mature enough to be interested in ssi." (m = 2.93) and "it is hard for primary students to understand ssi." (3.05), respectively. prospective teachers’ views about ssi according to their gender and academic achievement chi-square test was used to find whether prospective teachers’ views on the ssi differ significantly according to their gender and academic success. however, since in the first attempt the number of cells which had expected count less than five exceeded 20% of total number of cells, some categories were merged and the chi-square analysis was repeated (buyukozturk, 2005). for this purpose "strongly agree" and "agree" categories were merged under "agree" category, and "strongly disagree" and "disagree" categories were merged under "disagree" category. as a result of the chi-square analysis for gender variable, a significant difference was found only for the item “elementary students are not mature enough to be interested in ssi” [χ²(2) = 6.51, p = .038]. the analysis revealed that 25.6% of the male participants and 50% of the female participants disagreed with this item. thus, it can be said that female prospective teachers believe that elementary students are mature enough to be interested in ssi more that male prospective teachers do. chi-square test results revealed significant differences only for three items in terms of academic achievement. the first item with significant difference was “successful students would be more interested in ssi in science lessons.” [x²(2) = 8.93, p =.01]. the analysis proved that 41.8% of the prospective teachers who had 2.99 and lower average scores and 32.6% of the prospective teachers who had 3 and above average scores stated that successful students would be more interested in ssi in science lessons. this finding suggests that participants with 2.99 and below average scores believe more strongly that successful students would be more interested in ssi in science lessons. the second item with significant difference was “integrating ssi would increase the primary students’ interest in science lessons.” [x²(2) = 6.42, p = .04]. the analysis proved that 6.8% of the prospective teachers who had 2.99 and lower average scores and 23.5% of the prospective teachers who had 3 and above average scores were indecisive about statement that integrating ssi would increase the primary students’ interest in science lessons. this finding suggests that participants with 2.99 and below average scores are more decisive about the statement that integrating ssi would increase the primary students’ interest in science lessons. the last item with significant difference was “teachers can answer easily the student questions about, ssi.” [x²(2) = 6.37, p = .04]. the analysis proved that 51.5% of the prospective teachers who had 2.99 and lower average scores and 34.9% of the prospective teachers who had 3 and above average scores agreed that teachers can answer easily the student questions about, ssi. this finding suggests that compared to participants with 2.99 prospective elementary school teachers’ views about socioscientific issues / özden 345 and below average scores, those prospective teachers who had 3 and above average scores agreed more strongly that teachers can answer easily the student questions about, ssi. results, conclusions and recommendations it is of great importance for the student to learn to make decisions based on the information in ssi to achieve the goal of scientific literacy (sadler, 2004). it is the responsibility of the teachers to teach scientific literacy to students in a broader sense, and to teach how to think through ssi in a narrower sense. in this respect, the present study aimed to explore the views of prospective elementary school teachers' about ssi. the results obtained from this study are valuable in producing principles in terms of teaching ssi at elementary school. it was concluded in this study that prospective teachers described the ssi as current events which affect individuals, have no consensus on, include understanding the risks and probabilities, are structured in the form of open-ended dilemmas, necessitate moral and ethical choices to be made, have more than one alternative solutions, but having no definite solutions. these results have both similarities and differences with the findings of previous research in the literature. for example, the participants in ekborg et al. (2013)'s study also assessed ssi as a current and interesting context. however, unlike their research findings, present study found that participants mentioned that ssi have scientific basis, they require an understanding of the risks and possibilities, and they incorporate ethical dilemmas, though to a limited extent. ssi arise on the basis of developments in science and technology, but their solutions require not only thinking scientifically but also considering the ethical and moral values. therefore, when faced with any ssi, it is useful for the students or regular citizens to consider the ethical problem or problems inherent in the structure of the relevant socioscientific issue. this is because the active citizens of the future are expected to interpret the possible outcomes of the relevant ssi based on certain ethical and moral principles. in the qualitative strand of the study, only one participant mentioned about the ethical characteristics of the ssi. accordingly, it can be asserted that the participants are not aware of the moral and ethical values to be considered during the decision making process concerning the ssi. however, the opportunity to make choices in terms of ethical and moral issues concerning the ssi have been studied directly or indirectly in many research studies (barrett & nieswandt, 2010; fleming, 1986; sadler & zeidler, 2004; topcu et al. 2011). for example, sadler & zeidler (2004) examined how prospective teachers interpret ssi within the context of genetic engineering and found out that moral factors have important impact in decision-making processes regarding genetic engineering. topcu et al. (2011) found out that moral and ethical considerations were one of the components which affect the informal reasoning processes. fleming (1986) also concluded that moral issues are important in students’ decision-making processes. the literature reveals that students’ decision-making process concerning the ssi is a highly complex situation. students’ decision-making process cannot be explained by scientific knowledge alone. it should be noted that personal experiences, values, social and epistemological issues are also important beside scientific knowledge. it was also found that belief systems or religious properties, which are important agents of reasoning processes regarding ssi has not been mentioned at all. however, previous research findings suggest that individuals’ characteristics derived from their belief systems are effective on their way of thinking about their ssi (sadler & donnelly, 2006; topcu et al., 2011; zeidler et al, 2013). sadler and donnelly (2006) argue that rating ethical judgements regarding the ssi are affected from individuals’ religious point of view. in the relevant research, half of the participants stated that religious belief is an important international electronic journal of elementary education vol.7, issue 3, 333-354, 2015 346 factor. zeidler et al. (2013) posits that throughout the history belief system have always been effective in peoples’ discourses and reasoning about the ssi. this is because theological overtones do seem to be driven by the belief that humans are fulfilling a divine plan that implicitly removes one from the tacitly taking responsibility for a given decision. at this point arises the influence of beliefs in reasoning process about ssi. in the judgment process based on beliefs, individuals tend to merge religious beliefs with scientific data or explanations. people certainly are affected by the belief systems which are the product of culture and society. these systems affect individuals’ reasoning and decision-making processes while forming their judgements of what is right, wrong, good and evil. in this context, what matters is not to evaluate ssi with the characteristics of moral, ethical, and belief systems alone, but to do so considering scientific, economic and political components. the present study also found that socioscientific events are referred to as "scientific events". ssi are the dilemmas concerning economy, environment, politics, moral and ethical subjects, and bears in conflicts at least in one of these fields. as a matter of fact, while scientific developments emerge as the activities of scientists, ssi are the problems are outside the scope of the world of science and they have been debated for long and affecting the daily life of an ordinary individual. in this sense, it can be asserted that some of the participants’ fall into the misconception in defining ssi as "scientific events”. as specified by eastwood et al. (2012) for a problem to be named socioscientific, it is content must be based on scientific development, but it must also be meaningful socially. both qualitative and quantitative strands of the research revealed that participants believe ssi help primary school students gain higher order thinking skills. in this context, it was understood that participants believe science education involving ssi can have students gain such higher order thinking skills as argumentation, opinion development, scientific process skills and creativity. while this finding overlaps with some previous research findings in the literature (dawson & venville, 2013; dolan, nichols & zeidler, 2009; gresch et al., 2013; khishfe, 2014), it also contrasts with some others (foong & daniel, 2013). for example, dawson and venville (2013) found that using ssi improved the argumentation and informal reasoning skills of the students in the experimental group. similarly gresch et al. (2013) have also found that ssi have a positive impact on students’ decision-making skills. however, foong and daniel (2013) found that in their research that using ssi in certain instructional methods caused some progress in the argumentation skills of some students, but not on some others. it was determined in the present study that instruction based on ssi can improve the citizenship competencies of the students. there are similar findings in the literature. for example, lee et al. (2013) investigated the impact of instruction based on ssi on the development of favorable characters and values among students as global citizens. the research results indicated that students have developed sensitivity concerning the moral and ethical aspects of scientific and technological developments. the same study also revealed that students developed compassion for the students who are deprived of the benefits of advanced technologies or who suffered the adverse effects those technologies. also, it was understood that students promised to act more responsively in the future regarding the solution of ssi in the field of genetics. in another study, lee et al. (2006) found that education based on ssi help the students gain insights about the positive and negative aspects of science as citizens and develop a deep and unbiased understanding of science among students. it was found that while the using the ssi in science education, teachers “should not impose their views on students, guide students, lead the discussions, select challenging problems for the students, get prepared before the lessons, and ask intriguing and prospective elementary school teachers’ views about socioscientific issues / özden 347 thought-provoking questions”. in a research conducted by van rooy (1993) it was also reported that teachers should have similar roles. van rooy (1993) found that while using the ssi in their classes, teachers play the roles of helping, supporting, facilitating, impartiality, being devil's advocate, and counselling. zeidler and nichols (2009) argues that it is important to encourage students to think about alternative evidences. likewise, it is important for teachers to ask meaningful questions during class discussions and manage class discussions, thus they need to use the research and current information about the ssi discussed. ekborg et al. (2013) found that science teachers encouraged students to ask questions and answer to these questions, as well, arrange class debates, and perform web quests. foong and daniel (2013) indicated that teachers played the facilitator role instead of the traditional role of transferor or knowledge. throughout the study teachers refrained from affecting their students’ decisions, thus they neither supported nor rejected their decisions. the results obtained in the present study, as well as the previous research findings suggest that teachers should play the following roles regarding the use of ssi in general: firstly, after the teacher announces the socioscientific issue to be handled in the lesson, she should ensure that students are engaging in reading or inquiring about the relevant issue. in the second stage, teacher should check whether the students have understood the socioscientific issue and answer possible questions from the students. if students need and demand, teacher can give students some information in an objective manner. in this process, teacher’s objective attitudes is very important in order not to affect students’ assertions. after fulfilling these roles described, teacher should ask the students to express their viewpoints about the ssi justifying their assertions and supporting arguments. at the final stage, after listening to the explanations of each of the volunteering students, teacher should ask other students or the student who explained his/her view earlier to express opposite ideas which would disprove the initial views of their own or friends again with supportive ideas or arguments. on the condition that instruction is conducted in accordance with these steps, a teacher can improve the thinking skills of students and have the students discover scientific, political, personal, social, economic, religious, moral and ethical characteristics inherent in the ssi. it was found in the present study that prospective teachers believed that ssi would increase the interests of students in science classes. this result is in agreement with other research findings in the literature (anagun & ozden, 2010; ekborg et al., 2013; kara, 2012; lee et al., 2006; ottander & ekborg, 2012). the key for the students to understand science courses effectively and bear more responsibility in their lessons is their interest into the science. therefore current and dynamic topics like ssi can be used as an instrument to increase the students’ interest into the content of the course by making it easier for the students easier to establish a link between the real-life and the lessons like many studies in the relevant literature (anagun & ozden, 2010; ekborg et al., 2013; kara, 2012; lee et al., 2006), the present study showed, too, that prospective teachers have training needs regarding ssi. however, the dimensions of these training need are not known well. in general, competencies of teaching profession include learning and teaching process, monitoring and assessing student learning, school-family and social relationships, curriculum and content knowledge. it is important to determine in which field(s) the prospective elementary school teachers have training needs. on the other hand, there are research findings indicating that prospective teachers have positive perceptions of competency. for example kilinc et al. (2013) found that prospective science teachers found themselves efficient to teach ssi. researchers also detected that underlying reasons for the strong content knowledge of the prospective teachers include their undergraduate courses, informal environment, and participants’ personal interest in food international electronic journal of elementary education vol.7, issue 3, 333-354, 2015 348 technology. while content knowledge is undoubtedly important for effective teaching, it is not possible to acknowledge it as the only and most important condition due to some limitations. for an effective teaching one should have curriculum knowledge, competence in teaching methods and techniques, good command on the assumptions of development and learning psychology, as well as know how to measure and evaluate. however, the importance of the content knowledge cannot be denied. as a matter of fact, sadler & zeidler (2005b) also revealed that individuals with rich content knowledge face fewer problems during informal reasoning compared to those with poor content knowledge. it can be claimed that the most important component affecting the individuals’ perceptions of their competencies is content knowledge. the participants of the present research comprise prospective elementary school teachers. it is possible that since prospective elementary school teachers do not acquire in-depth knowledge about a particular discipline, lack of content knowledge may have a negative effect on their perceptions of competence. participants believe that elementary school students are mature enough to understand ssi. there are example researches in the literature proving that ssi can be used with younger age groups (dolan et al., 2009; pedretti, 1999; ritchie et al., 2011; rose & barton, 2012). for example, dolan et al. (2009) presented some sample activities in which ssi can be used with the fifth grade students and concluded that ssi improve learners’ scientific literacy. ritchie et al. (2011) found that in the science lessons where the ssi were used, students aged eleven showed significant improvement in terms of scientific content, with increased levels of interest and self-efficacy regarding the science lesson. in another study pedretti (1999) revealed the fifth and sixth grade students can improve their critical thinking and decision-making skills if faced with ssi. unlike the research findings above, some research (ekborg et al., 2013; lee et al., 2006; ozden, 2011) found that participants, though a few, consider the immaturity of the students as an obstacle for the use of ssi. for example ekborg et al. (2013) reported that some teachers believe that it is difficult for students aged 13-16 to work on ssi. according to the teachers, students from this age range have difficulty in focusing on specific questions and understanding the respective tasks. similarly, ozden (2011) reported that one participant of his research believed elementary school students would have difficulty in understanding the ssi. at this point, what matters is to decide how ssi can be used so as to contribute to the developmental features of the students at each class level, but not whether ssi can be used with certain age groups or not. teachers are responsible to design activities in which students will enjoy learning, discussing, and involving into the ssi, considering the characteristics of the age group. it was also found that participants in the present study believed that integrating ssi into science education would improve the scientific literacy of the students. there is evidence in the literature suggesting that using ssi in science education improve the learners’ scientific literacy. for example, ritchie et al. (2011) reported improved levels of scientific literacy on the part of learners who participated into scientific writing activities where ssi were used. it is important to use ssi in science lessons as an instrument to achieve the goal of scientific literacy. therefore, while the information and resources to be used by the teachers are important, what matters more is to provide prospective teachers with an understanding of how to teach scientific literacy using the ssi and to develop teaching skills through example practices. moreover, science curriculum should include the reflections of the features of ssi for scientific literacy. in the study, it was found that prospective teachers with low academic achievement believed successful students would be interested in the ssi more. this result is very important. as discussed earlier, learners should have adequate level of knowledge in order prospective elementary school teachers’ views about socioscientific issues / özden 349 to make reasoning against ssi. prospective teachers with low academic achievement might have remembered the problems they faced during the sessions where ssi were discussed, and reflected that rather successful students would attend the discussion about ssi. on the other hand, ssi does not address to a particular group of students. unlike the findings of the present study, lee et al. (2006) found that teachers believed that not only the successful students, but all students would benefit from the ssi. it was found that male and female participants’ views differed significantly only for one item. accordingly, female prospective teachers believed more strongly than the male ones that elementary students are mature enough to be interested in ssi. the absence of any significant differences for other items is in agreement with the research findings in kara (2012). present research has some limitations. first, the research is limited with the views and experiences of the prospective elementary school teachers studying at a university. thus, this limitation should be considered while making generalizations. also in the future, a qualitative research can be done in order to understand how (prospective) elementary school teachers integrate ssi in to their learning-teaching process; and a quantitative research can be done to determine (prospective) elementary school teachers’ senses of self-efficacy in teaching ssi. similarly, future researches can be done to explore prospective elementary school teachers’ epistemological patterns about ssi. • • • references alacam-aksit, a. c. (2011). sınıf öğretmeni adaylarının sosyobilimsel konularla ve bu konuların öğretimiyle ilgili görüşleri. (yüksek lisans tezi). https://tez.yok.gov.tr/ulusaltezmerkezi/searchtez adresinden alınmıştır. albe, v. (2008). students’ positions and considerations of scientific evidence about a controversial socioscientific issue. science & education, 17, 805-827. doi: 10.1007/s11191-007-9086-6. american association for the advancement of science. (2009). benchmarks for science literacy. retrieved from http://www.project2061.org/publications/bsl/online/index.php anagun, s. s. & ozden, m. (2010). teacher candidates’ perceptions regarding socio-scientific issues and their competencies in using socio-scientific issues in science and technology instruction. procedia social and behavioral sciences, 9, 981-985. ayres, l. (2008). thematic coding and analysis. in l. m. given (ed.), the sage encyclopedia of qualitative research methods (pp. 867-868). thousand oaks, ca: sage. barrett, s. e. & nieswandt, m. (2010). teaching about ethics through socioscientific issues in physics and chemistry: teacher candidates’ beliefs. journal of research in science teaching, 47(4), 380-401. barrue, c. & albe, v. (2013). citizenship education and socioscientific issues: implicit concept of citizenship in the curriculum, views of french middle school teachers. science & education, 22, 1089-1114. doi: 10.1007/s11191-012-9571-4 braun, v. & clarke, v. (2006). using thematic analysis in psychology. qualitative research in psychology, 3(2), 77-101. doi:10.1191/1478088706qp063oa buyukozturk, s. (2005). sosyal bilimler için veri analizi el kitabı. (5. baskı). ankara: pegem yayıncılık. byrman, a. (2006). mixed methods: a four-volume set. thousands oaks, ca: sage. creswell, j. w. & plano clark, v. l. (2011). designing and conducting mixed methods research. (2nd edition). thousands oaks, ca: sage. international electronic journal of elementary education vol.7, issue 3, 333-354, 2015 350 creswell, j. w. (2014). research design: qualitative, quantitative, and mixed methods approaches. (4th edition). thousand oaks: sage. creswell, j. w. (2015). a concise introduction to mixed methods research. thousand oaks: sage. dawson, v & venville, g. (2013). introducing high school biology students to argumentation about socioscientific issues. canadian journal of science, mathematics and technology education, 13(4), 356-372. doi: 10.1080/14926156.2013.845322 dawson, v. (2001). addressing controversial issues in secondary school science. australian science teachers journal, 47(4), 38-44. day, s. p. & bryce, t. g. k. (2011). does the discussion of socio‐scientific issues require a paradigm shift in science teachers' thinking? international journal of science education, 33(12), 16751702. dolan, t. j., nichols, b. h., & zeidler, d. l. (2009). using socioscientific issues in primary classrooms. journal of elementary science education, 21(3), 1-12. eastwood, j. l., sadler, t. d., zeidler, d. l., lewis, a., amiri, l., & applebaum, s. (2012). contextualizing nature of science instruction in socioscientific issues. international journal of science education, 34(15), 2289-2315. doi: 10.1080/09500693.2012.667582 ekborg, m. , ottander, c., silfver, e. & simon, s. (2013). teachers’ experience of working with socioscientific issues: a large scale and in depth study. research in science education, 43(2), 599617. doi: 10.1007/s11165-011-9279-5 erkus, a. (2011). davranış bilimleri için bilimsel araştırma süreci. (3. baskı). ankara: seçkin yayıncılık. firmin, m. w. (2008). themes. in l. m. given (ed.), the sage encyclopedia of qualitative research methods (pp. 868-869). thousand oaks, ca: sage. fleming. r. (1986). adolescent reasoning in socio-scientific issues, part i: social cognition. journal of research in science teaching, 23(8), 677-687. foong, c.-c., & daniel, e. g. s. (2013). students’ argumentation skills across two socio-scientific issues in a confucian classroom: is transfer possible? international journal of science education, 35(14), 2331-2355. doi: 10.1080/09500693.2012.697209 gibson, w. j. & brown, a. (2009). working with qualitative data. london, thousand oaks: sage. gresch, h., hasselhorn, m. & bögeholz, s. (2013). training in decision-making strategies: an approach to enhance students’ competence to deal with socio-scientific issues. international journal of science education, 35(15), 2587-2607. doi: 10.1080/09500693.2011.617789 ideland, m., malmberg, c., & winberg, m. (2011). culturally equipped for socio‐scientific issues? a comparative study on how teachers and students in mono‐ and multiethnic schools handle work with complex issues. international journal of science education, 33(13), 1835-1859. doi: 10.1080/09500693.2010.519803 jones a., buntting, c., hipkins, r., mckim, a., conner, l. & saunders. k. (2011). developing students’ futures thinking in science education. research in science education, 42(4), 687-708. kara, y. (2012). pre-service biology teachers’ perceptions on the instruction of socio-scientific issues in the curriculum. european journal of teacher education, 35(1), 111-129. doi: 10.1080/02619768.2011.633999 khishfe, r. (2012). relationship between nature of science understandings and argumentation skills: a role for counterargument and contextual factors. journal of research in science teaching, 49(4), 489-514. khishfe, r. (2014). explicit nature of science and argumentation instruction in the context of socioscientific issues: an effect on student learning and transfer. international journal of science education, 36(6), 974-1016. doi: 10.1080/09500693.2013.832004 prospective elementary school teachers’ views about socioscientific issues / özden 351 kilinc, a., kartal, t., eroglu, b., demiral, u., afacan, o., polat, d., demirci guler, m. p. & gorgulu, o. (2013). preservice science teachers’ efficacy regarding a socioscientific issue: a belief system approach. research in science education, 43(6), 2455-2475. doi: 10.1007/s11165-013-9368-8 kolstø, s. d. (2001). scientific literacy for citizenship: tools for dealing with the science dimension of controversial ssi. science education, 85(3), 291-310. doi: 10.1002/sce.1011 kolstø, s. d., bungum, b., arnesen, e., isnes, a., kristensen, t., mathiassen, k. & et al. (2006). science students’ critical examination of scientific information related to socioscientific issues. science education, 90(4), 632-655. lee, h., abd‐el‐khalick, f. & choi, k. (2006) korean science teachers’ perceptions of the introduction of socio‐scientific issues into the science curriculum. canadian journal of science, mathematics and technology education, 6(2), 97-117. doi: 10.1080/14926150609556691 lee, h., yoo, j., choi, k., kim, s.-w., krajcik, j., herman, b. c., & zeidler, d. l. (2013). socioscientific issues as a vehicle for promoting character and values for global citizens. international journal of science education, 35(12), 2079-2113. doi: 10.1080/09500693.2012.749546 levinson, r. (2006). towards a theoretical framework for teaching controversial socio-scientific issues. international journal of science education, 28(10), 1201-1224. ministry of national education [milli egitim bakanlıgı] (2013). i̇lköğretim kurumları (ilkokullar ve ortaokullar) fen bilimleri dersi (3, 4, 5, 6, 7 ve 8. sınıflar) öğretim programı. ankara: talim terbiye kurulu başkanlığı. morris, h. (2014). socioscientific issues and multidisciplinarity in school science textbooks. international journal of science education, 36(7), 1137-1158. doi: 10.1080/09500693.2013.848493 morse, j. m. (1991). approaches to qualitative-quantitative methodological triangulation. nursing research, 40(1), 120-123. national research council (1996). national science education standards. washington, dc: national academies press. ottander, c. & ekborg, m. (2012). students’ experience of working with socioscientific issues: a quantitative study in secondary school. research in science education, 42, 1147-1163. doi: 10.1007/s11165-011-9238-1 ozden, m. (2011). 4. ve 5. sınıflar fen ve teknoloji dersinin vatandaşlık eğitimi bakımından işlevselliği. (yayınlanmamış doktora tezi). anadolu üniversitesi eğitim bilimleri enstitüsü, eskişehir. patton, m. q. (2001). qualitative research and evaluation methods. (3rd ed.). thousand oaks: sage. pedretti, e. (1999). decision making and sts education: exploring scientific knowledge and social responsibility in schools and science centers through an issues-based approach. school science and mathematics, 99(4), 174-181. doi: 10.1111/j.1949-8594.1999.tb17471.x pedretti, e. (2003). teaching science, technology, society and environment education: preservice teachers’ philosophical and pedagogical landscapes. in d. l. zeidler (ed.), the role of moral reasoning on socioscientific issues and discourse in science education (p.219-239). london: kluwer academic publishers. ritchie, s. m., tomas, l. & tones, m. (2011). writing stories to enhance scientific literacy. international journal of science education, 33(5), 685-707. doi: 10.1080/09500691003728039 rose, s. l. & barton, a. c. (2012). should great lakes city build a new power plant? how youth navigate socioscientific issues. journal of research in science teaching, 49(5), 541-567. doi: 10.1002/tea.21017 rudsberg, k., öhman, j. & östman, l. (2013). analyzing students’ learning in classroom discussions about socioscientific issues. science education, 97(4), 594-620. doi: 10.1002/sce.21065 international electronic journal of elementary education vol.7, issue 3, 333-354, 2015 352 sadler, t. d. & donnelly, l. a. (2006). socioscientific argumentation: the effects of content knowledge and morality. international journal of science education, 28(12), 1463-1488. doi: 10.1080/09500690600708717 sadler, t. d. (2004). informal reasoning regarding socioscientific issues: a critical review of research. journal of research in science teaching, 41(5), 513-536. doi: 10.1002/tea.20009 sadler, t. d. (2009). situated learning in science education: socio-scientific issues as contexts for practice. studies in science education, 45(1), 1-42. doi: 10.1080/03057260802681839 sadler, t. d., & zeidler, d. l. (2004). the morality of socioscientific issues: construal and resolution of genetic engineering dilemmas. science education, 88(1), 4-27. doi: 10.1002/sce.10101 sadler, t. d., & zeidler, d. l. (2005a). patterns of informal reasoning in the context of socioscientific decision making. journal of research in science teaching, 42(1), 112-138. doi: 10.1002/tea.20042 sadler, t. d., & zeidler, d. l. (2005b). the significance of content knowledge for informal reasoning regarding socioscientific issues: applying genetics knowledge to genetic engineering issues. science education, 89(1), 71-93. doi: 10.1002/sce.20023 sadler, t. d., chambers, f. w., & zeidler, d. l. (2004). student conceptualizations of the nature of science in response to a socioscientific issue. international journal of science education, 26(4), 387-409. doi:10.1080/0950069032000119456 saldaña, j. (2011). fundamentals of qualitative research: understanding qualitative research. new york: oxford university press. saunders, k. j. & rennie, l. j. (2013). a pedagogical model for ethical inquiry into socioscientific issues in science. research in science education, 43(1), 253–274. doi: 10.1007/s11165-0119248-z seidman, i. (2006). interviewing as qualitative research: a guide for researchers in education and the social sciences. (3rd ed.). new york: teachers college press. spradley, j. p. (1979). the ethnographic interview. florida: holt, rinehart and winston inc. tabachnick, b. g., & fidell, l. s. (2000). using multivariate statistics. new york: allyn & bacon. tashakkori, a., & teddlie, c. (eds.). (2010). sage handbook of mixed methods in social and behavioral research. (2nd ed.). thousand oaks, ca: sage. ten have, p. (2004). understanding qualitative research and ethnomethodology. london: sage publications. tomas, l., ritchie, s. m. & tones. m. (2011). attitudinal impact of hybridized writing about a socioscientific issue. journal of research in science teaching, 48(8), 878-900. doi: 10.1002/tea.20431 topcu, m. s. (2010). development of attitudes towards socioscientific issues scale for undergraduate students. evaluation & research in education, 23(1), 51-67. doi: 10.1080/09500791003628187 topcu, m. s., mugaloglu, e. z., & guven, d. (2014). fen eğitiminde sosyobilimsel konular: türkiye örneği. kuram ve uygulamada eğitim bilimleri, 14(6), 1-22. topcu, m. s., sadler, t. d. & yilmaz‐tuzun, o. (2010). preservice science teachers’ informal reasoning about socioscientific issues: the influence of issue context. international journal of science education, 32(18), 2475-2495. doi: 10.1080/09500690903524779 topcu, m. s., yilmaz‐tuzun, o., & sadler, t. d. (2011). turkish preservice science teachers’ informal reasoning regarding socioscientific issues and the factors influencing their informal reasoning. journal of science teacher education, 22(4), 313-332. doi: 10.1007/s10972-010-9221-0 prospective elementary school teachers’ views about socioscientific issues / özden 353 vaismoradi, m., turunen, h., & bondas, t. (2013). content analysis and thematic analysis: implications for conducting a qualitative descriptive study. nursing and health sciences, 15(3), 398–405. doi: 10.1111/nhs.12048 van rooy, w. (1993). teaching controversial issues in the secondary science classroom. research in science education, 23, 317-326. yildirim, a., & simsek, h. (2013). sosyal bilimlerde nitel araştırma yöntemleri. (9. baskı). ankara: seçkin yayıncılık. zeidler, d. l. & nichols, b. h. (2009). socioscientific issues: theory and practice. journal of elementary science education, 21(2), 49-58. doi: 10.1007/bf03173684 zeidler, d. l., sadler, t. d., applebaum, s., & callahan, b. e. (2009). advancing reflective judgment through socioscientific issues. journal of research in science teaching, 46(1), 74-101. doi: 10.1002/tea.20281 zeidler, d., herman, b. c., ruzek, m., linder, a. & lin, s.s. (2013). cross-cultural epistemological orientations to socioscientific issues. journal of research in science teaching, 50(3), 251-283. doi: 10.1002/tea.21077 international electronic journal of elementary education vol.7, issue 3, 333-354, 2015 354 www.iejee.com this page is intentionally left blank http://www.iejee.com/ the concurrent development of spelling skills in two languages international electronic journal of elementary education vol. 3, issue 2, march, 2011. issn:1307-9298 copyright © iejee www.iejee.com experiencing technology integration in education: children‟s perceptions ahmet baytak* harran university, turkey bülent tarman selçuk university, turkey cemalettin ayas sinop university, turkey abstract the purpose of this phenomenological study was to explore the experiences of six children using technologies in their education. data were collected via in-depth interviews, classroom observations, and home observations. the results showed that students have common perceptions toward their experience with technology integration. furthermore, the following four themes emerged; the value of technology, authority over learning, misuses and misconceptions, and the border of integration. keywords: children‟s technology use, technology integration, technology integration at home, perceptions of children, learning and technology introduction students in today‟s schools are lucky enough to have access to many technology equipments and the internet technologies. almost every house has a computer available to children. according to the 2003 us census 69.9% of households had computer at home and 61.8% of them had the internet access. for example, based on an unofficial survey done in research site school, 98% of the middle school students had computer at home and almost all of them had access to the internet. the less developed countries have been also receiving aids to improve usage of technology in their schools. unesco and ngos (non-governmental organizations) are providing funds to such less developed countries in order to provide more technology equipments in their * correspondence: ahmet baytak, ph.d., harran university, department of computer engineering, osmanbey campus, p.k. 153, 63000-şanlıurfa / turkey. e-mail: abaytak@harran.edu.tr phone: +90 (0) 414-318-3000 (ext. 1088) http://www.iejee.com/ mailto:abaytak@harran.edu.tr international electronic journal of elementary education vol.3, issue 2, march, 2011 140 schools. one laptop per child (olpc, 2008), for instance, is one of the projects that tries to provide portable computers to schoolchildren in these less developed countries. as a result of this development, a „digital divide‟ or disparity tends to exist between generations. prensky‟s classification; digital natives and digital immigrants (2001), is a useful distinction in most communities. while digital natives have been born with new technologies, digital immigrants are still considering these technologies as luxury, extra, difficult, or troublemaker. this difference in generation, thus, brings new issues for use of technology in classroom (tarman, 2009). technology is becoming more and more a part of classroom instruction and teachers are encouraged to use technology for their lessons (ayas, 2006; beers et al., 2000; yücel et al., 2010). technology in education has the potential for improving teaching and learning. if the current technology is appropriately designed for instruction, earle (2002) believes, there is the potential to produce positive outcomes, social interactions, changes in teaching styles, more effective teaching, increased student motivation, and enhanced student learning. speaker (2004) reports that most students feel their learning are improved by integrating technology into their learning. therefore, educational technologies, specifically computer and the internet technologies, have inevitably become powerful in the classroom as they change the way we teach and learn (ayas, 2006). as technology makes learning more interesting, enjoyable and interactive, kids today love learning by doing, discovering, and interacting. review of literature while most of the technology integration research focuses on integration in classrooms, some scholars have specifically examined children‟s use of technology at home. mumtaz (2001) found that children spend more time with technologies at home than at school. however, lauman‟s study (2000) showed that students felt more comfortable using computers at school. kafai and sutton (1999) found that children‟s use of computers at home depends on permission from parents who have concerns about their children wasting time on the internet and not doing educational activities (mumtaz, 2001). however, it was also found that parents‟ support on the use of technology affects the level of integration at home (giacquinta et al., 1993). they also found that few children who integrate technology for learning had highly involved parents who helped choose appropriate software, coached their child on the computers, worked jointly with the child at the keyboard, and offered praise as well as practical. even though most studies reviewed mainly focused on technology integration at school and home, students‟ experiences with technology at school and at home have been rarely investigated. the history of the last decade is also evidence that technological tools are changing dramatically and experiencing technology integration in education / baytak, tarman & ayas 141 therefore technology integration in classroom essentially changes as well (yücel et al., 2010). student perception is an area in which a great deal of research has been conducted. for example, understanding their perceptions of parent involvement, professors‟ self-presentation styles, and discussion-driven classrooms has been studied in different studies. research on students‟ perception of technology in education has been sparse and mostly limited to technology in e-learning or college students‟ perceptions. among those studies, lim et al. (2006) examined students‟ perceptions on computer vs. pen based testing, mcmahn et al. (1999) studied college students‟ perceptions about barriers with computers, el-tigi, lewis, and macentee (1997) explored elementary school students‟ perception on the effectiveness of visuals on webbased instructions, and shell et al. (2005) examined high school students perception on computer supported classrooms. the study by levin and barry (1997) also showed that young students found computers as a game machine both at home and at school. according to the study done by i̇şman et al. (2004), students in undergraduate and graduate school perceived computers as a part of their life. these students also had a positive attitude towards computers since they think they are efficient tools for their life. thus, the researchers concluded that the students had a consciousness about effects and importance of computers. lui and his colleagues (2006) concluded from their students‟ perception on blogs that integration of blogs in the lessons could promote educational perception even though there are still some misuses of these technologies. according to student perception model by o‟malley and mccraw (1999), the perceived effectiveness of a technology is based three factors; the prior educational conditions, characteristics of students, and perceived characteristics of technology. differently, some scholars explored and examined children‟s views and preferences about technology materials (druin, 1999; druin, 2002; nesset & large, 2004). druin proposed cooperative inquiry and human-computer interaction community to examine technology tools that are proper for children. in these studies, children were involved in design and testing processes to find out their preferences. the user-design approach by nesset and large (2004) also looked at children‟s use of technological tools to design proper tools for their levels. however, in this approach involvement was found limited. even though these approaches are crucial to understand children‟s views about technology and their use, children‟s views about integration of technology into education is not studied. additionally, studies done in constructionism and design-based research have involved children to explore their learning with technology tools (harel & papert, 1991; kafai, 2005) but these studies are lack of children‟s perceptions about the characteristics of technologies they used and how those could improve their learning. international electronic journal of elementary education vol.3, issue 2, march, 2011 142 the purpose of the study the main goal of this study is to explore how children define and use technology in their education. in other words, this study attempts to observe the experiences of individuals in order to understand their perceptions of technology integration into their education. this study aims to begin fill the gap in several ways. first, the last decade is evidence that technologies are changing dramatically and therefore technology integration in the classroom must necessarily change as well. thus, it is important to get a sense of how students feel about recent technologies and the integration of them into learning lives as a whole. second, past studies primarily focused on upper level students in middle and high schools, thus the concentration here on elementary level students is an important contribution. third, the increased usage of technology in schools indicates a need for studies such as ours. finally, there is also an increased usage of technologies at home, which is rarely studied in relationship to technology integration into learning. moreover, children‟s future technology perception and imagination make this study unique. in another words, what kinds of new tools or programs students perceive for future and how these new developments can be used for learning also raise the importance of this study. by looking at students‟ perception for future technological developments may help technology designers to build more appropriate technological tools for students to use for education. research context and methodology the research site was an elementary and middle school located in a college town in the northeast of the united states. students attending this school are generally from the middle class whose parents are mostly affiliated with a well-known state university. according to mission of the school, technology is one of the key aspects of the curriculum. the school offers technological equipments for classes and after-school technology clubs. the participants of this study were selected from these technology clubs. the selection of the participants was based on their parents‟ consents. in technology education classes during the regular school hours, the students were taught about word processing, excel, and powerpoint. in the technology clubs, however, the students designed games and animations with the provided software. since the study was limited to students in the technology club, the interested students for the study were already good at technology use. for example, out of six study participants, three of them (john, geff, and allan) attended statewide conference workshop to display their animation designs. this phenomenological study attempts to understand and attain a description from the students regarding the perception of individuals and lived experience of individuals about this phenomenon. the discipline experiencing technology integration in education / baytak, tarman & ayas 143 investigates the why and how of decision making, not just what, where, when. creswell (1998) also defines qualitative research as “an inquiry process of understanding based on distinct methodological traditions of inquiry that explore a social or human problem. the researcher builds a complex, holistic picture, analyzes words, reports detailed views of informants, and conducts the study in a natural setting” (p. 15). in this study, we questioned the experiences of the students with technology integration. we did not expect any change in students‟ outcome or achievement. the study is neither has any hypotheses to prove. the current study is expanding the research in the participants‟ lives where they experience technology integration yet it is not based on „standards‟ or „certain objectives‟ as marshall & rossman (1980) puts it. furthermore, the study is concerned with the process and the meaning of technology integration for the students. thus, qualitative approach fits perfectly to apply in this research. this study aims to investigate not the external truths but their interpretations of emotions and events within the definition of phenomenology. data collection and analysis the data of this study are students‟ thoughts, ideas and perceptions from digitally recorded interviews, observations in their natural environment, and field notes. interviews are centered on getting in depth information of lived experience with the phenomena. for a broader perspective, there are two types of observations in this study; classroom observations and home observations. these both observations aimed to get more in depth understanding of phenomenon by recording non-verbal behaviors and physical settings. classroom observation was done before and after the interviews. the first observations were helpful to generate some interview questions. since the researcher had been working with the students before, students were familiar to the researcher‟s class visits during their technology usage hours. therefore, it was believed that observations did not influence students‟ behaviors. class observations were done by note taking while home observations included recordings in addition to note taking. different from previous studies, students‟ technology use and technology settings were observed at home as well. home observations were done after getting detail information from the students during the interviews. these observations were limited to 30 minutes and students were also asked some questions to get more information about the technology integration at home. in depth interviews was necessary for this phenomenological study to get more information about the experience of the students with technology integration. since the younger students were not that much talkative, we could not follow every step of irving seidman‟s (1998) interview protocol. for example, the interviews were less than 30 minutes each since the students international electronic journal of elementary education vol.3, issue 2, march, 2011 144 didn‟t have anything to say. however, we asked prompt questions based on students statements. that helped us gather clarification and amplification in their thoughts and ideas. interviews were also recorded with digital voice recorder and there were note taking for prompt questions and outline of the data. after the interviews, the recordings were transcribed with minutes. an ethical issue that may come up in this study is about the researcher‟s position at the research place. the students and their parents were informed that there was no grading for students‟ progress for participating into the study or leaving the study in the middle. there was also no intervention in this study to affect students‟ behavior or performance at school. findings background information according to mission of the school, technology is accepted as one of the key aspects of the curriculum and the school promises to provide cutting edge technology in its unique educational program. each classroom is equipped with lcd tvs connected to cable tv service, projectors, internal sound systems, classroom laptops, and cameras available for teacher and student use. the laptopstudent ratio was 4:7 and laptops were found more flexible for students to take the classrooms and integrate into any subject area. the participants of this study were six boys (john, geff, tony, allan, joe, and brianall names presented are pseudonyms) at fifth and sixth grades. joe was the one of the best in his sixth grade class for academic achievement. john‟s both parents are teachers and he speaks two languages. he was always interested in topics about computers. he had his own computer at home. geff could be the quietest students in his classroom but he was always doing his homework and class work on time. his both parents are professors in different majors. according to our home observation there are 4 computers at his house and most of them are available to him. even though tony moved to the country in last few years, he didn‟t have any problem with language. however, he still needed to improve his selfconfidence that was also showed up in the interviews and class observations. tony was also interested in computers and he attended technology club last four semesters. he was sharing a computer with his siblings. allan also attended all technology clubs sessions in last two years. his classmates called allan computer geek. he had already used several computer programs with his own computer at home. his parents are involved with university. joe and brian were also attended all technology club sessions but both were less interested in computers games comparing with other four students mentioned above. joe also had his own computer at home and was able to fix most of the problems with his computer. brian was sharing one computer with his siblings and limited time to access this computer during weekdays because of his parents‟ views. experiencing technology integration in education / baytak, tarman & ayas 145 children’s perceptions based on the data analysis, four themes emerged; the value of technology, authority over learning, misuses and misconceptions, and the border of integration. the value of technology: almost all the students had similar perceptions when defining technology. all of them believed that anything that works with electricity is technological. allan, however, added “controllable tools” to his definition as he thinks these tools must be helpful also in order to consider them as technological. john thinks that technological should “entertain.” when they were asked for examples, they started with computers and game boys. on the other hand, none of the student has ever heard of the term “technology integration.” but, they were aware of the influence of technology in their learning. when the students were asked about their first experience with technology, most of them recalled their first game boys and what they learned from these tools. all the students think that their first experience with technology was fun and now they still feel fun when they use for even educational purposes. for example, john mentioned that “… [for] example like writing essays writing, instead of your hand for writing you can type and i think typing is fun and less tired. doesn‟t tire you that much.” authority over learning: class observations and interviews transcripts are evidence that students feel an authority over their learning in classroom with computers. it was observed that students‟ behaviors in technology based classes, comparing in their other classes, altered from followers to semifollower. in other words, students were acting as they were fully dependent to teachers but in classes with computers they were more independent. similarly, students perceived that they cognitively feel ore powerful when they use computers in their lessons. for instances, as other five students, john mentioned in the interview that: “i think technology make me feel smarter because of all these lots of parts put together so how can i explain well (…example?) like going on internet seeing like math reading all these kinds of stuff, finding out new links to easy ways, they make so much easier and make smarter. i do better that way.” misuses and misperceptions: since integration of technology especially with computer technology is new topic in most school communities, there are still some misuses ands misperceptions by teachers and parents who have has hold the main power for the decision on integration. even though the students‟ technology experience at school considered as integration, the use at home is not common. similar to a previous study (kafai & sutton, 1999), this study also found that the parents‟ concerns, limitations, and less experience with integration becomes a barrier for the international electronic journal of elementary education vol.3, issue 2, march, 2011 146 integration at home. the home observation and interview also projects parents concerns since the students used computer at home mainly for gaming and chatting. however, brian pointed out “i just try to use that time (his has 30 minutes every day to use for anything) to do my homework, not after i do my homework i can others (he listed others as games etc.).” it was observed that only few teachers use technology tools in their classrooms and therefore students had less integration experiences in other classes. the students believe that technology can only be integrated in certain subject areas. for examples, tony preferred use technology in math class but not either in science or physical education classes. similar to that brian also think that music should not be taught with computers. the border of integration: when the students were asked about advantages and disadvantages of using technology for learning, they built a border of technology in education. for example, most of the students listed “searching on the internet” as one of the main benefits of technology for education. similar to other findings (saye, 1997) in the literature, students in the current study apparently valued the efficiency, speed, and clarity that technology provides for education. allan specifically focused on how that internet based communication could help economy based on his mother‟s experience. he, indeed, thinks that animated content on computers also get attention and therefore implementing computers for learning will be not as boring as reading book. nevertheless, some students think that use of technology in learning should be limited. for example, brian and geff prefer to have a person teach them instead of learning with computers since machines may not give them instant feedback. allan thinks that “it is funny to use term ‘educational’ for the cartoons on tvs since they are not.” all the students have fear that the computers may get broken and they lost their files. this fear was experienced during the researcher‟s class observation that some younger students delete a student‟s file for his social studies work. it can be driven from the interview and observation that students think that technical problems and viruses, less feedback functionality, and physical damage on eyes are the common barriers to integrate technology into education. because of those listed benefits and barriers, students have drawn an imaginary border of technology in education. it was found that students, such as allan, with more experiences in technology use, had a wider border when they described advantages and disadvantages of technology in education. based on the students‟ radius of the border they had drawn, their views of technology integration for future were shaped. when we asked them what kind of technologies would be in classrooms in next ten years, john expected that there would be holograms, better quality microscopes, and machines that type for users. besides his dreams of teleports in future, he also noted that “… we can use to study like other recourses such as machines that will pick up recourses and study it and give a description of it or even maybe our experiencing technology integration in education / baytak, tarman & ayas 147 own kind of microchip. we can and explore with to more field trips to places. learn more about fossils in a technological way.” another student, geff, was wishing a common problem in technology integration to be solved without being aware of some programs; “…maybe if you loose a file you could get back.” discussions and conclusion the aim of this study was to explore the experiences of six young students using technologies in their classroom. as a discussion topic of this paper, there are some conclusion could be drawn from the findings of this current phenomenological study. first of all, it was found interesting that when the students were asked to define the term „technology‟, most of them listed the features of technology that has value of fun and entertainment. in other words, the educational value that the children gave to technology was more about the motivational factors. another value that the students listed for technology was the feature of a tool that makes things easier. especially, when the students mentioned about communication tools as technology, they emphasized that these tools make their life easier and therefore the process of learning becomes effortless. secondly, whether in student-centered or teacher-centered classrooms, students in this study were acted more independent when they were observed in their classroom with computers. even though the students have ownership of learning and they have more authority over their learning, at these age levels, teachers are still the authority that believed to know everything. the students, for example, think certain website trustworthy because the teacher said so. in other words, students are aware of fact that they need scaffolding in their learning process where the teachers could act as milestone when they needed. thirdly, as it has been indicated in the previous studies (kafai, 2005; lauman, 2000; mumtaz, 2001) that children like to use home computers for gaming purposes came up in this study as well. the high percentage of computer use for gaming (77% of children regularly used computers for gaming), has a factors on parents misconception about the use of computers at home. it was found in this study that most of the parents think that their children were not doing anything educational on the computers. a parallel misconception was found among the students‟ teachers that technology is tool to transfer information, and therefore, they think that home computers are still not under their control to give educational task for students to do. similar to previous studies (kafai & sutton, 1999), this study also found that parent concerns and limited experience with the use of technology for learning could be a barrier for integration at home. for example, brian pointed out, “i just try to use that time [he has 30 minutes every day to use for anything] to do my homework, after i do my homework i can do others [he listed others as computer games etc].” thus, there is a need for schools and international electronic journal of elementary education vol.3, issue 2, march, 2011 148 teachers to rearrange the types of homework, which may require more technology use such as doing more research, designing digital artifacts, or building their own portfolios. in addition, it is necessary to setup more communication channels between teachers and parents to increase effectiveness of home technologies for educational purposes. course management systems are available options to start this communication. finally, this study revealed that students draw the border for the integration of technology into education. even though all the children of this study see technological tools as fun and entertainment channel, they were mostly conscious about the balance of technology integration into their lessons. the children were able to list the advantages and disadvantages of this integration. however, it was found that, the students‟ less experiences of the integration in both classroom and at home had influenced their perceptions. supporting to this idea, i̇şman et al. (2004) pointed out in their study that “this means that there is a consciousness about effects and importance of computers but there are a few tendencies to apply the consciousness or willingness of new technological style because of not having particular education, encouragement and facilitative environment” (p. 20). in addition, it was discussed in the previous studies that teachers and k-12 schools and faculties in higher education complained about technical problems and lack of support (mcmahon et al., 1999). however students in this study mentioned those as teachers‟ problems. the reason for that could be because they do not see the technical problems as their responsibility or they found their ways overcome to problem. for example, based on classroom observation and interviews, students try to solve technical problems by themselves. it could also be concluded that the more implementation, the wider the border of integration could be. in conclusion, this research disclosed the reality that changes in technology influence students‟ experience with technology. thus, this study should be helpful for the curriculum and technology designers, and educators to consider these perceptions of the students in the future educational plans and policies. our participants‟ experiences with technology integration also support smith‟s findings that some faculty may not be well prepared or trained for the available technology and which creates distance between students and teachers. parallel to that, a participant of this current study, allan, also made a recommendation that “more people should use [technology in classrooms] but they have to have backup plans if there might be virus etc.” thus, as an implementation of this study, schools may setup their own course management systems to enrich students‟ learning both at school and at home. for more encouragement of technology integration at home, teachers need to provide more educational games that they should be able to control the content of the games, which becomes both educational and fun for children. based on the previous studies about teachers‟ perceptions and the experiencing technology integration in education / baytak, tarman & ayas 149 results of this study, it is also important to note that school administrations and teachers should develop new ways to integrate technology into education for an effective learning environment. it is noteworthy that children of the internet generation enjoy communicating through online and sharing the things they liked. thus, ageappropriate chat and discussion platforms and information and artifact sharing sites are necessary for these students to productively use technology both at school and at home. at the same time students could be required to build their learning portfolios in secure sites manageable by school administrations and accessible to their parents. • • • received: 23 november 2010 / revised: 5 february 2011 / accepted: 10 february 2011 ahmet baytak is an assistant professor in the department of computer engineering at harran university, turkey. he received his ph.d. degree in educational technology from the pennsylvania state university. his major research interests include instructional technologies and the use of computer and the internet applications in educational settings. bülent tarman is an assistant professor in the department of social studies education and serves as the vice chair for the institute of educational sciences at selcuk university, turkey. he has a ph.d. degree in social studies & comparative education from the penn state university. his research and teaching focused on teacher preparation in social studies, democracy education, and european union & educational systems. cemalettin ayas is an assistant professor of social studies education at sinop university, turkey. he received his ph.d. degree in social studies & global education from the ohio state university. his current research and teaching mainly focus on social studies & geography education, teacher preparation, and educational technology. references ayas, c. (2006). an examination of the relationship between the integration of technology into social studies and constructivist pedagogies. the turkish online journal of educational technology, 5(1), 14-25. beers, m., paquette, k., & warren, j. (2000). student view of classroom technology use. society for information technology and teacher education international conference: proceedings of site 2000. creswell, j. w. (1998). qualitative inquiry and research design: choosing among five traditions. thousand oaks, ca: sage. druin, a. (1999). cooperative inquiry: developing new technologies for children with children. proceedings of chi‟99, acm press. international electronic journal of elementary education vol.3, issue 2, march, 2011 150 druin, a. (2002). the role of children in the design of new technology. behavior and information technology, 21(1), 1-25. earle, r. (2002). the integration of instructional technology in to public education: promises and challenges. retrieved december 10, 2008 from http://www.asianvu.com/bookstoread/etp/earle.pdf. el-tigi, m. a., lewis, b. a., & mac entee, v. m. (1997). perception of elementary students of visuals on the web. in: visionquest: journeys toward visual literacy. selected readings from the 28th annual conference of the international visual literacy association (cheyenne, wyoming, october, 1996). giacquinta, j, b., bauer j, a, & levin, j, e, (1993). beyond technology's promise: an examination of children's educational computing in the home. cambridge, u.k.: cambridge university press. harel, i. & papert, s. (1991). software design as learning environment. in i. harel s. papert. (eds.). children designers: interdisciplinary constructions for learning and knowing mathematics in a computer-rich school (pp. 41-85). norwood, nj: ablex. i̇şman, a, çağlar, m., dabaj, f., altınay, z., & altınay, f., (2004). attitudes of students toward computers. the turkish online journal of educational technology, 3(1), 11-21. kafai, y.b (2005). the classroom as “living laboratory”: design-based research for understanding, comparing, and evaluating learning science through design. educational technology, 45(1), 28-34. kafai, y. & sutton, s. (1999). elementary school students‟ computer and internet use at home: current trends and issues. journal of educational computing and research 21(3), 345-362. lauman, d. j. (2000), student home computer use: a review of the literature. journal of research on computing in education, 33(2), 196-204. levin, b. b., & barry, m. s. (1997). children‟s views of technology: the role of age gender and school setting. journal of computing in childhood education, 8(4), 267-290. lim, e., ong, b., wilder-smith, e., seet, r., (2006). computer-based versus pen-and-paper testing: students‟ perception, annals of the academy of medicine singapore, 35(9), 599603. lui, a. k., choy, s.-o., cheung, y. h. y., & li, s. c. (2006). a study on the perception of students towards educational weblogs. informatics in education, 5(2), 233-254. marshall, c., & rossman, g.b. (1989). designing qualitative research. newbury park, ca: sage publications. mcmahon, j., gardner, j., gray, c., & mulhern, g. (1999). barriers to student computer usage: staff and student perceptions. journal of computer assisted learning, 15, 302311. mumtaz, s. (2001). children‟s enjoyment and perception of computer use in the home and the school. computers & education, 36, 347–362. nesset, v., & large, a. (2004). children in the information technology design process: a review of theories and their applications. library and information science research, 26(2), 140-61. olpc, (2008), one laptop per child. olpc.org. o‟malley, j., & mccraw, h. (1999). students‟ perceptions of distance learning, online learning, and the traditional classroom. online journal of distance learning administration, 2(4). retrieved december 10, 2008, from http://www.westga.edu/~distance/ omalley24.html. http://www.asianvu.com/bookstoread/etp/earle.pdf experiencing technology integration in education / baytak, tarman & ayas 151 prensky, m., (2001), digital natives, digital immagrants. on the horizon vol. 9(5) retrieved december 02, 2008 from http://www.marcprensky.com/writing/prensky%20%20digital%20natives,%20digital%20immigrants%20-%20part1.pdf. saye, j. (1997). technology and educational empowermena: students' perspectives. educational technology research and development, 45(2), 5-24. seidman, i. (1998). interviewing as qualitative qesearch: a guide for researchers in education and the social sciences (2nd ed.). new york: teachers college press. shell, d. f., husman, j., turner, j. e., cliffel, d. m., nath, i., & sweany, n. (2005). the impact of computer supported collaborative learning communities on high school students‟ knowledge building, strategic learning, and perceptions of the classroom. journal of educational computing research, 33(3), 327-349. speaker, k. (2004). student perspective: expectations of multimedia technology in a college literature class. reading improvement, 41, 241-254. tarman, b. (2009). the digital divide in education, eric # 508213 [retrieved on 14.12.2010 from http://eric.ed.gov/pdfs/ed508213.pdf]. yücel, c., acun, i̇., tarman, b. and mete, t. (2010). a model to explore teachers‟ ict integration stages. the turkish online journal of educational technology, 9(4), 1-9. u.s. census, internet use at home. retrieved january 27, 2008 from http://www.census.gov/population/www/socdemo/computer/2003.html and http://www.ntia.doc.gov/reports/2008/table_householdinternet2007.pdf. http://www.marcprensky.com/writing/prensky%20-%20digital%20natives,%20digital%20immigrants%20-%20part1.pdf http://www.marcprensky.com/writing/prensky%20-%20digital%20natives,%20digital%20immigrants%20-%20part1.pdf http://eric.ed.gov/pdfs/ed508213.pdf http://www.census.gov/population/www/socdemo/computer/2003.html http://www.ntia.doc.gov/reports/2008/table_householdinternet2007.pdf international electronic journal of elementary education, 2015, 7(2), 169-188. issn:1307-9298 copyright © iejee www.iejee.com non-challenging education and teacher control as factors for marginalization of students in diverse settings åsa wedin  dalarna university, sweden received: 16 january 2015 / revised: 14 february 2015 / accepted: 10 march 2015 abstract this article discusses teachers’ attitudes towards immigrant students in poor settings and the effect these attitudes have on organization of education on classroom level. it draws on results from two ethnographic studies where some primary school classes in sweden were followed with participant observation and interviews as main research methods. the article focuses on classroom activities and teachers’ attitudes towards immigrant students and students with low socio-economic status. in the article is argued for the importance of presenting students in poor settings with demanding tasks and challenging education. in these cases, intellectually undemanding tasks in combination with little room for students’ own initiatives resulted in low enthusiasm among students regarding schoolwork and accordingly low learning, while classroom work that demanded active involvement by students in combination with high level of students’ influence on what took place in classrooms resulted in high level of students’ engagement and high outcome. keywords: challenging education, immigrant students, diverse settings, teacher attitudes, deficiencies introduction in this article teachers’ attitudes towards immigrant students in poor settings is investigated, and also the effect these attitudes have on organization of education on classroom level. during my years in teacher education in sweden, i have been struck by the fact that so many teacher students seem to view children as vulnerable, neglected and in need of intense teacher supervision. it might be that children are perceived as legitimate objects for goodwill and nurturing, but i find it problematic when they are perceived as representing a collection of deficiencies. this seems particularly to be the case in many contexts involving children with immigrant backgrounds; runfors (2003) claims that they are made to represent “a minuskultur,” a culture of deficit. runfors shows how teachers in sweden have constructed “immigrants” based on assumptions about defects and marginalization. teachers in her study  åsa wedin, faculty of education, dalarna university, sweden. phone: +46 023-77 83 65 e-mail: awe@du.se http://www.iejee.com/ international electronic journal of elementary education vol.7, issue 2, 169-188,2015 170 emphasized the need to teach swedish to students with immigrant backgrounds and to compensate for their lack of contact with the swedish society. they focused on the experiences the students lacked, not on those they possessed. cases where teachers have tended to treat students with multilingual, immigrant or socio-economically disadvantaged backgrounds as less able have also been shown by knapp et al. (1995), lahdenperä (1997), parszyk (1999), queensland school reform longitudinal study (2001) and johnston and hayes (2008). although students with immigrant and/or multilingual backgrounds do not constitute a homogenous group, general academic results within these groups in sweden are below those achieved among students with monolingual swedish backgrounds. in 2005/06, 14 % of the students in the nine-year compulsory school in sweden had immigrant backgrounds, that is, they were either born abroad or both of their parents were born abroad (skolverket 2009). of these, 28 % did not pass the final exams in grade nine, while the figure for students with swedish backgrounds who did not pass was 16 %. of students with swedish backgrounds, 2.4 % did not receive a final grade at all, while the figure for students with foreign backgrounds was 8 %. one out of five students with foreign backgrounds did not qualify for a national program at the swedish gymnasium (upper secondary school, equivalent to form four to six) while the rate for students with swedish backgrounds was one out of ten. among students with swedish backgrounds who took their exam in grade nine, the final year of compulsory school in sweden, 6.4 % did not pass mathematics and 4.4 % did not pass english in 2008. the figures for students with foreign backgrounds were 14.9 % for mathematics and 13.6 % for english (skolverket 2009). this means that the failing rate is more than double for students with foreign backgrounds. the educational gap between the groups is slowly widening. in sweden, as in many western countries, the majority of immigrants are relatively isolated, living in suburbs whose populations are largely made up of immigrants and people of low socio-economic status. this means that schools with a majority of students for whom swedish is a second language are also schools in which many students are from low income backgrounds. in this article i will refer to this type of setting as “diverse”. the reasons for the widening gaps are many and in this article i will focus on one possible reason: teachers’ attitudes towards students in diverse settings and the effect these attitudes have on the organization of education at the classroom level. the article draws on the results from two case studies in sweden on language and learning in classrooms in diverse settings. 2. diverse classrooms and challenging education many teachers seem to support the “deficiency theory” regarding students in diverse settings, and this tends to marginalize these students. in her study, runfors (2003) found that students who were denoted as immigrant children tended to be kept apart, side-stepped and subordinated in ways that diminished their range of personal initiative. through her study in three different types of schools, she showed that students in a middle-class school in sweden, in which students came mainly from monolingual swedish backgrounds, were offered more opportunities to engage in individual initiatives in which the teacher acted more as a supervisor. in a mixed school, teachers often pointed out problems with immigrant students, and runfors established that the students in question were segregated. she talks about immigrant fatigue among these teachers. in a third school, in which few students had swedish as their mother tongue, runfors found both teachers and students to be more enthusiastic and more engaged. while, in these immigrant-dense schools, students helped each other and immigrant students frequently asked for help, she found that, in the mixed school, immigrant students tended to hide any lack of knowledge. from this study, one may infer that the risk of teachers focusing on the lack of swedish-language ability among immigrant students is non-challenging education and teacher control / wedin 171 particularly high in a mixed setting. in a study of “programs for measures to be taken” (sw. åtgärdsprogram) for students with immigrant backgrounds, lahdenperä (1997) found that teachers focused on deficiencies and also that they held a compensatory view particularly towards parents with immigrant backgrounds. studies by johnston & hayes (2008) and knapp et al. (1995) revealed a pattern of low educational demand resulting in poor academic results in schools in similar settings. according to knapp et al., in linguistically and culturally diverse classrooms in low-income areas, what teachers perceive as “good” education tends to have students working along a linear study path, progressing from basic to more advance levels. however, while teachers focused on the basics, they seldom reached the more advanced levels. johnston & hayes analysed implicit classroom practices in diverse classrooms in australia, what they called “taken-for-granted classroom practices”, and found that conventions and practices in classrooms were restricting students’ opportunities to learn. their findings showed that teachers practised a high level of control in class and assigned intellectually unchallenging tasks, which together resulted in low student engagement. johnston & hayes (2008) characterized this mode of teaching as a “survival mode”. johnston & hayes (2008) were surprised to find similarities in classroom patterns in all four schools in their study. similarities included minimal literacy demands, intellectually undemanding tasks, teachers asking questions on procedural levels that required a oneor two-word response and the lack of choice given to learners regarding what they could learn, and when and where they could learn it. they reported that if students were not constantly reengaged they lost interest in the lesson “game” and instead occupied themselves with activities such as chatting among themselves, moving around, listening to ipods, and daydreaming. johnston & hayes also argued that in this way teachers create “zone(s) of relative comfort” while actually restricting the learning environment. newmann & associates (1988) argued that students in what they call disadvantaged schools are often disadvantaged by the absence of demanding education. also, parszyk (1999) found teachers and students talking about undemanding teachers. in an american study of high-poverty classrooms, knapp et al. (1995) found similar practices, although in their study teachers sometimes offered students a greater variety of routines and strategies. they noted that on these occasions students showed more enthusiasm. they reported that teachers who tried to connect what happened in school to students’ earlier experiences also used more meaning-oriented approaches in their teaching. they also found that in classrooms where teachers exerted slightly less control and also offered a wider range of routines and varied instructional strategies, students showed greater enthusiasm for school work. in a longitudinal school study in australia (the queensland school reform longitudinal study, qsrls, 2001) teachers rated basic literacy and numeracy skills as the most important in school, while intellectual engagement and demand were rated least important. it was determined that constructivist views of learning were rarely implemented for students from traditionally marginalized groups and that some teachers avoided high intellectual quality and referred to behaviorist pedagogy instead. the conclusions in the qsrls were that levels of intellectual demand and social support both have significant statistical correlations with improved productive performance and, hence, improved student outcomes. the study highlighted the need for a shift in teachers’ focus, from basic skills to higher-order thinking, problematic knowledge and sustained conversations, ranging from minimal to higher levels of expectation and demand. the study concluded that schools provided supportive environments and that teachers identified students’ social needs. however, appropriate and international electronic journal of elementary education vol.7, issue 2, 169-188,2015 172 effective interventions were not developed for students from diverse backgrounds nor were interventions developed to effectively improve these students’ performance. some teachers perceived making greater intellectual demands and providing social support as alternatives while the research emphasized the importance of both. the importance of providing intellectually challenging education for second-language learners (l2 learners) in combination with high support and the explicit teaching of language has been emphasized by, among others, cummins (1984, 2000), cummins and early (2011), gibbons (2006, 2008), hammond (2006, 2008), schechter and cummins (2003), and author (2010a). these researchers oppose the tradition of modifying the curriculum for l2 learners by claiming the importance of offering students intellectually undemanding tasks. gibbons (1998) warns that a simplified approach towards l2 learners can lead to a reductionist curriculum that works against successful second-language learning and “provides an insufficient basis for the development of a language-for-learning in school” (1998:100). cummins (1996, 2000) shows the relation between these factors in his well-known coordinate system with two intersecting continua. one of the continua represents the degree of contextual support, where one extreme stands for context-embedded and one for contextreduced. the other shows the degree of cognitive demands in the language use or activity. this axis has at its extremes cognitively demanding and undemanding language. if the students’ knowledge and experience of the topic is high, the language use has been automatized and the linguistic challenge is minimal. in the opposite case, if the students’ knowledge is low, then the linguistic challenge becomes high. teaching in a second language should, according to cummins, start at the less demanding and concrete end but proceed through to the more highly demanding end, although still dealing with concrete tasks. cummins (2000) uses the terms bics and calp for these types of language respectively, with bics denoting conventional everyday language and calp denoting language that is agerelevant in academic school settings. he draws attention to the difference in the amount of time required to learn bics (1 to 2 years) and calp (5 to 7 years). thus, he stresses the need to provide l2 learners with age-relevant education at the same time as they receive language teaching and linguistic support to enable them to elaborate on curricular knowledge. cummins’ model has been criticized for being too simplified to describe such a complex and dynamic process as second-language acquisition (see for example baker, 2006). however, i find that it may serve as a model for educational planning for students whose ability to communicate in swedish as a second language has not yet reached the level that is required in school. gibbons (2008) argues that a high level of challenge and support favours all students, while low demands give rise to self-fulfilling prophecies. newmann & associates (1996) argue that all students regardless of background become more engaged when classroom activities grow more demanding. thus all students reach higher levels in an intellectually challenging environment and justice gaps diminish within this educational format. also, hammond (2006, 2008) stresses the importance of providing an intellectually challenging education to l2 learners to promote the development of higher-order thinking. these scholars all stress the importance of a linguistically supportive curriculum of knowledge in combination with highquality teaching. gibbons (2008) gives the following examples of challenging education: students are offered opportunities to participate in higher-order thinking, transformation of information, and exploration-based activities, and they construct their own understanding by taking part in real conversations with others. these examples challenge earlier-mentioned traditions for education in diverse settings, in which the curriculum has been reduced and simplified. non-challenging education and teacher control / wedin 173 by drawing on two case studies i aim to show how teachers’ attitudes and the way that they organize education in their classrooms in diverse settings may affect students’ opportunities for learning. 3. two swedish cases in the two studies i followed a number of primary school classes. the studies took an ethnographic approach and aimed at visualizing relations between students’ language development and their knowledge of school subjects, with a focus on students with immigrant and/or multilingual backgrounds. particular aims were to understand teachers’ and students’ assumptions and understandings about language and learning. the classes were observed during ordinary school days and the observations were spread over different times of day and during different types of activities. i particularly tried to find occasions where knowledge of typical school subjects was presented or dealt with, such as knowledge related to mathematics, reading and writing, science and social science. as a participant observer i did not intervene in what happened in class but responded when teachers or students addressed me. students sometimes asked me to help them or initiated conversations with me, to which i responded. teachers sometimes approached me and commented on what was going on in the classroom and shared their plans and reflections. apart from keeping field notes, i also collected written materials and students’ written texts. in the first study, in north school, about 350 hours were spent in two classes over a period of three years, and about half of the time was recorded. a tape recorder was placed visibly, usually near to where i sat and sometimes on particular students’ desks or in the teacher’s pocket. when students asked if they could listen, i arranged so that they could listen to parts where they themselves had been recorded. students and teachers were interviewed every year. in the second study, in south school, the study focused on two classrooms in which grades three and five were taught. in these classrooms video recordings were used instead of tape recordings. in both studies teachers were interviewed at the end of each school year. 3.1 the case of north school the school where the first study was carried out is located in a medium-sized town in sweden. the school is small, with just over 100 students from pre-school to grade six, consisting of students from six to thirteen years of age. most of the students in this school lived in apartments and came from relatively low-income backgrounds, while some lived in private houses. about half of the students had immigrant backgrounds, which means that one or both parents were born abroad, and most of these students lived in the apartments. some of the parents could be considered middle class, while some were unemployed and relied on support from the welfare system. the students with immigrant backgrounds in this school constituted a heterogeneous group. some were born in sweden and some abroad. some had one parent with swedish as the mother tongue and swedish as the main language in the home, while some rarely spoke any swedish with adults except in school. some (but not all) students who did not know any swedish were placed in a preparatory class for some months before being placed in the class where only swedish was used. in some cases students were placed a grade below their own age level. swedish as a second language was taught through a “pull-out model” (thomas & collier 1997), which means that students were taught swedish separated from their class for some of the lessons. this only applied to newcomers or low-performing students. some of the multilingual students also received weekly lessons in their mother-tongue. international electronic journal of elementary education vol.7, issue 2, 169-188,2015 174 3.1.1 teacher attitudes all of the teachers who taught these two classes were experienced teachers who qualified in the 1970s or early 1980s. none of the teachers had special education in the role of language in learning, nor in the learning conditions required by students being educated through a language they have not yet mastered. some of the teachers had education in special needs but none had training in teaching swedish as a second language. the teachers who were responsible for the main part of education in the two studied classes were asked about their students and the classes in the interview. the teachers were maria, responsible for the class that was followed from pre-school to grade three, and john, who taught the class that was followed from grades three to six. both focused mainly on the needs they perceived some of their students to have. they showed concern about the students they perceived as low performing. they also showed social concerns about the students, particularly as a group. maria described her class in grade-three as “a great team”, “positive” but also as “socially unstable” and she stressed the necessity to “weld the group together and make them respect each other”. she stressed that mathematics was difficult for some of them and that “the swedish-two boys” will have a tough time higher up in school”. when she talked about individual students she said that they “are on their way”, have to “polish some skills” and to “pick up one thing after the other” and she also said about some of them that they “hardly get anything done”. when john talked about his class in grade six he said that many of them had lost a great deal of time and that they “need a lot”. he said that many of them had difficulty performing at all and that it “takes a lot of push and pull” (to make them do any of the assigned work). when talking about individual students he used expressions such as “does not have time to consolidate new information”, “it takes time to pull an answer put of him, “he hasn’t got his own motor” and “it’s hard to get anything out of him”. concerning the high-performing students, john and maria did not say much other than that they were “linguistically enormously developed”, “made progress” and worked “terribly well”. 3.1.2 organization at the classroom level the way that maria and john perceived their students’ needs affected what happened in their classrooms. the main concerns that they expressed were about the needs of the lowperforming students and about social needs. they did not express concerns regarding what was taught, teaching methods or level of education or whether what they were teaching suited the students. they did not express much concern about the high-performing students, which shows that they were more focused on what they perceived as problems. there were many patterns in these two classrooms that resembled the results shown by johnston & hayes (2008). the atmosphere was very friendly and inclusive. newly arrived students were welcomed and included in peer groups, and students who had recently come to sweden and had only begun to learn swedish were supported by teachers and peers. the tasks they were occupied with resembled those described by johnston & hayes (2008) in that they were largely procedural tasks and behaviourist types of exercises with little scope for arriving at individual solutions. in the lower grades, tasks were commonly related to reading, writing and mathematics and consisted mainly of drawing lines, colouring and filling in single numbers, letters or words. the talk in class was dominated by what can be characterized as small talk or chat. discussions about school subjects were rare, as were occasions when anyone, teacher or non-challenging education and teacher control / wedin 175 student, was engaged in expressing abstract, complex thoughts. most of the time students worked individually or in small groups while they chatted with each other, and the teacher walked around the classroom assisting those who called for help. as a result, the main part of the classroom interaction consisted of dialogues with fragmented speech, overlapping speech, frequent repairs and interruptions (see author 2008, 2010a, 2010b, 2011). one example of work that followed a behaviourist pedagogy, and that did not promote or demand complex language use, was a study kit that was used in all classes in this school, and hence also in the two classes in this study. the study kit, named pilen (the arrow), was used from the second term in grade one to the end of grade six, and its importance was emphasized by all teachers. teachers and the study kit’s producers claimed that the kit developed students’ linguistic skills, although in reality it was limited to written swedish. the study kit consisted of cards and worksheets that students were supposed to work through following a fixed order. i never observed a teacher initiating a change in this pre-planned order. there was little or no room for students to use creative language in the tasks, and opportunities to make individual choices were minimal. when students happened to do a task out of its planned order, this was corrected by the teacher. the exercises trained discrete skills out of context, such as changing nouns from singular to plural or joining different given morphemes to form words. reading and writing consisted of single words or short sentences, and the writing mainly entailed copying from the cards. the work was for the most part done individually, except when students were required to play a game. a game could be a competition where the one who first reached a certain goal, such as getting five dots, won. students were not explicitly required to talk during these activities, but as they usually sat in pairs during work they chatted and asked each other about different tasks. the tasks were not cognitively demanding and, as they were of a type that mainly tested students’ skills, they did not give students many chances to produce language of their own, apart from the small talk they engaged in. one example from grade three of how the work was carried out involves an immigrant boy working with a sheet with a number of nouns, each preceded by a line on which he was expected to fill in en or att (indefinite article in singular). he came to the noun barr (needles), which he seemed not to know. then he asked his classmate sitting next to him: “vad ska det vara här?” (what should it be here?) and received the answer: “ett”, which he wrote on the line. thus he did not learn the meaning of the word but solved the task by writing down the word given by his classmate. some high-performing students quickly developed fluency in reading and writing and they used some of the time in class reading and writing extended texts. others were late in developing literacy skills and were observed to mainly read and write single words or sentences right up into the higher grades. most of the immigrant students were among this group. some of the immigrant students were barely observed to read anything even in grade six, only writing texts of a few lines and only with great support from a teacher. in the higher grades students were sometimes arranged in groups to work on various projects. here students had some opportunity to choose tasks, such as what part of sweden to focus on or whom to work with. however, within the project groups, those students who had reached high proficiency in reading and writing were the ones who did the reading and writing for their groups, while others mainly occupied themselves with activities such as throwing things at each other, tilting their chairs or daydreaming (author 2010a). consequently, low-performing students were observed to read and write very little during class time. it may be noted that, regarding reading, some of the low-performing students said in interviews that they never read outside school while some of the more proficient readers claimed that they preferred to read at home and not in school as they found the school international electronic journal of elementary education vol.7, issue 2, 169-188,2015 176 environment not good for reading. this means that, although the low performers in reading and writing did not engage in much reading and writing in school, it may be the case that they did even less outside school. the following is one example of a conversation during science lessons. it is from grade four, where a group of six students are looking for information about the swedish island öland. one of them, malin, is doing the writing while lena reads from a book about the island. this is supposed to be presented to the class afterwards. 1 lena: em … (läser) högsta höjd xxx 57 2 xxx m …ee största bredden på öland är cirka 3 20… kilometer och ölands största längd 4 är 130 km 5 maria: antal invånare 6 malin: den den största höjden på öland 7 maria: största höjden? 8 malin: ja 9 maria: den störs em … 10 love: em … kan inte nån utav oss skriva em 11 … frågor 12 (tjut i bakgrunden) 13 malin: så 14 lena: men det gör vi hela tiden 15 maria: mm 16 (tjut) 17 malin: mm där var det störs höjd 18 lena: ja em … den största höjden är … 18 malin: 57 19 lena: ja 57,4 20 malin: (skriver) komma 4 meter över havet 21 22 lena: ja em … punkt ö 23 maria: em … punkt ö punkt h punkt happ 24 love: em … antal invånare xxx ja 25 men det där har vi xxx 26 amir: kalmar e de e 27 malin: e … deras län är kalm kalmar 28 lena: öland ölands län … vänta … 29 ölands största län landskap … vänta … 30 ölands största län 31 amir: ja 32 malin: nej vänta ölands län 33 lena: ölands län heter kalmar 31 malin: ja det blir bra lena: um … (reads) the highest point xxx 57 xxx um … ee the greatest breadth of öland is about 20 … kilometres and the greatest length of öland is 130 kilometres maria: number of inhabitants malin: the-the highest point of öland maria: highest point? malin: yes maria: the highest um … love: um … can’t one of us write um questions (a howl in the background) malin: so lena: but that’s what we do all the time maria: um (a howl) malin: um there it was the highest point lena: yes um … the highest point is malin: 57 lena: yes 57.4 malin: (writes) dot four metres above the sea level lena: yes um … dot a maria: um … dot a dot h dot well love: um … number of inhabitants xxx yes but that we have xxx amir: kalmar is that is malin: um … their county is kalm kalmar lena: öland the county of öland … wait … the greatest county landscape of öland … wait … the greatest county of öland amir: yes malin: no wait the county of öland lena: the county of öland is called kalmar malin: yes that’ll be fine in this example we see that the main work is done by lena and malin, the two highperforming girls. maria also contributes to the work, but she is a slow reader and throughout the work she is usually one step behind, such as in the beginning of this excerpt when she reads the question about number of inhabitants from their question paper while malin and lena work with the highest point. the three boys in the group, love, amir and valton, are mainly occupied with other things not connected to the group work. love tries to involve himself in the group’s work twice in this example, in line 10 where he suggests that one of them should write questions and lena answers that this is what they are doing, and in line 24 non-challenging education and teacher control / wedin 177 where he talks about the number of inhabitants. this time lena and malin seem to ignore him as they work on writing about the highest point and go on reading about the county. amir has a short turn where he mentions kalmar while valton is not verbally involved in the work at all. this pattern, with a few students being occupied with the assigned task while the majority show little enthusiasm or occupy themselves with other things, was common during class time. it may be noted that the teacher’s low expectations of these three boys had been adopted by the girls, who did not demand that they involve themselves in the work. similar to the classes studied by johnson & hayes (2008), students in these classrooms had little chance to make their own choices, and they showed little enthusiasm towards the school work. particularly low-performing students needed to be motivated by the teacher repeatedly so that they would do the assignments. this was especially true with regard to boys, and in grades four to six it was true of all immigrant students. 3.1.3 one school day to give an impression of life in these two classes, a glimpse of two school days will be given, the first one in grade one and the second one in grade five. the days have been chosen to represent what commonly happened in the classrooms. grade one, may 11th 8:15 students drop in to the classroom. they walk around, talking to their classmates and move towards their individual benches, where they sit down. some bring notebooks, pencils or other things that they put in their individual drawers. the teacher, maria, comes in and puts her bag close to her table. she unpacks some things and arranges them on her table. a school assistant comes in and sits on a sofa. 8:20 maria stands in front of the class, at the blackboard. she calls in those students who are still out in the corridor. they come in and sit at their desks. some small talk is going on. the teacher calls for attention. she tells the children to put their homework in front of them on their desks. one of the children is called up to draw the calendar and to tell the class the day, the date and the food that will be served at lunch. 8:35 maria starts to talk about a task with geometry boards that they are going to work with later on. she turns to talking about flowers that the children were supposed to have picked on their way to school and bring in to press. then she starts to talk about an excursion they will be are going on the next week. after that she talks to one of the students about a written story she has found, and then she starts to give instructions to the class for the work with the geometry board. 8:45 students start to work individually with the geometry boards. 9:10 maria tells them to put the geometry boards on a table and to sit down at their desks. then she reads a story to the class. 9:20 students are sent out for a study break. 9:45 students come in and are instructed to take out their individual books for reading. they sit down at different places to read. the assistant sits with one student and reads with him. 10:25 lunch break 11:40 students come in and sit down at their desks, chatting. maria calls for attention and starts the lesson by reminding the students to bring clothes for the sports lesson next day. the students take out individual work on the alphabet. 12:10 maria and the class sing an abc song that is played on the tape recorder. 12:20 they all go out to the playground. maria hands out skipping ropes and all of the children start to jump in groups that they arrange themselves. 12:55 in the classroom again. maria hands out a test with figures in the shapes of international electronic journal of elementary education vol.7, issue 2, 169-188,2015 178 circles, loops, summits and eights. students are supposed to follow the lines with their pencils and then to construct their own figures. 13:20 maria collects the papers. students put their chairs on the tables and maria finishes the day by saying goodbye. in this classroom there is little room for students to take initiative, and the level of intellectual demand is low. the atmosphere is friendly and calm, but students are not challenged during their school work. students’ enthusiasm is low, but they engage with the work they are given. they show more enthusiasm when singing and skipping rope than when working with the alphabet and the test. there are few examples where students are required to think creatively and independently. however, most of the tasks are context-embedded and thus those students who have quite recently started to learn swedish are also included in the work and are involved in the tasks, such as working with the geometry boards. the impression is given that they are included and engaged in the tasks, but the question is, what do they learn? during the work with the geometry boards, for example, students worked individually and then showed each other the patterns they made, but no names of geometric shapes were mentioned either by the teacher or the students. thus what students actually did was form shapes and chat. similar observations were made during other types of exercises—that children did not focus on what they were supposed to focus on. one example was when children worked with scales, comparing the weight of different things and writing the results down. in this case most students focused on getting the “right” guess on their papers and comparing with their peers rather than actually estimating and guessing in advance. the following is an example of a day in grade five. grade five, may 7th 8:15. the teacher, john, walks around the classroom and prepares for the day. students walk around or sit at their tables, chatting with each other, taking things out of their bags or desks. 8:20 john starts to talk about the day. an extra teacher enters the classroom. one student is called to the table to draw the calendar and say what the date is and what food is going to be served for lunch. students who know somebody with the same name as the current name day raise their hands and in turn mention the person. john starts to call attention to those who are chatting and proceeds to give a short overview of the day. then he gives the class the instructions for a game on the geography of the nordic countries. students are quite unfocused and the instructions are interrupted by questions and small talk. students start to play the game and the extra teacher sits down at the table of two immigrant students and helps them play. when students have finished playing, john tells them to take out their maths work. 9:45 break 10:05 english lesson: john instructs the students to play a vocabulary game in pairs. for some students the words are easy and the level of enthusiasm among these students is low, while others do not know the words and need help from their classmates. when students have finished, john tells them to take out their group work on nations and to prepare for presentations. 10:50 lunch 11:40 students present the work they have done in groups on different nations. the nations are chosen according to some students’ backgrounds: finland, somalia, turkey and kosovo. students are attentive but they have difficulty understanding what is said by their classmates. when the groups ask a few questions about their presentations the classmates seldom know the answers. 12:20 students start to work with pilen. non-challenging education and teacher control / wedin 179 12:50 short break 13: 00 work with pilen continues. 13:35 students put their work away, put their chairs on their desks and then john dismisses the class for the day. although the assignments in grade five were more varied than in grade one students were still not given the freedom to plan their own work regarding what, when and how to study. for some of the students the level of the work was too low while for others it was too high. note the routine in which one student draws the calendar and says the date, a routine that had begun in pre-school and continued through grades one to six. the only thing added through the years was the reading of the name day, which started in grade four, and students mentioning some people they knew with that name. note also that the name days include predominantly swedish names, while half of the students had immigrant backgrounds. this routine may have had a disciplinary function, helping students focus on the start of the school day, but it did not challenge students or increase their enthusiasm. the main activities consisted of constructed exercises that were not very cognitively demanding, offering few challenges to students who were highly proficient. as in grade one, the social climate was friendly. yet students showed low levels of enthusiasm and concentration. the boys in the class had to be encouraged repeatedly to engage in any of the required work. as only 14 students were left in the class at this time and there were usually two teachers present, some of the students were frequently given individual support. this was true particularly of the immigrant students and some of the other students who showed low performance. the work was more context-reduced at this stage and the level of understanding and involvement of some of the students was low. this was the case, for example, during the group work in the earlier example where we could see that reading and writing was done by high-performing students while the others tended to occupy themselves with other things. 3.2 the case of south school south school, the school in focus in the second study, is situated in a large town in sweden. in south school roughly half of the students had multilingual backgrounds and teachers estimated that half of the students in the school had not yet reached a level in swedishlanguage proficiency that was expected of students of their age. similar to north school, some students came from families of a relatively high socio-economic status while the majority came from more or less poor or low-income backgrounds. despite the diversity in the school and a high number of students with special needs, the school was known to receive good results, for example, in the national exams. the school was highly regarded for its educational outcomes, and teachers expressed a high level of awareness of the needs of both students with special needs and students with immigrant backgrounds. several of the teachers had attended various in-service courses, including courses on education for students developing swedish as a second language. teachers stressed the need for challenging education, and they demonstrated awareness about language development among students. some of the teachers organized their teaching according to the systemic functional linguistics theory (halliday 1993, gibbons 2006, 2008), which stresses the importance of focusing on language development in the context of all school subjects. here examples will be taken from grades three and five, the classes of anna and nina. 3.2.1 teacher attitudes in interviews, both anna and nina stressed the potentials and abilities they saw in the students and talked about their progress. anna’s classroom, grade three, may be international electronic journal of elementary education vol.7, issue 2, 169-188,2015 180 characterized as highly diverse, with some of the students being “the cream of the crop” while others had been diagnosed with autism and/or with severe language disorders. while some children came from high-status families, others came from homes of low socio-economic status and homes with criminal backgrounds. in the interview anna talked about the importance of maintaining a sense of humor and compassion, and she stressed the need to create a spirit of community in the class. she talked about her students as melodramatic saying that there were “many artists, sentimentalists” in the class and that they were “an impatient class”. she said “they are at all ends but they need to be framed in” and “there is lots of talent here, but there is the matter of keeping order”. nina talked about individual students as either having high standards or having “developed tremendously well”, and she used words such as “exceptionally”, “marvellous emphatic ability”, and “giant vocabulary” when she described the development and performance of the students. concerning the development of one student she said, “he had a decline when he was sort of … tired but then he made tremendous progress again.” 3.2.2 organization at the classroom level the focus that anna and nina placed on students’ potentials, abilities and progress was visible in the organization of their classrooms. students in both classrooms were active and enthusiastic. anna in particular put a great deal of energy into getting students’ attention and getting them to focus on what they were supposed to be doing. in the mornings, and in other cases when she wanted to gather the class for joint conversation or discussion, she would gather them in a ring on a mat at the front in the classroom. the following example is from one morning — the first one after the midwinter holiday. fourteen students, the teacher and an assistant teacher sat around the blue mat. anna started by making sure she had the students’ attention. she counted “one, two, three: good morning!” and the whole class answered “good morning” in chorus. she stated that one of the girls, marie, had got a tan as she had just returned from a holiday trip to thailand. she asked marie about the holiday and marie had started to tell the others about it when she was interrupted by a parent who entered the classroom and asked about some homework for her child, who was ill. marie continued to tell her story but anna asked her to wait and rose to attend to the parent. after a few minutes the parent left and anna sat down again. she made sure everybody was listening again before she asked marie to continue: marie: ja och sen fick vi åka en jättejätte stor båt som var jättefin den var så här vit hela den var vit den var så här anna: är det som man åker ut till öar då … den båten marie: ja… em och då så vi fick sitta där uppe … det fanns så här som staket så fick man sitta där framme vid båten där uppe alltså vid taket det fanns sånt här staket (visar med händerna) där kan man sitta då och vinden blåste så jag kunde inte andas jag bara a-a-a (drar efter andan och skrattar) det var asroligt och sen … i … ee när vi var på stan anna: vilken stad är det man är i då? marie: mm det är vänta … det är patong … patong är det väl eller nej puket nej anna: mig kan du säga vilket vi går på vilket marie: yes and then we got to go with a huge boat; that was very nice, it was like white all of it was white it was sort of anna: is it that you take to islands then ... that boat? marie: yes … um, and then we could sit up there … it was this sort of fence so you could sit there in front of the boat, that is, at the roof there was this fence (shows with her hands) so you can sit then and the wind blow so that i couldn’t breathe i like a-a-a (pretends to have lost the breath and laughs) it was awesome and then ... in ... um when we were in town. anna: what town are you in then? marie: um it is wait ... it is patong ... patong isn’t it or no puket no… anna: to me you can say whatever one and we’ll believe it non-challenging education and teacher control / wedin 181 som helst marie: ja och sen då am anna: patong eller puket am marie: och sen så sen när vi när vi var på stan då så det var massor såna där som säljer saker alltså det var så himla många och så sen så här kommer dom så här: ”ei tuk tuk!” (sträcker upp höger arm i vädret) eh det är taxi så säger dom ”ei tuk tuk!” (höger arm upp) och så här på pappa (slår med armen på kamraten till höger) bara för att man ska få så där taxi eller ja marie: yeah and then am anna: patong or puket um marie: and then, well then when we, when we were in town there were lots of these who sell things, that is there were amazingly many and then there they come like this: “ei tuk tuk” (puts her right arm up in the air) eh that’s taxi so they say “ei tuk tuk” (right arm up) and like this on dad (hits the girl to her left with her arm) just in order that you get this taxi or yes she continued by talking about the pools, and beaches, and when she finished, one classmate who did not yet have a high level of swedish said: arin: men vänta … om man springer och sen bara åker in i den där början på bassängen så bara sjunker man arin: but wait … if you run like this and then go into that beginning of the pool then you only sink anna assumed that what he meant may not be understood so she tried to clarify: “he means if it becomes deep all at once in the pool.” arin verified this with “yes” and marie answered that this was the case. in this case anna used marie’s experience to give her the opportunity to take the floor. she explained afterwards that the reason she chose to ask marie and give her so much time during that morning assembly was that marie was among those who seldom spoke out in class and that she usually did not narrate with cohesion. anna made sure that marie got the attention of all the students and she supported her by asking her strategic questions. afterwards, when arin had problems expressing his question, she supported him by clarifying the meaning of the question. by ensuring the attention from all students and giving support, she managed to engage students in real conversation and to give one student an opportunity to take the role as knower. in similar ways she often related to students’ experiences and tried to make the most of their initiatives when possible. she explained in the interview that she wanted to build on students’ experiences although she had to make sure that all students got their chance and thus had to hold some of the more talkative ones back sometimes. in the grade five classroom, the teacher, nina, also made a point of keeping a level of teaching that challenged all students and related to students’ own interests. she planned the work by using different types of teaching aids and did not follow pre-made plans in particular study kits. when students were doing individual work they were involved with various types of tasks and on different levels. in the centre of the classroom there was a computer, which was usually being used by one of the students. there was another computer in one corner, and there were also some laptops available. students used the computers to work on different learning programs but also to work on their own home pages or, less frequently, to search for information on the internet. three of the boys would bring their own memory sticks from home to work on their own home pages. during the lesson from which the following example is taken, students worked individually on different types of tasks. ten students were in the classroom and two of them were practising multiplication together at a table. two girls worked with geography using programs on the computers. one boy was at a computer writing a text about his favourite interest, motocross training. one boy and a girl were reading fiction and two boys worked on a project about famous artists. one of the boys, victor, spent extended time at the computer. he had international electronic journal of elementary education vol.7, issue 2, 169-188,2015 182 brought some pictures on his memory stick and was editing them in photo shop for his home page. nina, the teacher, walked around in the classroom helping students who needed help or just sitting down to listen to questions on their individual work. most of the time she spent with three of the students who she thought needed extra support. she started with the two boys practising multiplication. then she sat down a while with the girl who was reading and talked with her about the book. after about 20 minutes she came to victor, sat down on a chair next to him and asked him to explain what he was doing. he explained about his work on the home page, pointing out things on the screen as he talked, and nina listened attentively and watched the screen. then she said: n: ok ja … ja jag kan inte det där alltså men jag förs … jag vet att eller jag kan inte göra det själv men jag förstår det ja … jag kanske skulle sätta mig och träna … jag tycker att jag skulle vilja ha en hemsida kanske det vore kul … mm vad tänker du göra idag då? v: det här gjorde jag idag … den här själva knappen n: jaha ok v: det här är exakt samma description som jag har n: ok yes … yes i don’t know that that is i unders… i know or i can’t do it but i understand it yes … maybe i should sit down and try it out … i think that i’d like to have a home page, maybe that would be fun … um what are you going to do today then? v: this i did today … this the very button n: yeah ok v: this is exactly the same description that i have while victor explained, he flipped between pages, scrolled on a page and pointed at the screen to indicate what he was talking about. note that he used the english “description” instead of the swedish beskrivning. nina followed and watched attentively. after a while she nodded her head to show that she understood. she said, “perfekt!” (perfect) and continued to another student. in these two examples we see that students concentrate on their work and that there is room for students’ initiatives. if we relate to gibbons’ description of challenging education, we see that in this classroom there is not only room for individual construction of knowledge and authentic conversation, but also that there is room for switching expert-learner relations. in both examples students became experts and the teacher agreed to take the role of the learner, without losing control over the classroom. both anna and nina made room for students’ initiatives and gave them responsibilities while keeping the level of demands high and making sure students focused on their work. 3.2.3 one school day to create an image of life in the classrooms of anna and nina i will give an overview of a day from grade three. 8:25 morning assembly. students are gathered on the mat in front of the classroom. after some small talk anna focuses the students’ attention on the date and the schedule for the day. she explains generally what will happen and writes in english on the whiteboard. one of the students complains about some classmates’ behaviour in relation to her birthday party and anna reminds her that birthday parties have to be discussed outside class and that this is because some children may have them while others may not, as their families are unable to afford them. 8:40 anna starts to talk about a new theme in science, the ear and hearing. she starts by asking the students what they know about the topic. students raise their hands and the talk continues. 9:10 anna shows a short film about the ear and hearing. non-challenging education and teacher control / wedin 183 9:15 she tells the students to go back to their desks. she asks them if they learned anything new. she takes the class’s lizard out of his terrarium to show his ears. when some boys start to talk about other things not related to the topic she asks them, “boys, should this be about you or about hearing?” and the boys answer, “hearing”. she continues: “are you to be brought up or to learn?” and they answer, “learn”. 9:45 break 10:05 mathematics lesson. anna starts by presenting some multiplication and division questions. she instructs them to work together and the students work in groups to solve mathematic problems based on a zoo. 10:40 students go to another part of the school where they have their music lesson with another teacher. 11:30 the music teacher sends them to lunch. 12:10 english lesson. students work on a story about pippi longstocking in english. anna has copied the book and they start by reading a few pages together. anna reads first and the students read after. she explains some words and phrases. then students read the text aloud in pairs. 12.50 silent reading. students take out their books and read silently. every student has borrowed a book from a shelf in the classroom or from the school library. 13.30 anna asks students to put the books away and dismisses the class. as we can see, anna put a great deal of effort into making the students focus on the school work. this became apparent when she refused to let talk about birthday parties occupy too much time and when she reminded the boys to focus on the topic of the lesson. in the three topics that were dealt with, english reading, multiplication/division and hearing, we can see that pre-fabricated teaching aids were not used as the basis for planning but were used as supplementary resources, such as the film about hearing. in english, anna tried to make the lesson authentic and put it into a natural context by using english when she presented the schedule for the day and using a storybook as the basis for a lesson. it should be noted that english is taught as a foreign language starting in grade three, which means that these students had only just started formal english learning. when introducing the topic of hearing, anna made an effort to start from students’ prior knowledge and also provided students with an opportunity to give feedback after the film. the tasks in mathematics were of a type that demanded engagement from all students in the groups, and the groups were small enough to ensure that all were involved in solving the problems. 3.3 teacher attitudes and organization at the classroom level although i state that there are many similarities between what happened in the classrooms of maria and john in north school, and in those of anna and nina in south school, there are certain differences in attitude and classroom organization. while the north school teachers mainly assigned procedural-level tasks, in the south school classrooms there were examples of challenging lessons and opportunities for students’ influence over their learning. maria and john seemed to adhere to what runfors (2003) called a deficiency theory regarding their students, which results in undemanding education and a high level of teacher control regarding learning. anna and nina, on the other hand, may be characterized as holding a theory of ability and potential regarding their students. in their classrooms we see a high level of student involvement. in the classrooms of maria and john, tasks were mainly procedural and questions rarely demanded more than oneor two-word answers. while work in their classes was mainly based on teaching aids, followed from the first stage to the last, students in anna’s and nina’s classes were assigned work that demanded active involvement. when comparing the morning routines in the classrooms, we see that maria and john followed a international electronic journal of elementary education vol.7, issue 2, 169-188,2015 184 strict formula, for example, having students in each grade say what the date is, whereas anna took a more flexible approach, inviting a student to talk about a personal experience. this flexibility, along with anna’s use of english when writing the schedule, placed more demands on students and enabled them to influence what took place in the classroom. nina’s inclusion of students’ personal interests in school work, by having them design home pages, for example, gave students opportunities to develop particular skills. by making room for students to work with topics she herself had not mastered, nina extended the potential for learning. these types of skills were not observed in the classrooms of maria and john. as may be expected, anna’s and nina’s students performed well in national tests, while the students of maria and john did not. 3.4 language use in the classrooms linguistic factors in the school situation are particularly relevant for students who have not yet achieved the expected proficiency level in the language of instruction, in this case swedish. in north school, language may be characterized as fragmented. talk consisted mainly of short statements, in the form of small talk. rarely did anyone, student or teacher, take the floor to express complicated thoughts or to talk about school subjects; in fact, the school subjects were seldom talked about at all. most of the students for whom swedish was a second language spent very little time reading or writing, and usually what they read or wrote consisted of single words or short sentences. in south school, in contrast, there were more frequent lengthy discussions, in which complex thoughts were expressed. through small-group work, which demanded active participation of all students, language use became more varied; this applies mainly to oral expression, but also to written forms. teachers focused on the form of the language by directing students’ attention to words, phrases and styles, and also on its use by ensuring that students were given opportunities to use language in demanding situations in both oral and written forms, as in the case of marie’s narration about her experiences in thailand. the higher level of engagement among students in the classrooms of anna and nina also had an effect on the amount of reading and writing done by students. when considering that the low-performing students in maria’s and john’s classes claimed to read and write only in school, classroom organization stands out as of particular importance for the development of these students’ literacy skills. 4. discussion i conclude that some of the teachers studied in these projects seem to hold a deficiency perspective, as did the teachers in the studies by runfors (2003), knapp et al. (1995) and johnston & hayes (2008). however those two teachers who dared to risk giving responsibility to students, to change roles and take the role of learner, to give the floor to students and at the same time to demand students’ attention and require them to focus on their work may be characterized as holding a perspective of ability, potential and development. in the examples from north school, students’ proficiency was low and so were the cognitive and linguistic demands of their school work. the classrooms were inclusive in the sense that students took care of each other and made sure that new students were included, but tasks in the classrooms seldom challenged students’ thought processes. in earlier grades, when tasks were context-embedded, students who had recently started to acquire swedish were involved in classroom activities, but in the higher grades second-language students whose proficiency in swedish was low tended to occupy themselves with other things and needed to be re-engaged constantly by the teachers. non-challenging education and teacher control / wedin 185 as in the cases studied by johnston & hayes (2008) and knapp et al. (1995), in these classrooms students spent most of their time occupied with lower-order intellectual tasks, showed little enthusiasm and poor concentration. similar to the results of the qsrls (2001), in this case the teachers seemed to identify social needs and to provide a socially supportive environment in school but at the cost of academic challenges. while students seldom actively opposed the teachers’ efforts, the energy they invested in classroom activities remained low. furthermore, the students’ level of engagement decreased over the years. in the classrooms from south school, however, classroom work that demanded active involvement by students in combination with high level of students’ influence on what took place in classrooms resulted in high level of students’ engagement and high outcome. it is interesting to see that also the actual amount of reading and writing done by students, particularly among students from low income backgrounds, was higher in these classrooms. that teachers gave room for students to develop skills in areas where students’ knowledge went beyond the teacher’s extended students opportunities for learning. in the technology intensive development we see today this is particularly important. high teacher control over content and learning seemed to be linked to low student engagement in these studies also, while low teacher control in combination with high control regarding focus on learning was linked to high student engagement and high outcome. the results highlight the risk of treating immigrant students, as well as other students from socioeconomically unprivileged backgrounds, as less competent and less able than those from other backgrounds. a focus on basic skills at the cost of a demanding and challenging education denies students equal opportunities to achieve academic success. ensuring that all students have access to an education that is linked to their experiences and builds on their potential is a democratic issue. in any school with democratic aspirations, teachers need to be aware of each student’s potential. social, economic or linguistic background should not exclude any student from access to an exciting and challenging education. this means that teachers need to dare to decrease their control of details in the learning process, such as when and how different things are learned. in order to create classrooms where negotiated interactive learning takes place, teachers need to present students with intellectually demanding tasks and challenging education. • • • åsa wedin is associate professor in education. she achieved her phd in linguistics on a thesis with the title: literacy practices in and out of school in karagwe the case of primary school literacy in rural tanzania. she has published on multilingualism and literacy education and has also written and edited several books. she is currently a head of the department of swedish as a second language at högskolan dalarna, sweden. contact: awe@du.se international electronic journal of elementary education vol.7, issue 2, 169-188,2015 186 references author, (2008). monologen som en resurs i klassrummet. (the monologue as a resource in the classroom) pedagogisk forskning i sverige, 13(4), 241-256. author, (2010a). a restricted curriculum for second language learners. a self-fulfilling teacher strategy? language and education, 24(3), 171-184. author, (2010b). narration in preand primary school. a resource for language development and multilingualism. language, culture and curriculum, 23(3), 219-233. author, (2011). utveckling av tal och skriftspråk hos andraspråkselever i skolans tidigare år. (development of oral and written language in second language students in primary school) didaktisk tidskrift, 20(2), pp. 95-118. baker, c. (2006). foundations of bilingual education and bilingualism. clevedon: multilingual matters. cummins, j. (1984). bilingualism and special education. issues in assessment and pedagogy. clevedon: multilingual matters. cummins, j. (1996). negotiating identities. education for empowerment in a diverse society. ontario: california association for bilingual education. cummins, j. (2000). language, power and pedagogy. bilingual children in the crossfire. clevedon: multilingual matters ltd. cummins, j. & early, m. (2011). identity texts. the collaborative creation of power in multilingual schools. london: institute of educational press. gibbons, p. (1998). classroom talk and the learning of new registers in a second language. language and education, 12(2), 99-118. gibbons, p. (2006). bridging discourses in the esl classroom. students, teachers and researchers. london: continuum. gibbons, p. (2008). "it was taught good and i learned a lot". intellectual practices and esl learners in the middle years. australian journal of language and literacy, 31(2), 155-173. halliday, m. a. k. (1993). towards a language-based theory of learning. linguistics and education, 5(344), 93-116. hammond, j. (2006). high challenge, high support. integrating language and content instruction for diverse learners in an english literature classroom. journal of english for academic purposes, 5, 269-283. hammond, j. (2008). intellectual challenge and esl students. implications of quality teaching initiatives. australian journal of language and literacy, 31(2), 128-154. johnston, k. & d. hayes, (2008). “this is as good as it gets”: classroom lessons and learning in challenging circumstances. australian journal of language and literacy, 31(2), 109-127. knapp, m., p. shields, & b. turnball, (1995). academic challenge in high poverty classrooms. phi delta kappan, 76, 770-776. lahdenperä, p. 1997. invandrarbakgrund eller skolsvårigheter. en textanalytisk studie av åtgärdsprogram för elever med invandrarbakgrund. phd diss. stockholm: h l s förlag. newmann, f. m. & associates (1988). authentic achievement. restructuring schools for intellectual quality. san francisco: jossey-bass. parszyk, i.-m. (1999). en skola för andra. minoritetselevers upplevelser av arbetsoch livsvillkor i skolan. phd diss. stockholm: h l s förlag. queensland school reform longitudinal study. 2001. school of education, university of queensland. australia: the state of queensland. runfors, a. (2003). mångfald, motsägelser och marginaliseringar: en studie av hur invandrarskap formas i skolan. stockholm: prisma. schechter, s. & cummins, j. (2003). (eds.) multilingual education in practice. using diversity as a resource. portsmouth, nh: heinemann. non-challenging education and teacher control / wedin 187 schleppegrell, m. (2004). the language of schooling. london: lawrence erlbaum. skolverket (2009). [retrieved june 16, 2009]. available at http://www.skolverket.se/statistik/grundskolan/betyg och prov thomas, w. & v. collier, (1997). school effectiveness for language minority students. ncbe resource collection series, no. 9, george washington university. http://www.ncbe.gwu.edu/ncbepubs/resource/effectiveness international electronic journal of elementary education vol.7, issue 2, 169-188,2015 188 www.iejee.com this page is intentionally left blank http://www.iejee.com/ students’ performance calibration in a basketball dibbling task in elementary physical education international electronic journal of elementary education, 2013, 5(2), 171-184. issn:1307-9298 copyright © iejee www.iejee.com at the roots of finnish elementary education – how were children raised in the first finnish elementary schools? satu uusiautti university of lapland, rovaniemi, finland merja paksuniemi university of lapland, rovaniemi, finland kaarina määttä university of lapland, rovaniemi, finland received: 8 february 2013 / revised: 5 march 2013 / accepted: 15 march 2013 abstract this study dissects the history of finnish elementary education and the way children were raised during the initial phase of finnish education in the 18th century. the development of finnish education was studied through contemporary decrees and laws and studies of finnish school history. the preliminary aim was to focus on the principles and practices of raising children in finnish schools. this study focuses on (1) describing the birth, goals, and practices of finnish elementary education and development toward compulsory education, (2) describing the way children were raised toward the contemporary goals, and (3) dissecting the connection between teacher training and the goals of raising children in finnish schools. all these viewpoints are discussed from the viewpoint of how the aspirations and objectives were realized and implemented in practice in raising children. as a conclusion, we discuss the influence of the past in today’s educational practices. keywords: education, history of education, finnish educational system, child raising, elementary education. introduction nowadays, in developed countries such as finland, both parents of the vast majority of families with children under school age are in full-time employment. along with the changes in working life, the role of day-care centers and schools in rearing has strengthened (määttä & uusiautti, 2012). professionals define what is good child rearing and what kind of rearing should be implemented inside citizens’ homes. the rearing task has become professionalized  satu uusiautti, university of lapland, p. o. box 122, 96101 rovaniemi, finland, phone: +385 40 484 4167. email: satu@uusiautti.fi http://www.iejee.com/ international electronic journal of elementary education vol.5, issue 2, 171-184, 2013 172 (bimbi, 1992; greenwood & hickman, 1991) and from the parents’ point of view, parenthood can be considered shared with public education institutions (bauch, 1994; björnberg, 1992; cutting, 1998). in the society of rapid change, the future generation is likely to differ from the present and therefore, educational systems are in constant change too (jakku-sihvonen & niemi, 2007; naumanen & rinne, 2008). the changes cause uncertainty about the norms and what kind of rearing is the best for children (see lahikainen & strandell, 1987; young, 1995) and what would be the best for the society too (e.g. milligan, morretti, & oreopoulos, 2004); and this topic is under constant debate (see e.g. howard, 2003). this kind of development is evident in the educational history of finland (iisalo, 1979; see also simola, 2005). in this article, we direct our attention to the past. we dissect the history of finnish elementary education and the way children were raised during the initial phase of finnish education. here, the concept of elementary education refers to the first form of basic education in finland but which was not yet defined compulsory or comprehensive. certain educational trends influenced on finnish educational system laying the foundation of today’s finnish education. our article focuses on the period that started in the 18th century and the development of finnish education was studied through contemporary decrees and laws and studies of finnish school history. the article is based on our previous educational historical studies on the history of finnish teacher training colleges (paksuniemi, 2009; paksuniemi & määttä, 2011a,b,c; paksuniemi, uusiautti, & määttä, 2012, 2014) complemented with a review of previous research and an analysis of relevant laws and decrees. our preliminary aim was to focus on the principles and practices of raising children in finnish schools. this article focuses on (1) describing the birth, goals, and practices of finnish elementary education and development toward compulsory education, (2) describing the way children were raised toward the contemporary goals, and (3) dissecting the connection between teacher training and the goals of raising children in finnish schools. all these viewpoints are discussed from the viewpoint of how the aspirations and objectives were realized and implemented in practice in raising children. as a conclusion, we discuss the influence of the past in today’s educational practices. socially acceptable citizens as the goal until the mid-19th century, the christian church defined the content and direction of finnish citizens’ education. since the reformation, the cornerstones of education were the lord’s prayer, creed, and the decalogue. the 1723 king’s decision said that parents had to teach their children to read or send them to be taught by the parish personnel—otherwise, they would be fined. in 1773, all children had to go to confirmation classes and in 1776 it was defined that parishioners’ reading skills had to be tested before their take their first communion. education provided by the church was based on the teaching of reading skills and learning the catechism and dogma of christianity (juva, 1995; lipponen, 2003). the supporters of comprehensive education wanted to remove education from the church to be the society’s responsibility but the church did not agree because of the fear of comprehensive education being too liberal (lipponen, 2003). the father of finnish elementary education, uno cygnaeus, had adopted the basic idea of pestalozzian pedagogy (iisalo, 1989). according to the pedagogical trend, it was important to invest in individual education for the development of the society. every adult had to take care of the balanced development of children’s body and mind. both theoretical and manual skills had to be taught at school, such as handicraft and gym. the school was responsible for turning information into “living conviction” in pupils (cygnaeus, 1910; iisalo, 1989; tuomaala, at the roots of finnish elementary education / uusiautti, paksuniemi & määttä 173 2004). according to cygnaeus, elementary education had to be based on christianity but it also had to teach practical skills. consequently, elementary schools were not supposed to be just a place of doing schoolwork. elementary education was to be the whole nation’s basic education. moreover, cygnaeus aimed at arousing love toward the underprivileged. his objective was to develop elementary education through practical skills: pupils had to train their coordination, observation, and sense of aesthetics and working with tools (cygnaeus, 1903; isosaari, 1961; tuomaala, 2004). these thoughts of cygnaeus differed totally from j. v. snellman’s, one of the most influential fennomans, ideas: snellman emphasized “intellectual education” as the most important task of elementary education (iisalo, 1989, pp. 123–124). snellman’s views were mostly based on hegel’s tenets. according to snellman, the school had succeeded if it made a pupil fall in love with information seeking (koski, 1999; lehmusto, 1951). the most essential difference between cygnaeus’s and snellman’s opinions was that snellman would emphasize intellectual education whereas cygnaeus aimed at balanced education. snellman’s thoughts won because elementary education had emphasis on the former. snellman argued that the route through discipline from the nature to culture, from necessities to freedom, and from insanity to intelligence varied by pupils’ ages. snellman distinguished three educational environments: the home, the civil state, and the society. strict discipline was to be implemented in schools: “we consider discipline as the strongest support of education at school.” snellman highlighted that the task of school was to educate decent citizens who follow the society’s habits and law. obedience to the law was indirectly taught through discipline and order at school. the primary task was, however, to educate the whole nation (ojakangas, 2003; melin, 1980; paksuniemi, 2009). the starting point of folk education leaned on the idea of the nation’s development as the primary purpose of individuals’ lives. the individual virtue was considered to represent the nation’s virtue. every human being has to perceive himself or herself as unique and morally and spiritually responsible individual who is a part of something greater, common earthbound and spiritual good. as hegelian thinking was combined with the traditional lutheran ethics prevailing in finland of that time, the idea of a good child and educated citizen was based on the development of a decent, god-fearing, and useful citizen (koski, 1999). the general purpose was to educate the people and raise the general level of education in order to avoid conflicts, such as the civil war in finland in 1918. moral-societal aspirations were based on the lutheran tradition of upbringing (koski, 1999). children were considered a gift from god, not for parents but the heaven. lutheran upbringing focused on the devout everyday life and emphasized the divine origin of civic virtues (cunnigham, 1995, kuikka, 2003). snellman and cygnaeus had disagreements especially on questions related to primary education. snellman would have left primary education to parents but cygnaeus wanted to have small children taught at school. thus, the debate concerned the relation between upbringing at home and at elementary education (lipponen, 2003). education in elementary schools – toward christian-decency through order and discipline decree on elementary education took effect in 1866. due to the decree, education in finland started to have uniform and organized features (decree on elementary education 11 may 1866 § 102, 105; paksuniemi, 2009; valta, 2002). according to the decree, elementary education had to be six years long in cities and four years long in the countryside (nurmi, 1981). in cities, pupils had to be 8–14 years old. additionally, the law defined the maximum number of pupils per classroom: the first grade could have maximum of 40 children and other grades 60. international electronic journal of elementary education vol.5, issue 2, 171-184, 2013 174 (decree on elementary education 11 may 1866 § 102, 105.) the decree did not aim at making elementary education available for all but at giving directions to the development of education (halila, 1949a). elementary schools had to be established in cities so that every 8– 12-year-old children who wanted to study could go to school. elementary education had two levels: lower and upper elementary education. at the lower level, the primary school, girls and boys were taught together but at the upper level, they were taught in separate classrooms. (decree on elementary education 11 may 1866 § 104.) the decree was changed in 1889 when in new schools, girls and boys could be taught together if needed (decree on elementary education 27 jun 1889; the merciful decree of imperial majesty, 1989; finnish decree collection no. 26/1889). the decree on elementary education did not make education compulsory and therefore, it could not oblige municipalities and cities to establish elementary schools (hiltunen, 1983). the social-political background was evident in the elementary school curriculum. the goal was to teach pupils to be diligent and to introduce new livelihoods for example by teaching handicrafts. finland was the pioneer of handicrafts education in the world. education emphasized everyday skills and the ability to make everyday utility articles because the idea was that pupils would return farm work after school (halila, 1949b). pupils were part of the family workforce and farm work determined children’s participation in education. due to this, children attended school irregularly (lipponen, 2003). christianity was strongly present in teaching and the teacher’s task was to instill true fear of god, gratitude, and trust in god into pupils’ hearts and to encourage them to show their fear of god by obeying parents and teachers (cygnaeus, 1910; haavio, 1941; salo, 1934). in christian-decent education, innocent heart, teetotalism, humbleness, obedience, sense of duty, charity, helpfulness, kindness, tidiness, honesty, and bravery were considered the features of a good child as it could enable the relationship between a child and god. respectively, a bad child would bully, mock, be proud, lie, steal, and be disobedient, sloppy, and lazy (koski, 1999). the decree on elementary education also included means of punishment that a teacher could use if a pupil did not, regardless of the teacher’s request, follow the school rules. this did not only concern disobedient pupils but also lazy and remiss pupils. before the actual punishment, the teacher had to give a warning (valta, 2002). if necessary, the teacher could use the following forms of punishment: to reprehend and give a warning to a pupil in the presence of the whole class, to move a pupil’s place lower (pupils would sit in a ranking order and a lazy and careless pupil would be replaced), to separate a pupil from other pupils, to set detention, to punish physically which meant six rod strokes on palms in the presence of the whole class, and finally, to expel a pupil from school (decree on elementary education 11 may 1866 § 96–97). furthermore, the decree provided detailed instructions on who could punish and how to use and interpret the punishment directions. for example, reprehending was the immediate way of reacting to a pupil’s disobedience and separation from others meant usually standing in the corner. the grosser a pupil’s offence was—such as disobedience or questioning the teacher’s authority—the more severe was the punishment (valta, 2002; heporanta, 1945). other forms of punishments were pulling hear, hitting with a pointer, and putting a pupil stand behind the blackboard (valta, 2002). consequently, the decree on elementary education formed the basis for punishments, too. the directions were applied in finnish elementary schools with little changes, such as how the forms of punishments were called. otherwise, the forms of punishments did not change much at the roots of finnish elementary education / uusiautti, paksuniemi & määttä 175 in later decrees and laws. the most significant change happened in 1914 when physical punishments were forbidden (law on physical punishments, 1914 decree 6 june 1914 n:o 24). pupils had to be raised to be god-fearing, patriotic, and obedient to law and authority, to show good manners and diligence, and therefore, discipline at school had to be strict. for example, raising hand during a lesson had separate directions: in order to maintain good order, it is necessary to require that children, immediately after the bell ringing, go next to their desks, without any noise and rattle and place their chair close enough to the desk (that is also practicing of nice behavior) while the teacher reads 1, 2, 3 and puts everyone in order and sit straight, in natural positions with hands on the desk, looking at the teacher. the teacher’s short and rigorous and otherwise educational question is addressed to the whole class; those who think they can answer the question will raise their right-hand forefinger a little (do not raise and wave their whole arm) and the one, who is allowed to answer, answers with a perfect sentence so that the question is enclosed in the answer. this way of raising hand and perfect answering must not be neglected by any means in elementary schools (cygnaeus, 1910, p. 525). if the aforementioned procedure was followed, the teacher did not have to use punishments (cygnaeus, 1910). as the elementary education system was new and class sizes were big, strict discipline was needed in schools. some pupils adjusted in this regimentation but in some pupils it caused hostility to school. especially, restless pupils found it difficult to sit still because they were not used to it. at the beginning period of elementary education, the number of punishment was high because pupils were not familiar with habits required at school. their parents had not usually gone to any school and therefore, they could not prepare their children for school. neither did they appreciate education which was shown in pupils’ low motivation toward studies. the occurrence of disturbance was a problem of the beginning phase of elementary education, but as pupils got used to school habits, discipline was not considered a problem. on the other hand, there were also children who were subservient to school and there was no need to punish them. moreover, elementary education being voluntary made the situation a little easier as ill-bred children quit or did not come to school (valta, 2002). the birth of compulsory education and standardization of education one of the most important reforms of the 19th century took place in 1866 when emperor alexander the ii gave the command for establishing the elementary school. the merciful decree of imperial majesty in 1898 obliged municipalities to establish schools although compulsory education had not yet taken effect (decree on school districts, 1898 § 1, finnish decree collection no. 20/1898; paksuniemi, 2009; päivänsalo, 1971). decree on districts was regarded as the first law on compulsory education in finland although it did not mean compulsory education as such. however, according to the decree countryside municipalities had to form elementary school districts and to provide elementary education to every schoolage children in their native language. a school had to be established if the district had at least 30 children willing to go to school. school commute had to be no more than five kilometers (~3.1 ml) (decree on school districts, 1898 § 1–3; finnish decree collection, 20/1898; nurmi, 1983; paksuniemi, 2009). this part of the decree was not followed everywhere, for example in sparsely-populated municipalities (decree on school districts, 1898 § 2 and 3; finnish decree collection, 20/1898). if a school had over 50 pupils, a new school had to be established or an assistant teacher had to be hired (decree on school districts, 1898 § 3; finnish decree collection, 20/1898; heporauta, 1945). the decree on districts was considered an improvement. during the first year after the decree took effect, the number of pupils increased by 5,000. more and more pupils went to school in countryside. along the decree on international electronic journal of elementary education vol.5, issue 2, 171-184, 2013 176 districts the total number of pupils covered by school districts increased from 68,000 to 190,000 in 1920. the decree solved some of the flaws in the decree of elementary education although primary education still remained scattered and defective. most importantly, the development of the finnish elementary education system accelerated (decree on elementary education, 11 may 1866 § 96–97; paksuniemi, 2009; tuunainen-nevala, 1986). a bill of compulsory education was proposed already in 1910 but was not passed because of the objection of tsar nikolai the ii. the situation changed after finland gained independency in 1917 (männistö, 1994). compulsory education was objected because of its costs and because it would turn elementary education obligatory. this was considered contrary to the general sense of justice and it was feared of causing reluctance, which, for its part, was believed to weaken the position and appreciation of elementary education (halila, 1949b). moreover, children’s parents objected the law on compulsory education even more than was expected. one reason was ignorance. in addition to that, parents were afraid that their appreciation as educators would weaken. some parents considered elementary education as a factory that did not recognize individuality (oinasmies, 1945). traditionally, countryside children had been taught by the church or at home (werkko, 1903). until the 1840s, vergers took care of teaching but only a few of them could actually teach writing and math. after the decree of elementary education, it was still possible to hire unqualified staff in primary education and therefore, teachers represented quite a motley group of people (halila, 1949a; iisalo, 1989). parents had the main responsibility for teaching and education but if they were unable to teach reading, finnish, and the teachings of christianity adequately, charter schools provided by the church arranged education also in writing, singing, and math (the merciful decree of imperial majesty 1866 §114; hyyrö, 2006; iisalo, 1968; soikkanen, 1966). even after the decree took effect, primary education was still mainly provided at homes and was controlled by priests and thus, the quality of teaching varied greatly—resulting to a discussion over the standardization of primary education and the education of primary teachers (hyyrö, 2006; paksuniemi, 2009). the temporary regulation of 1918 stated that as many two-year-long primary education teacher colleges had to be established as needed (letter of finnish senate, church and educational administration 25 may 1918). this was the incentive to the development of the finnish primary education and to organizing necessary teacher education. primary education was organized within elementary education: the lower elementary education comprised grades 1–2 (pupils aged 7–8 years) (hyyrö, 2006; melin, 1980; paksuniemi, 2009.) after a wide debate, compulsory education took effect on the 15th of april 1921. according to the law on compulsory education, finnish children had to go to school during the year in which they reach the age of seven until the spring semester of the year in which they reach the age of 13 (elementary education laws and decrees 1932, 45 § 3; law on compulsory education, 101/1921). the law on compulsory education did not, however, treat school-age children equally. conditions were greatly different in cities and in the countryside. in cities, children commonly went to school and the law on compulsory education was considered only formal. economic circumstances, especially during the great depression at the beginning of the 1930s, hindered municipalities from establishing new schools. consequently, compulsory education was fully achieved after the second world war (law on compulsory education, 101/1921; jauhiainen, 1993). the law on compulsory education did not by any means solve problems of education but rather increased them because children of a certain age had to go to school. as teachers could not use physical punishments any longer and pupils could not be expelled from school since at the roots of finnish elementary education / uusiautti, paksuniemi & määttä 177 education was compulsory, the forms of punishment were considered inefficient. especially, forbidding physical punishments was problematic because it could not be replaced with any other form of punishment. it was also found conflicting because the forms of punishment that were previously in use in elementary school were already regarded as relatively few in number. in addition, the law forbidding physical punishments took effect in a bed time because the era was more restless than before. in order to maintain peace in classrooms, physical punishments were allowed in exceptional cases because elementary school was otherwise powerless with ill-bred pupils. actually, the school system did a bill for restoring physical punishments at school but the national board of education never accepted it but instead it encouraged teachers to negotiate with parents about punishments that could be given at home for bad behavior at school (salmela, 1933). according to the law on compulsory education, teachers had the right to punish pupils who do not follow the orders and regulations at school. the same concerned lazy and remiss pupils, who did not, regardless of warning, change their behaviors (laurila, 1926). at the initial phase of elementary education, there were not any clear didactic guidelines. at the beginning of the 20th century, education was unified due to herbartism and reform pedagogy. while herbartism emphasized intellectual, teacher-led education, reform pedagogy favored pupils’ freedom, teachers’ role as guides, and working school ideology. the thought of practicality originated in the societal change and its needs. on the other hand, teacher-led teaching made pupils passive and cause restlessness and concentration problems. finnish teachers worked using the forms of punishing provided by the decree on elementary education. the decree was complemented by additional directions that were to prevent malpractice of punishments. the most crucial change in punishments took place when physical punishments were forbidden. the change roused discussion both for and against. education was further developed so that pupils’ difficulties could be prevented better. lessons were planned to be as interesting as possible so that pupils would not have wanted to disturb teaching (valta, 2002; syväoja, 2000). on the other hand, carefully planned lessons did not leave room for movements, mimicry, or gesticulation, not to mention free socializing with peers (syväoja, 2000). broady (1986) saw discipline as school as an unconscious method to teach pupils who to become patient and fit citizens for work and life in the society. features that formed teacher education teacher education was under constant development and by the end of the 1860s, primary education teacher colleges operated in few places and the number increased little by little at the beginning of the 20th century. still, not all teachers were educated in teacher training colleges. in the 1910s, the state started to take care of elementary school teacher training and in 1917 (halila, 1950; hyyrö, 2006; melin, 1980; kuikka, 1978; nurmi, 1989). herbartianism influenced the finnish elementary school system at the beginning of the 20th century. it was considered a didactic deliverance after the non-uniform didactics of the end of the 19th century (lahdes, 1961; paksuniemi, 2009). the herbartian trend emphasized decency, hobbyism, and concentration (halila, 1949c). being based on the teacher-led ideology, teachers were responsible for the contents studied in the classrooms (hyyrö, 2006; iisalo, 1989; paksuniemi, 2009). therefore, teachers had a central position in classrooms and it was accentuated, for example, by situating the teacher’s desk on a podium (koskenniemi, 1944; paksuniemi, 2009). mikael soininen became interested in herbartianism already in the 1890s and became known as the finnish protagonist of herbartianism. soininen was a finnish educationalist and politician who specialized in school issues fighting the case of compulsory education and worked as the chief director of the national board of education (1917-1924). in line with the fundamental ideology of herbartianism, soininen thought that teaching should international electronic journal of elementary education vol.5, issue 2, 171-184, 2013 178 be educative, in other words, it should pay a special attention to decency. teachers’ activity was seen as the disadvantage of the trend because pupils were regarded as passive receivers of information (iisalo, 1989). already in the 1910s, reform pedagogical ideas took root in finland and many elementary school teachers tried them in practice in the 1920s and 1930s (somerkivi, 1977). the new school had many names: it was called for example the school of life, experiential school, and even experiential pedagogy. the new school highlighted the demands for patriotism and socialization. in addition, the importance of sport and environment was emphasized. furthermore, each classroom was seen as a miniature society (peltonen, 1989). the new school had emphasis on active pupils who could follow their nature. teachers had to recognize their individualism and societal demands. in teaching, teachers were supposed to highlight the life outside school and its demands. the new school was considered a working school that pursued finding pupils’ own resources. studying had to be true-to-life and pupils’ working autonomous. they could, for example, set their own goals and find suitable means and plan their action by themselves. in the new school, information had to useful, self-sought. all action highlighted independent initiative that was based on children’s natural action. teaching was expected to emphasize pupils’ socialization, autonomy, and the needs of the society. the purpose of increasing pupils’ freedom and diversifying teaching methods was to diminish disturbances of conduct (hyyrö, 2006; paksuniemi, 2009; valta, 2002). the teacher was regarded as the most important motivator who maintains working peace in the classroom. for the success of teaching, teacher education, teaching facilities, classroom sizes, and curricula were considered the most important. teachers themselves acted as embodiments of discipline: they had to practice strict and coherent self-discipline in their behavior and inner thoughts. teachers had to be able to maintain peace through their own personality, calm behavior, and first and foremost, fair attitude. the more demanding a teacher was to himself or herself, the better teacher he or she was considered. furthermore, much was dependent of how well teachers knew their pupils in order to be able to read their thoughts. teachers had to have a many-sided picture their pupils’ characters, aptitudes, and habits. thus, discipline was adjusted to individual pupils’ needs. to do this, teachers had to observe pupils during breaks and excursions in addition to lessons. these expectations were taught at teacher training colleges of finland and they laid the foundation of the pedagogy applied in finnish teacher training colleges (halila, 1949b; paksuniemi, 2009; tuomaala, 2004; valta, 2009). elementary school teachers had to work diligently for children and earn parents’ respect and trust. teachers had great responsibility as they had to show through their work that they are genuinely working for children’s good. teachers also had to be up to date and aware of happenings in the surrounding environment (mäntyoja, 1951). teachers had to visit homes— and not just when problems occurred. parents were invited to school to discuss issues concerning their children’s studies. in order to enhance cooperation between the school and home, various parties were arranged with the emphasis on parents. in addition to these occasions, parents meetings were organized where participants discussed, socialized, and gave relevant presentations. the school could arrange theme days, for example that focused on homes. likewise, the school arranged mother’s, father’s, and parents’ day celebrations (laurila, 1926). successful cooperation was based on mutual trust, respect, and understanding. the school and home had to have a common goal: to raise and educate children. parents had to be convinced that teachers at school acted as parents’ substitutes (etelälahti, 1920). at the roots of finnish elementary education / uusiautti, paksuniemi & määttä 179 discussion the review provided in this article showed that at the time the finnish nationhood was constructed, the education responsibility was moved from parents to the state. moreover, education was strongly ideologically oriented: all teaching and education aimed at forming a moral character and the christian-decent civilization ideal. the very same educational trend lied behind the curriculum for elementary education teacher training and for elementary schools. welfare was not just considered provision by the state but also what people can do for each other, the goal was active, for-the-nation-style citizenship in all education (see also milligan, morretti, & oreopoulos, 2004). although this active societal orientation is a sum of many factors and the school is not the only source of civic skills and active attitudes, teaching was and still is considered to have strong effect on behavior as citizens (campbell, 2008)—but the objectives of citizenship education change along the change in the society (eränpalo, 2012). furthermore, the history shows the importance of finding a suitable educational ideology was crucial in forming the unified educational system. in finland, the science of education as a key component of teacher education was and is a sustained and consistent tradition (jakku-sihvonen et al., 2012; kansanen, 1990). rearing practives have changed, but still today, parents have the main responsibility for rearing their children (määttä & uusiautti, 2012). according to kemppainen’s (2001) study on finnish child-rearing across three generations, finnish parents use fewer and milder punishments today than they did before. instead of punishing, children are rewarded, guided, and advised more often. principles in raising children have moved from authoritarian to more guiding practices where children’s opinions are taken into account. although strict methods have been replaced by more constructive options, finnish parents still want to hang on to limits and rules. indeed, the review also showed how the responsibility for raising children moved from parents into a more professionalized form where the responsibility in raising children was shared between parents and educators. the history shows that even rather strict punishments were used in raising children. the elementary education was at its infancy but the ideology behind the way children were raised and punished at school was in line with the national endeavors of becoming an independent national state—on the other hand, the raising methods and forms of punishments did not differ from the ones used at homes. . . . satu uusiautti, ph.d., works as a specialist at university of lapland, finland, and as a post-doctoral researcher in the research project love-based leadership – an interdisciplinary approach (http://www.ulapland.fi/lbleadership). her personal research interests are in positive psychology and human strengths, happiness, success, and well-being in life in general but especially in education and teacherhood, and in diverse educational contexts. her latest publications include many faces of love (authored by. k. määttä and s. uusiautti, sense publishers, 2013), early child care and education in finland (edited by k. määttä and s. uusiautti, routledge, 2013), and sámi education (authored by p. keskitalo, k. määttä and s. uusiautti, peter lang, 2013). merja paksuniemi, ph.d., works as a lecturer at university of lapland, finland, and does postdoctoral research on war time children in northern finland. her personal interests and research topics are school history, teacher education and its historical background, teacher image, teacherhood and international electronic journal of elementary education vol.5, issue 2, 171-184, 2013 180 teacher profession, war time in northern finland during 1939–1945, and childhood in the shadow of war. kaarina määttä, ph.d., is the professor of educational psychology at the faculty of education, university of lapland, and deputy vice-chancellor at the university of lapland, finland. she supervised merja paksuniemi’s and satu uusiautti’s doctoral thesis. during her career, she has supervised 47 doctoral theses, written hundreds of articles and dozens of text books, especially about love, human strengths, early childhood education and student guidance and teacher training and teacherhood. her latest publications include obsessed with the doctoral theses. the supervision and support in the phases of dissertation process (edited by k. määttä, sense publishers, 2012), many faces of love (authored by. k. määttä and s. uusiautti, sense publishers, 2013), early child care and education in finland (edited by k. määttä and s. uusiautti, routledge, 2013), and sámi education (authored by p. keskitalo, k. määttä and s. uusiautti, peter lang, 2013). references bauch, j. p. (1994). categories of parent involvement. the school community journal, 4(1), 53-60. bimbi, f. (1992). parenthood in italy: asymmetric relationships and family affection. in u. björnberg (ed.), european parents in the 1990s. contradictions and comparisons (pp. 141-154). london: transaction. björnberg, u. (1992). parenting in transition: an introduction and summary. in u. björnberg (ed.), european parents in the 1990s. contradictions and comparisons (pp. 1-44). london: transaction. broady, d. (1986). piilo-opetussuunnitelma. mihin koulussa opitaan [hidden curriculum. what is taught in finnish schools] (transl. p. kämäräinen). helsinki, vastapaino. campbell, d. (2008). voice in the classroom: how an open classroom climate fosters political engagement among adolescents. political behavior, 30(4), 437–454. cunningham, h. (1995). children and childhood in western society since 1500. london and new york: longman. cutting, e. (1998). supporting parents. first year report identifying the support needs of parent and families in four communities in scotland. edinburgh: save the children scotland. cygnaeus, u. (1903). uno cygnaeuksen kasvatusopilllisia periaatteita [uno cygnaueys’s educational principles]. kansakoulun lehti [elementary school journal], 123–128. cygnaeus, u. (1910). kirjoitukset suomen kansakoulun perustamisesta ja järjestämisestä [writings about the establisment and organization of the finnish elementary school]. helsinki: kansanvalistusseura. eränpalo, t. (2012). onko kuva suomalaisnuorten yhteiskunnallisesta passiivisuudesta harhaa? [is the image of finnish youngsters’ civic passivity fallacy?] kasvatus & aika [education & time], 6(1) 2012, 23–38. etelälahti, e. (1920). kansakouluja sen työväen asema suomessa [the position of elementary education and its work force in finland]. helsinki: edistys. greenwood, g. e., & hickman, c. w. (1991). research and practice in parent involvement: implications for teacher education. elementary school journal, 91(3), 279-288. haavio, m. (1941). kansakoulun uskonnonopetuksen siveellinen kasvatustehtävä nykyisen tutkimuksen valossa [the moral educational task of religion education in elementary schools in the light of current research]. jyväskylä: gummerus. halila, a. (1949a). suomen kansakoululaitoksen historia i [the history of the finnish elementary school system i]. turku: uuden auran osakeyhtiön kirjapaino. at the roots of finnish elementary education / uusiautti, paksuniemi & määttä 181 halila, a. (1949b). suomen kansakoululaitoksen historia ii [the history of the finnish elementary school system ii]. turku: uuden auran osakeyhtiön kirjapaino. halila, a. (1949c). suomen kansakoululaitoksen historia iii [the history of the finnish elementary school system iii]. turku: uuden auran osakeyhtiön kirjapaino. halila, a. (1950). suomen koululaitoksen historia iv [the history of the finnish elementary school system iv]. turku: uuden auran osakeyhtiön kirjapaino. heporauta, f. a. (1945). suomen kansakoululaitoksen historia [the history of the finnish elementary school system] (2nd ed.). helsinki: otava. hiltunen, m. (1983). kempeleen kansakoululaitoksen historia 1882–1982 [the history of the kempele elementary school 1882-1982]. oulu: oulun kirjateollisuus oy. howard, t. c. (2003). culturally relevant pedagogy: ingredients for critical teacher reflection. theory into practice, 42(3), 195-202. doi:10.1207/s15430421tip4203_5 hyyrö, t. (2006). alakansakoulujen valmistuksen kehitys suomessa vuosina 1866-1939 [the development of elementary school system in finland 1866-1939]. (phd diss., acta universitatis tamperiensis 1147, university of tampere, tampere, finland). iisalo, t. (1968). suomen koulujen uskonnonopetus 1843-1883 [religion education in finnish schools 18431883]. uusikaupunki: suomen kirkkohistoriallinen seura. iisalo, t. (1979). the science of education in finland 1828–1918. helsinki: societas scientiarum fennica. iisalo, t. (1989). kouluopetuksen vaiheita keskiajan katedraalikoulusta nykyisiin kouluihin [the phases of edication from medieval catherdral schools to modern schools]. keuruu: otava. isosaari, j. (1961). jyväskylän seminaarin kasvatusja opetusopin opetus vuosina 1865–1901 [education of pedagogy and didactics in jyväskylä teacher training college in 1865-1901]. jyväskylä: jyväskylän kasvatusopillisen korkeakoulun julkaisuja xxv. jakku-sihvonen, r., & niemi, h. (eds.) (2007) education as a societal contributor. reflections by finnish educationalists. frankfurt a.m.: peter lang. jakku-sihvonen, r., tissari, v., ots, a. & uusiautti, s. (2012). teacher education curricula after the bologna process – a comparative analysis of written curricula in finland and estonia. scandinavian journal of educational research, 56(3), 261-275. doi:10.1080/00313831.2011.581687 jauhiainen, a. (1993). koulu, oppilaiden huolto ja hyvinvointivaltio. suomen oppivelvollisuuskoulun oppilashuollon ja sen asiantuntijajärjestelmien muotoutuminen 1800-luvun lopulta 1900-luvulle [school, pupil welfare, and the welfare state. the development of pupil welfare in finnish compulsory edication and its professional system from the end of the 19th century to the beginning of the 20th century]. (phd diss. university of turku, turku, finland). kansanen, p. (1990). education as a discipline in finland. scandinavian journal of educational research, 34, 271–284. kemppainen, j. (2001). kasvatusmuistot rakentavat vanhemmuutta [child-rearing in three generations]. (phd thesis, university of jyväskylä, jyväskylä, finland). koskenniemi, m. (1944). kansakoulun opetusoppi [didactics of elementary education]. helsinki: otava. kuikka, m. t. (1978). kansakoulunopettajankoulutussuunnitelmien kehitys suomessa vuosina 1917-1923 [the change in elementary school teacher education curricula in finland in 1917-1923].joensuu: university of joensuu. kuikka, m. t. (2003). koulun merkitys suomalaisen kansallisvaltion ja kansalaisyhteiskunnan kehityksessä [the meaning of school in the development of the finnish national state and civic society]. in j. rantala (ed.), koulu ja kansalaisyhteiskunta historiallisessa perspektiivissä [the school and civic society from the historical perspective] (pp. 31-44). helsinki: historiallisyhteiskuntatiedollisen kasvatuksen tutkimusja kehittämiskeskus. international electronic journal of elementary education vol.5, issue 2, 171-184, 2013 182 lahdes, e. (1961). uuden koulun vaikutus suomen kansakouluun [the influence of the new school to the finnish elementary education]. keuruu: otava. laurila, j. (1926). kansakouluasian käsikirja [the manual of elementary education]. helsinki: otava. koski, l. (1999). hyvä tyttö ja hyvä poika. ihanteelliset yksilöt aapisten moraalisissa kertomuksissa [good girl and good boy. ideal individuals in the moral stories of abc books]. in t. tolonen (ed.), suomalainen koulu ja kulttuuri [the finnish school and culture] (pp. 21-50). tampere: osuuskunta vastapaino. lahikainen, a. r., & strandell, h. (1987). lapsen kasvuehdot suomessa [the conditions of children’s growing in finland]. helsinki: painokaari. lehmusto, h. (1951). kasvatusopin käsikirja [the manual of education]. kuopio: kirjapaino osakeyhtiö savo. lipponen, p. (2003). kansakouluaate saapuu kuopion maalaiskuntaan [the ideology of elementary education arrives in the countryside municipality of kuopio]. in j. rantala (ed.), koulu ja kansalaisyhteiskunta historiallisessa perspektiivissä [the school and civic society from the historical perspective] (pp. 62-77). helsinki: historiallis-yhteiskuntatiedollisen kasvatuksen tutkimusja kehittämiskeskus. melin, v. (1980). alkuopetus suomen maaseudulla ennen oppivelvollisuuslakia 1866-1921. ii alkuopetuksen liittäminen kunnalliseen oppivelvollisuuskouluun 1906-1921 [elementary education in the countryside of finland before the law on compulsory education 1866-1921 ii. integrating primary education within the municipal compulsory education 1906-1921]. (phd diss., acta universitatis tamperiensis. ser. a vol 117, university of tampere, tampere, finland). milligan, k. s., moretti, e., & oreopoulos, p. (2004). does education improve citizenship? evidence from the u.s. and the u.k. journal of public economics, 88(9-10), 1667-1695. männistö, j. (1994). sivistyksen kylvö. suomen kansakoululaitos johtavien puoluelehtien mielenkiinnon kohteena vuosina 1918–1939 [the sowing of education. the finnish elementary school system as the point of interest of the leading political journals in 1918-1939]. (phd. diss., university of tampere, tampere, finland). mäntyoja, a. (1951). kansakoululainsäädäntö [elementary education legislation]. helsinki: otava. määttä, k., & uusiautti, s. (2012). parental love –irreplaceable for children’s well-being. global journal of human social sciences, 12(10), 1-8. naumanen, p., & rinne, r. (2008). eurooppalainen koulutuspolitiikka ja suomen linjaukset [european educational policy and the finnish outlines]. in r. rinne, l. jõgi, r. leppänen, m. korppas, & k. klemelä (eds.), suomalainen ja virolainen koulutus ja eu:n uusi koulutuspolitiikka [the finnish and estonian education and the new educational policy of the european union] (pp. 327–367). turku: university of turku, department of education. nurmi, v. (1981). maamme koulutusjärjestelmä [the finnish school system]. porvoo: wsoy. nurmi, v. (1983). kasvatuksen traditio [the tradition of education]. porvoo: wsoy. nurmi, v. (1989). kansakoulusta peruskouluun [from elementary education to compulsory education]. juva: wsoy. oinasmies, o. (1945). kansakoulu suurennuslasin alla [elementary school under the micoscope]. lahti: tammi. ojakangas, m. (2003). j. v. snellman ja kansallinen kasvatus [j. v. snellman and national education]. in j. rantala (ed.), koulu ja kansalaisyhteiskunta historiallisessa perspektiivissä [the school and civic society from the historical perspective] (pp. 47-61). helsinki: historiallis-yhteiskuntatiedollisen kasvatuksen tutkimusja kehittämiskeskus. at the roots of finnish elementary education / uusiautti, paksuniemi & määttä 183 paksuniemi, m. (2009). tornion alakansakoulunopettajaseminaarin opettajakuva lukuvuosina 1921-1945 rajautuen oppilasvalintoihin, oppikirjoihin ja oheistoimintaan the teacher image in the lower primary school teachers’ college of tornio in 1921-1945]. (phd diss., acta universitatis lapponiensis 161, university of lapland, finland). paksuniemi, m., & määttä, k. (2011a). at the roots of the pedagogy of finnish teacher training – practicum as a core of teacher training in teacher college in tornio in the 1920s. international journal of educational administration and policy studies, 3(8), 121-128. paksuniemi, m., & määttä, k. (2011b). how were the students turned into decent teachers by the freetime activities at the teacher college of tornio in 1921-1945. elixir psychology, 39, 4782-4786. paksuniemi, m., & määttä, k. (2011c). the most decent girls make the best responsible teacher teacher selection for teacher education in northern finland in 1921-1945. prime research on education, 1(1), 10-17. paksuniemi, m., uusiautti, s., & määttä, k. (2012). teetotalism as the core of education at the teacher training college of tornio. history of education and children´s literature, 7(1), 389-411. paksuniemi, m., uusiautti, s., & määttä, k. (2014). teacher education in finland during the war years 1939–1945. war & society, 33(1), in press. peltonen, m. (1989). jyväskylän kansakoulun historia [the history of jyväskylä elementary school]. jyväskylä: gummerus. päivänsalo, p. (1971). kasvatuksen tutkimuksen historia suomessa vuoteen 1970 [the history of educational research in finland until 1970]. helsinki: ylioppilastuki. salo, a. (1934). lapsuuden kultamailla. alakansakoulun ja kodin lukukirja ii [the golden childhood. the reading book for elementary school and home ii]. helsinki: otava. salmela, a. (1933). kansakoulun hallintoa. selostuksia ja ohjeita kunnallismiehille, johtokunnan jäsenille, opettajille ja kaikille kansakoulun ystäville [the administration of elementary education. reports and directions to municipal administrators, board members, teachers, and all friends of elementary school]. porvoo: wsoy. simola, h. (2005). the finnish miracle of pisa: historical and sociological remarks on teaching and teacher education. comparative education, 41(4), 455-470. doi:10.1080/03050060500317810 soikkanen, h. (1966). kunnallinen itsehallinto kansanvallan perusta. maalaiskuntien itsehallinnon historia [the municipal self-governance as the basis of democracy]. helsinki: maalaiskuntien liitto. syväoja, h. (2000). kansakoulu – suomalaisten kasvattaja. perussivistystä koko kansalle 1866-1977 [elementary school – the educator of finns. basic education for the whole nation 1866-1977]. juva: ws bookwell. somerkivi, u. (1977). helsingin kansakoulun historia [the history of helsinki elementary schooling]. helsinki: the city of helsinki. tuomaala, s. (2004). työtätekevistä käsistä puhtaiksi ja kirjoittaviksi. suomalaisen oppivelvollisuuskoulun ja maalaisten kohtaaminen 1921-1939 [from working hands to clean and writing hands. the confrontation between the finnish compulsory education and peasantry]. helsinki: finnish literature society. tuunainen, k., & nevala, a. (1986). erityiskasvatus koko ikäluokan kasvatuksen osana. erityiskasvatuksen historiallisen kehityksen piirteitä suomessa 1860-luvulta peruskoulun tuloon [special education as a part of the educatio of the whole generation. the trends of the historical development of special education in finland from the 1860s to the establishment of comprehensive education]. joensuu: university of joensuu. valta, j. (2002). ongelmaoppilaat oulun kansakoulussa vuosina 1874-1974 [problem students at the oulu elementary school 1874-1974]. (phd diss. university of oulu, oulu, finland.) international electronic journal of elementary education vol.5, issue 2, 171-184, 2013 184 werkko, k. (1903). kansansivistysoloista ruotsissa, norjassa, tanskassa ja pohjois-saksassa sekä alemmasta kansanopetuksesta suomessa ja pääsytutkinnoista maamme kansakoulunopettajaseminaareihin [on civilization in sweden, norway, denmark, and northern germany and lower elementary education in finland and entrance tests at teacher training colleges in finland]. helsinki: weilin. young, i. m. (1995). mothers, citizenship, and citizenship: a critique of pure family values. ethics, 105(3), 535-556 laws, decrees, and letters: laws and decrees on elementary education 1932. decree on elementary education 1866, the merciful decree of imperial majesty 11 may 1866. the merciful decree of imperial majesty, 27 jun 1889. law on compulsory education 15 apr 1921. law on physical punishments 1914, decree 6 jun 1914. decree on school districts, 1898. letter of finnish senate, church and educational administration 25 may 1918. international electronic journal of elementary education, december 2016, 9(2), 337-348. issn:1307-9298 copyright © iejee www.iejee.com establishing a common vocabulary of key concepts for the effective implementation of applied behavior analysis traci m. cihon a * joseph h. cihon b guy m. bedient ** a university of north texas. usa b autism partnership foundation & endicott college, usa. received: august, 2016/ revised: october, 2016 / accepted: november, 2016 abstract the technical language of behavior analysis is arguably necessary to share ideas and research with precision among each other. however, it can hinder effective implementation of behavior analytic techniques when it prevents clear communication between the supervising behavior analyst and behavior technicians. the present paper provides a case example of the development of a shared vocabulary, using plain english when possible, among supervisors and supervisees at a large public school district in which behavior analytic services were provided for children diagnosed with autism spectrum disorders. a list of terms and definitions are provided as well as suggestions on how to develop shared vocabularies within the readers’ own service provision context. keywords: autism spectrum disorder, behavior analysis, jargon, verbal behavior introduction for several years we have observed an increase in the prevalence/incidence of autism spectrum disorders (asds) worldwide. even though there are many types of interventions, therapies, and services that are available to individuals with asds and their families, behavior analysis as applied to autism intervention remains one of the only empirically supported treatments (smith, 2012; smith & iadarola, 2015). as the incidence of asds increased so did the demand for behavior analytic practitioners (smith, 2007). however, behavior analysis has been and continues to be a relatively small discipline. recently, several authors have commented on the need for more and better quality training for behavior analysts (e.g., hughes & shook, 2007; leaf et al., 2016; leaf, * corresponding author: traci m. cihon, department of behavior analysis, the university of north texas, 1155 union circle #310919, denton, tx 76203 email: traci.cihon@unt.edu ** authors’ note: guy michael bedient (1956-2007) was instrumental in the development of the list of the key terms/concepts and definitions included in this manuscript. http://www.iejee.com/ international electronic journal of elementary education vol.9, issue 2, 337-348, december, 2016 338 mceachin, & taubman, 2008; mcgee, & morrier, 2005). however, we have a shortage of behavior analysts at the doctoral level, creating challenges to prepare a sufficient number of behavior analysts at the master’s and bachelor’s levels which results in a further shortage of well prepared and sufficiently supervised behavior technicians. to illustrate the magnitude of the training, supervision, and practice needs let us look at some numbers in the state of texas. in 2014, it is estimated that there were 399,915 individuals diagnosed with asds and 130,316 were children below the age of 22 (texas council on autism and pervasive developmental disorders, 2014). currently, there are only two, association for behavior analysis international (abai) accredited masters training programs (no accredited doctoral training programs and no abai accredited bachelors training programs; association for behavior analysis international, n.d.) in texas. there are 11 behavior analysis certification board (bacb) approved course sequences (bacb, n.d.) in texas. these university-based training programs, abai accredited degree programs or approved course sequences in texas, employ approximately 30 doctoral level behavior analysts (note that not all of these academics focus on training students in behavior analysis and autism treatment, many focus their work on basic principles with nonhumans, education more broadly, organizational behavior management, or behavior and social issues). estimating a caseload of 10 children diagnosed with asds for each master’s level behavior analyst, in order to meet the current need for each child under the age of 22 diagnosed with an asd, the state of texas alone would need over 13,000 full-time, supervising behavior analysts dedicated to service delivery. there are currently 1207 board certified behavior analysts (bcbas; bacb, n.d.) residing in the state of texas. to produce the number of full-time, supervising behavior analysts needed in texas for service delivery alone, each of the 30 faculty members (including those with other specialization areas) would need to train and graduate nearly 400 students (roughly 13 students per faculty member) who chose to focus on autism intervention this year. the challenge to prepare a sufficient number of behavior analytic practitioners, supervisors, and faculty members is even greater outside of the united states (keenan et al., 2015). given the insufficient number of behavior analysts to meet the current service delivery needs for individuals diagnosed with an asd, practicing behavior analysts are often hindered by high caseloads. for example, in one public school system, the master level behavior analysts have upwards of 70 children diagnosed with an asd on their caseload. to put this in perspective, lovaas (1996) suggested 1-2 hours per week of supervision for each case on the supervisor’s caseload (with 70 cases, that requires 70-140 hours of supervision each week). more recently, the bacb (2014) recommends that bcbas, supervising comprehensive intervention programs, not carry caseloads of more than 6 to 12 individuals with an asd (without the support of an assistant behavior analyst) or 12 to 16 individuals with an asd (with the support of an assistant behavior analyst). nonetheless, the contingencies supporting higher caseloads (including both the supervisees and individuals with an asd) are often demanded by agencies (e.g., public school systems who cannot control enrollment but do not have sufficient funding for personnel), funding sources (e.g., insurance companies reimburse only a certain number of service hours so clinic owners/managers increase caseloads to meet overhead costs), etc. in spite of this, many service providers still cannot meet the demand for services and we need more, well-trained behavior analysts to increase the availability of empirically supported treatment globally. the confluence of the shortages of behavior analysts and the increasing need of quality persons to deliver behavior analytic services to individuals diagnosed with asds are also confounded by the “uniqueness” of the behavior analytic worldview and terminology creating common vocabulary / cihon, cihon & bedient 339 (hineline, 1980; jarmolowicz et al., 2008; kazdin & cole, 1981; michael, 1975; rolider & axelrod, 2005; rolider, axelrod, & van houten, 2009). supervising behavior analysts are often tasked with teams of sometimes upwards of 50 behavior technicians at a time to implement behavior analytic principles and procedures with individuals diagnosed with asds (c.f., sellers, alai-rosales, & macdonald, 2016 for a discussion of supervisory volume considerations). often, these supervising behavior analysts find themselves directing behavior technicians to do the “opposite” of what comes naturally. for example, the supervising behavior analyst might ask the behavior technician to implement planned ignoring with a child who is saying provocative things. supervising behavior analysts also frequently have to ask behavior technicians to engage in very complex sets of behaviors in the context of interlocking behavioral contingences (ibcs) (glenn, 1988) that go beyond the simple instruction of, “if the child does this, do this.” it requires an advanced, skilled set of analytic skills to teach a child to say a word when his or her current repertoire consists of pulling an adult toward the desired item. to a doctoral or even masters and/or bachelors level behavior analyst, the statement – “you need to shape the vocalization” can be sufficient. but, to a behavior technician with little to no formal training in behavior analysis, that instruction will unlikely evoke skillful behavior that will lead to the child emitting a vocalization when they want something. the need for plain english technical jargon is arguably necessary for many disciplines to communicate among themselves but can present a problem when communicating to the layperson. to overcome this challenge, some have suggested that we translate our technical jargon into plain english (e.g., lindsley, 1991). when the discipline expert must communicate something that requires action or doing something in response to a layperson, this barrier to effective communication can become particularly problematic. moreover, being able to hear an instruction and then implement the associated behavior(s) is frequently the repertoire that is necessary in behavior analytic interventions. the behavior analyst must communicate to someone without the same history and training how to do something that requires a particular level of precision and fidelity of implementation. this person, who must behave as the behavior analyst instructs, is responsible for thousands upon thousands of interactions with a child diagnosed with an asd. the quality of those interactions determines what the child will learn and how quickly the child will learn. however, behavior analysts talk funny. the ability of the behavior technician to implement, with integrity, what the behavior analyst is asking depends on whether or not the two persons have a shared understanding of the terms used. consider what a behavior technician’s response might be hearing the following on their first day of work: supervisor: “ok, i can’t stay long but i just want you to play. be sure to follow the student’s lead. if you see any challenging behaviors, you are doing something wrong. you want him to be running to you and not away from you. don’t worry about data collection today; just hang out with him; comment on what he does. see if you can get him to run to you, rather than away from you. the goal is for you to keep him wanting more. once you can do that, i will try to get out here to show you some of his programs. ok?” creating a common language between behavior analysts and supervisees – verbal communities (skinner, 1957) – can make staff training “easier.” to do this we can provide a definition for each concept, process, procedure, etc. the definitions will be composed of critical features (cf., tiemann & markle, 1983). once critical features are provided in a definition, examples and non-examples can be made explicit. this helps ensure a complete understanding of the concepts and the corresponding behavioral repertoires they describe. international electronic journal of elementary education vol.9, issue 2, 337-348, december, 2016 340 consider, for example, how the term “behavior” is employed across a variety of educational environments. for many, the term behavior equates to undesirable behavior. many times you hear someone say something like, “he had a lot of behaviors today” – the implications are that the he engaged in many instances of challenging behaviors. however, in behavior analysis, the definition of behavior also includes desirable behaviors. we can make the definition of behavior commonly employed in behavior analysis with three critical features: behavior: anything an (1) organism (2) does that can (3) be counted. we will be able to provide examples that allow for a common understanding and language. example: john stands up. in examining this example, we find that all three critical features are present. john is an organism (critical feature 1); he stood up (satisfying the action component of critical feature 2); and we can count the number of times john stands up (satisfying the third and final critical feature). on the other hand, if we exclude one of the critical features as in the following example, we find we can adhere to the critical features method for allowing us to maintain a common language. non-example: john feels melancholy. in this non-example we find john (presumably an organism), qualifying the first critical feature, but then we begin to run into trouble. can “feels” be clearly conveyed as an action (does not satisfy the criteria for critical feature 2); and further, is it possible to quantify “melancholy” (does not qualify critical feature 3)? a simple rearrangement of the example allows for a critical features-based example to meet the requirements of a common language. consider for example the following statement. example: john makes depressive statements. or, john saying, “i can’t do anything right” or, “no one likes me” or, “i’ve got the blues so bad i just want to stay in bed all day.” these three statements lend themselves to a common language because we can agree upon the statements due to the establishment of the critical features. each one of these three contains the following: they were all emitted by an organism (john), they all contain action (he said these statements), and we can count what a person says. behavior technician training can be made more efficient through the creation of a set of key terms, concepts, and principles with corresponding definitions that detail the critical features of each term, concept, and principle that behavior technicians need to know to be effective service providers. frequently, these terms are quickly established in the verbal community of the school district, clinic, or in-home service providing agency and can make communication among employees and supervisors much more efficient. toward a common vocabulary the authors of this manuscript developed a list of 50 key terms and definitions that were used during staff training in a large public school district in the midwest region of the united states. the complete list of key terms and definitions can be found in table 1. the key terms that were selected for inclusion were created by considering what particular behaviors the behavior technicians could learn quickly, what behaviors they could learn with little initial training, and what behaviors would create the biggest, positive impact on their initial interactions with their students. creating common vocabulary / cihon, cihon & bedient 341 table 1. key terms and definitions for effective implementation of applied behavior analysis with children with autism (*)definitions modified after the third author was deceased behavior anything an organism does that can be counted interspersal mixing easy and hard instructions meaningless attending no learning is taking place function on the fly moment to moment analysis of problem behavior instruct with meaning therapist knows focus of outcome *parsimony always use the simplest, accurate explanation terminal reinforcer chosen reinforcer provided contingent upon set completion pace moving fast enough to keep a student on task complete set overkill therapist fails to read the student and makes them do the whole set wanting more leaving a student in a state of deprivation teacher directed therapy lack of shared control, joint attending, and student choice prompt monitoring continually monitoring minimal prompt effectiveness contingency if-then relation contingent access procedure in which reinforcers are delivered upon minimal response effort operants class of responses that are functionally distinct actively engaged student is working toward a meaningful end response effort amount of effort required to perform a task nag repeating the sd explanatory fiction real event explained by an unobservable cause commenting teacher vocal based on event noticed rule governed behavior responding based on statement of contingencies establishing operation motivating condition slide in/slide out watch/do technique provided by a senior therapist quit while you’re ahead stopping when student is performing best outside of the box it’s okay to play at the desk joint attending focus of task is shared by therapist and student silly slides interspersing unexpected, fun instructions multiple exemplars using varied stimuli to teach one concept one step ahead ability of the therapist to prepare future activities while the student is presently engaged direct instruction explicit teaching with examples and non-examples. shared control control of effort and reinforcement is shared by both student and therapist keep ‘em surprised student doesn’t know what is coming next prompt fading systematically decreasing the prompt level required stereotypy repetitive behavior maintained by automatic reinforcement following the student’s lead base tasks on student’s interests spontaneous behavior behavior that occurs without instructor provided cues circular reasoning using the event to explain the event short sets keep sets brief and positive what should you hear? student talking/teacher quiet on-task actively engaged in a meaningful task behavioral momentum presenting easy tasks right before a more difficult one fluency accuracy + speed variable effort programming using highly randomized, gradual increase of effort, motivation and reinforcement anticipation knowing what’s going to happen before it happens non-compliance failure of an instruction to evoke a response instruction pairing (slide) pairing an instruction with the behavior about to occur empirical observable or measurable in some way diminishing field removing stimuli based on correct responses precision teaching teaching that incorporates fluency and standard celeration charts learning channels modes of acquisition skill input and output international electronic journal of elementary education vol.9, issue 2, 337-348, december, 2016 342 we focused on creating a set key terms that might serve as a sort of set of behavioral cusps (rosales-ruiz & baer, 1997) for behavior technicians, supporting the development of the skills and behaviors that could produce the most impact in their practice until more rigorous training could be conducted. then, the authors considered what vocabulary they used most frequently when training. we asked ourselves, “what technical jargon could be replaced with more plain english terms?” that could be highlighted when we saw areas in need of improvement or teaching interactions that were desirable. for the purposes of this manuscript, only a few key terms and definitions will be explained in detail in an effort to provide a general idea of how terms and definitions were selected and created to fit the context in which we were working. more information regarding any of the concepts can be obtained by contacting the first author, but readers are encouraged consider the context in which they are providing services and supervising behavior analysts and behavior technicians in the creation of their own key terms and definitions†. let us look at the key terms that were included in the example of the supervising behavior analyst helping the behavior technician get set up on their first day. it is important to note that in the environment in which the authors were working at the time it was not unusual for the masters level behavior analysts to be tasked with more than half of their caseload of children diagnosed with asds (often between 50 and 70) starting school in several different locations (upwards of 12). further, several different brand new staff members (40 or more) holding only a high school diploma or the equivalent were starting work for the first time with only 2 to 6 bachelors level behavior analysts available to help with staff training. nonetheless, the children were at school and the behavior technicians were going to be interacting with the children, supervisor present, training exhaustive, or not. in this example, the supervisor said several things that the behavior technician likely had no prior association with or at least not in a behavior analytic service delivery context: behavior, follow his lead, commenting, wanting more. these instructions likely had little impact on the behavior technician’s behavior. the concept of following the student’s lead has two critical features: following the student’s lead: (1) base tasks on (2) student’s interests the supervising behavior analyst wanted the behavior technician to take the student’s interest into account when arranging teaching opportunities. an example and nonexample can clarify what it was the supervising behavior analyst did and did not want the behavior technician’s behavior to look like. example: a child walks to a shelf in the classroom, takes down a star wars™ puzzle, and the therapist does the puzzle with the student, asking the student to count the pieces, to differentiate between piles of pieces that have more or less than the other, and to count how many pieces are needed to finish the puzzle at different points of completion. non-example: a child walks to the shelf in the classroom, takes down a star wars™ puzzle, and the therapist either says, “no we are not doing that right now, it’s time for math” and puts the puzzle back on the shelf. in this non-example, we see that only one critical feature of following the student’s lead is present, critical feature 1 base tasks. assuming this student is not interested in math but is interested in star wars™ or puzzles, the teacher violated the second critical feature of following the students’ lead – on student’s interests. the supervising behavior analyst wanted the behavior technician to understand that in order to work on math concepts the † readers are encouraged to work through tiemann and markle (1990) to learn how to identify critical and variable features of their own staff training concepts. creating common vocabulary / cihon, cihon & bedient 343 student did not need to do math worksheets sitting at a table, or only answer math related questions based on the behavior technician’s perspective as to how math should be taught. instead, the supervising behavior analyst wanted the behavior technician to embed instruction (critical feature 1 – base tasks) within activities that the student imitated or found interesting (critical feature 2 – student’s interest). consider the critical features for commenting commenting. a (1) teacher vocal (2) based on a stimulus event (3) to which the student is responding. example: the student points to yoda™ on the puzzle box. the teacher says, “yoda.” the student points (3 the student is responding) to yoda (2 a stimulus event) on a puzzle box. the teacher says, “yoda” (1 a teacher vocal). the example can be easily contrasted with a non-example: non-example: the teacher and the student are walking in the hallway. the student is running his hand along the wall and the teacher says, “the floor is dirty.” in this non-example, we find only one critical feature of the definition of commenting – a teacher vocal (1 – “the floor is dirty”). with close examination of this non-example, one may notice the presence of what may seem to be a stimulus event (2 – dirty floor). upon closer examination, however, it becomes clear that the second critical feature is not met due to the lack of critical feature 3 – to which the student is responding. in this nonexample, the stimulus event to which the student is responding is not the same stimulus event for which the teacher is providing the vocal. the student is attending to the sensation he is receiving from running his hand along the wall and the teacher is commenting on the cleanliness of the floor. in this case, the supervising behavior analyst was trying to communicate to the behavior technician to comment only when the student was also attending to the stimulus. the supervising behavior analyst did not want the behavior technician talking or asking the student to respond when the student was not emitting an observing response (violation of critical feature 3 – the student is responding). wanting more. (1) leaving a student in a (2) state of deprivation example: after the student puts the border or some pieces of the star wars™ puzzle together, the behavior technician asks the student to go to the math center, has the student complete one easy problem and then takes the student back to the finish the puzzle or to another preferred activity. non-example: after the student grabs and starts to complete the star wars™ puzzle, the behavior technician lets the student do the entire puzzle and then takes him to the math center. in this situation, the supervising behavior analyst wanted the behavior technician to arrange his/her session in a way that prevented the student from becoming quickly satiated in regard to preferred items/activities and to arrange quick teaching opportunities with which the student would be more likely to be successful. in the nonexample, the behavior technician fails to capitalize on naturally occurring motivating operations (mo; michael, 1988) within the activity (i.e., an uncompleted puzzle). the uncompleted puzzle in this example, exemplifies the critical features of leaving the student wanting more in the sense that the student would want to get back to the puzzle. the supervising behavior analyst did not want the behavior technician to lose opportunities to reinforce responding due to satiation, there is nothing “to want” if the puzzle is finished. international electronic journal of elementary education vol.9, issue 2, 337-348, december, 2016 344 creating a common vocabulary over the course of 3 years, over 200 behavior technicians, parents, supervisors, teachers, and related service providers were coached regarding their teaching interactions with children diagnosed with asds of many ages and functioning levels with this common vocabulary. in a setting in which the masters level behavior analyst (third author) carried a caseload of over 70 children diagnosed with asds and supervised at least 6 bachelors level behavior analysts (first author) who carried a caseload of 15-25 children with asds who each supervised upwards of 40 behavior technicians (second author) who worked with 1-6 children with asds each academic year, a common vocabulary that led to effective implementation and efficient staff training was critical. the first step was to teach the behavior technicians to relate the term with the definition. behavior technicians can be taught the terms and definitions that begin the process of developing a shared vocabulary through supervisors saying the term and giving the definition, examples, and non-examples or by using a behavior analytic teaching strategy derived from precision teaching (pt; lindsey, 1971) using safmeds (say all fast, minute every day, shuffled) (eshleman, 1985). we used both; in essence, we followed the behavior technician’s lead. if a behavior technician showed an interest in learning more about pt, safmeds, competing with oneself, graphing data, making data-based decisions, presenting at a standard celeration society chart share, or doing extra work outside his/her teaching sessions, the first and third authors helped the behavior technicians to learn the terms and definitions using safmeds and/or pt. for behavior technicians who did not, the supervising behavior analysts used a combination of stating the term and definition, describing what it was they wanted to the behavior technician to do, noting the critical features of the concept, and contrasting the description with a non-example in which one or more of the critical features were absent and then restating the term. each term/definition related to a tangible set of examples and non-examples to bridge the gap between paired associate learning and concept learning (cf., tiemann & markle, 1990) for the behavior technicians. a critical feature analysis was used as the foundation for generating a set of examples and close in and far out non-examples (tiemann & markle) for each concept. these examples and non-examples were then taught across a variety of input and output modes. a learning channel matrix (cf., binder, 1996; kubina & yurich, 2012) can be used to help supervising behavior analysts to determine the modes of input and output a skilled behavior technician needs to be able to perform. for example, the behavior technician needs to not only be able to hear the term and say (or think) the definition, s/he must also be able to hear the term and engage in the desired behavior, see the supervising behavior analyst engage in an example of the behavior and say (think) the term and then engage in the behavior. in essence, saying, saying and doing, and hearing and doing are different repertoires and each of these repertoires may need to be established separately (greer, 1991). therefore, supervisors must move staff training beyond simply establishing a common vocabulary. it is critical not to leave the behavior technician with only a repertoire of new verbal behavior (stating terms and definitions and examples and non-examples) but also to teach them to engage in behaviors related to the concept. we used a model-lead-testretest, direct instruction strategy (cf., adams & engelmann, 1996). in practice, this method of instruction involves modeling the desired behavior, assisting the behavior technician in engaging in the desired behavior, providing an opportunity for the behavior technician to engage in the desired behavior, and having the behavior technician engage in the desired behavior several more times, receiving frequent feedback on his/her creating common vocabulary / cihon, cihon & bedient 345 performance. this method is similar to those staff training methods described in other empirically evaluated staff training paradigms (e.g., weinkauf, zeug, anderson, & ala’irosales, 2010) in which the supervisor provides the exemplar and provides opportunities for the supervisee to practice the skill. even though this method was successful in our experiences, other training methods may also result in the desired outcomes. regardless of the training method selected, it is critical that shared vocabularies lead to shared responses to the developed vocabulary. conclusions behavior analysts have an uncommon technical vocabulary. this technical vocabulary can hinder effective implementation of behavior analytic techniques by supervisees. however, developing a common vocabulary, using plain english where possible (lindsley, 1991), can assist in the effective implementation of behavioral interventions for individuals diagnosed with asds. this common vocabulary should include the critical features that should come to control the behavior of the listener, or, in this case, the behavior technician. doing a careful analysis of each concept and its critical features has several benefits for the supervising behavior analyst. first, it creates an opportunity for the supervisor to have a quick checklist to examine if the behavior technician is behaving in the desired manner. for example, when the supervisor asks the behavior technician to follow the student’s lead, s/he can quickly check if the behavior technician is engaging in the appropriate response based on the critical features of the concept. second, and similar to the first example, instructions and feedback can be more readily applied by both those who are giving and receiving the information. moreover, when the critical features of each concept are shared by the supervisor and supervisee, instructions and feedback are more likely to have the desired effect on the listener (i.e., the supervisee). lastly, creating a shared vocabulary utilizing plain english (similar to table 1) creates a more welcoming environment for new staff and parents. walking into a place where everyone is speaking differently (e.g., with technical jargon) can be very overwhelming and even off-putting. using “softer” language, without losing the critical features and desired outcomes on the listener, can create a more welcoming, natural environment. despite the rationale for developing a common vocabulary among staff, it is imperative that it translates into action. simply agreeing on or providing the same definition of a word and teaching paired associates is insufficient. the shared vocabulary should also occasion shared topographies of behavior. for example, when a supervisor requests that a behavior technician “follow the student’s lead,” it should occasion a series of responses that are contextually appropriate and responsive to the student’s needs, not just an agreed upon definition. taking shared vocabularies into action allows the supervisor to conduct more efficient trainings and assessments of student and supervisee progress. ultimately, shared vocabularies that result in shared topographies of responding lead to more effective implementation of behavior analytic techniques and should be a goal of any successful treatment program. • • • references adams, g. l., & engelmann, s. (1996). research on direct instruction: 25 years beyond distar. seattle: educational achievement systems. international electronic journal of elementary education vol.9, issue 2, 337-348, december, 2016 346 association for behavior analysis international. (n.d.). accredited programs. retrieved august 28, 2016, from https://www.abainternational.org/accreditation/accredited-programs.aspx bacb (2014). applied behavior analysis treatment of autism spectrum disorder: practice guidelines for healthcare funders and managers. retrieved august 28, 2016, from http://bacb.com/wpcontent/uploads/2016/08/aba_guidelines_for_asd.pdf. bacb (n.d.). approved course sequences. retrieved august 28, 2016, from http://info.bacb.com/o.php?page=100358. binder, c. (1996). behavioral fluency: evolution of a new paradigm. the behavior analyst today, 19(2), 163–197. eshleman, j. w. (1985). improvement pictures with low celerations: an early foray into the use of safmeds, 6(3), 54–63. glenn, s. s. (1988). contingencies and metacontingencies: toward a synthesis of behavior analysis and cultural materialism. the behavior analyst today, 11(2), 161–179. greer, r. d. (1991). the teacher as strategic scientist: a solution to our educational crisis? behavior and social issues, 1(2), 25-41. doi: 10.5210/bsi.v1i2.165 hineline, p. (1980). the language of behavior analysis: its community, its functions, and its limitations. behaviorism, 8(1), 67–85. hughes, j. c., & shook, g. l. (2007). training and certification of behavior analysts in europe: past, present, and future challenges. european journal of behavior analysis, 8, 239-249. jarmolowicz, d. p., kahng, s. w., ingvarsson, e. t., goysovich, r., heggemeyer, r., & gregory, m. k. (2008). effects of conversational versus technical language on treatment preference and integrity. intellectual and developmental disabilities, 46(3), 190–199. doi: 10.1352/2008.46:190-199 kazdin, a. e., & cole, p. m. (1981). attitudes and labeling biases toward behavior modification: the effects of labels, content, and jargon. behavior therapy, 12(1), 56–68. doi:10.1016/s00057894(81)80106-2 keenan, m., dillenburger, k., röttgers, h. r., dounavi, k., jónsdóttir, s. l., moderato, p., et al. (2014). autism and aba: the gulf between north america and europe. review journal of autism and developmental disorders, 2(2), 167–183. doi: 10.1007/s40489-014-0045-2 kubina, r. m., & yurich, k. k. (2012). precision teaching book. lemont, pa: greatness achieved publishing company. leaf, j. b., leaf, r., mceachin, j., taubman, m., smith, t., harris, s. l., et al. (2016). concerns about the registered behavior technician™ in relation to effective autism intervention. behavior analysis in practice, 1–10. doi: 10.1007/s40617-016-0145-9 leaf, r. b., mceachin, j. j., & taubman, m. (2008). sense and nonsense in the behavioral treatment of autism: it has to be said. new york, ny: drl books. lindsley, o. r. (1971). from skinner to precision teaching: the child knows best. in j. b. jordan & l. s. robbins (eds.), let’s try doing something else kind of thing (pp. 1–11). arlington, va: the council for exceptional children. lindsley, o. r. (1991). from technical jargon to plain english for application. journal of applied behavior analysis, 24(3), 449–458. doi: 10.1901/jaba.1991.24-449 lovaas, o. i. (1987). behavioral treatment and normal educational and intellectual functioning in young autistic children. journal of consulting and clinical psychology, 55(1), 3–9. doi: 10.1037/0022-006x.55.1.3 lovaas, o. i. (1996). the ucla young autism model of service delivery. in c. maurice, g. green, & s. c. luce (eds.), behavioral intervention for young children with autism: a manual for parents and professionals (pp. 241-248). austin, tx: pro-ed. creating common vocabulary / cihon, cihon & bedient 347 mcgee, g. g., & morrier, m. j. (2005). preparation of autism specialists. in f. r. volkmar, r. paul, a klin, & d. cohen (eds). handbook of autism and pervasive developmental disorders: assessment, interventions, and policy (pp. 1123-1160). hoboken, nj: john wiley & sons, inc. michael, j. (1975). positive and negative reinforcement, a distinction that is no longer necessary; or a better way to talk about bad things. behaviorism, 3(1), 33–44. michael, j. (1988). establishing operations and the mand. the analysis of verbal behavior, 6, 3–9. rolider, a., & axelrod, s. (2005). the effects of ‘‘behavior-speak’’ on public attitudes toward behavioral interventions. a cross-cultural argument for using conversational language to describe behavioral interventions. in w. l. heward, t. e. heron, n. a. neef, s. m. peterson, d. m. sainto, g. cartledge, r. gardner, iii, l. d. peterson, s. b. hersh, & j. c. dardig (eds.), focus on behavior analysis in education (pp. 283– 293). upper saddle river, nj: pearson/merrill prentice hall. rolider, a., axelrod, s., & van houten, r. (2009). don't speak behaviorism to me: how to clearly and effectively communicate behavioral interventions to the general public. child and family behaviour therapy, 20(2), 39–56. doi: 10.1300/j019v20n02_03 rosales-ruiz, j., & baer, d. m. (1997). behavioral cusps: a developmental and pragmatic concept for behavior analysis. journal of applied behavior analysis, 30(3), 533–544. doi: 10.1901/jaba.1997.30-533 sellers, t. p., alai-rosales, s., & macdonald, r. p. f. (2016). taking full responsibility: the ethics of supervision in behavior analytic practice. behavior analysis in practice, 9, 299-308. skinner, b. f. (1957). verbal behavior. acton, ma: copley publishing group smith, t. (2012). evolution of research on interventions for individuals with autism spectrum disorder: implications for behavior analysts. the behavior analyst today, 35(1), 101–113. smith, t., & iadarola, s. (2015). evidence base update for autism spectrum disorder. journal of clinical child & adolescent psychology, 44(6), 897–922. doi: 10.1080/15374416.2015.1077448 smith, t., scahill, l., dawson, g., guthrie, d., lord, c., odom, s., et al. (2007). designing research studies on psychosocial interventions in autism. journal of autism and developmental disorders, 37(2), 354–366. doi: 10.1007/s10803-006-0173-3 texas council on autism and pervasive developmental disorders (2014). the texas council on autism and pervasive developmental disorders. retrieved from https://hhs.texas.gov/texascouncil-autism-and-pervasive-developmental-disorders-2014-report tiemann, p. w., & markle, s. m. (1990). analyzing instructional content: a guide to instruction and evaluation. champaign, il: stipes pub. weinkauf, s., zeug, n., anderson, c., & ala'i-rosales, s. (2010). evaluating the effectiveness of a comprehensive staff training package for behavioral interventions for children with autism. research in autism spectrum disorders, 5, 864–871. doi: 10.1016/j.rasd.2010.10.001. international electronic journal of elementary education vol.9, issue 2, 337-348, december, 2016 348 www.iejee.com this page is intentionally left blank http://www.iejee.com/ copyright © www.iejee.com issn: 1307-9298 © 2022 published by kura education & publishing. this is an open access article under the cc by ncnd license. (https://creativecommons.org/ licenses/by/4.0/) 225 international electronic journal of elementary education january 2023, volume 15, issue 3, 225-232 leading with compassion: a discussion and steps forward for behavior analysts britany meltona,*, nancy marcheseb, mary jane weissc abstract introduction the provision of applied behavior analytic (aba) services is a highly efficacious intervention approach most often used to improve the lives of individuals with autism spectrum disorder (asd)/autistics. given the advancement of the field, more nuanced skill sets of behavior analysts, such as compassionate care skills, are emerging as the focus of intervention as measurable, observable, and essential. as the field progresses, the identification, refinement, and assessment of more nuanced skills become crucial to the success of our interventions. leading with a compassionate approach that balances habilitation with client happiness, assent, and engagement is of the utmost importance. this paper discusses the current trend in existing compassionate care literature and how those evaluations may potentially be extended to direct interventionists and registered behavior technicians. in this paper, we argue that a behavioral framework should be used to conceptualize, train, and evaluate the demonstration of these skills in front line aba practitioners. applied behavior analysis (aba) can be described as inherently compassionate (baer et al., 1968). it is important to recognize that the science of behavior analysis is compassionate. however, the implementation of interventions informed by the science requires measures of social validity to verify social significance of the interventions. measures of compassion in the provision of aba-based services can aid in the evaluation of the social validity of interventions. compassion has been defined as a direct action that is delivered to alleviate the suffering of another individual (taylor et al., 2018). the provision of ababased intervention is a direct action (i.e., service) that may potentially alleviate perceived challenges of the individual or family and improve the lives of those we serve. while the idea of compassion in service delivery is not a new concept (e.g., bannerman et al., 1990; leaf & mceachin, 1999; taylor et al., 2019; wolf, 1978), there has been an increased focus and discussion on embedding it further into practice related to aba-based interventions for individuals with autism spectrum disorder (asd). keywords: compassion, autism, happiness, rbt, interventionist, aba received : 20 january 2023 revised : 2 february 2023 accepted : 14 march 2023 doi : 10.26822/iejee.2023.295 a,* corresponding author: britany melton, endicott college, journeys autism center, usa. e-mail: bmelton@endicott.edu orcid: https://orcid.org/0000-0002-9513-0240 b nancy marchese, endicott college, usa. breakthrough autism, canada. e-mail: nancy@breakthroughautism.ca orcid: https://orcid.org/0000-0003-4265-4152 c mary jane weiss, endicott college, usa. e-mail: mweiss@endicott.edu orcid: https://orcid.org/0000-0002-2836-3861 226 january 2023, volume 15, issue 3, 225-232 in an effort to better serve clients and families as well as improve collaborative relationships, compassionate care has taken a front seat in behavior analytic discussion, training, and research (chadwell et al., 2018; leblanc et al., 2019; taylor et al., 2018). individuals most frequently responsible for the direct implementation of aba-based interventions for individuals with asd or intellectual disabilities (id) are registered behavior technicians (rbts; behavior analyst certification board, n.d.). the rbt is an individual who has met the knowledge and competency requirements for certification from the behavior analyst certification board (bacb). these individuals are often the face of aba-based intervention for autism (bacb, n.d.). it is imperative that these individuals are compassionate, caring, technical, and collaborative. in this paper, we will discuss broad issues of rbt certification and training that lead to future areas of improvement in the field of behavior analysis. we will also suggest potential remedies for contexts in which there is a deficit in compassionate care. we also propose that all behavior analysts must first lead with compassion to better serve our clients and families, as well as to improve the broad perception of the application of our science. at the same time, it is important to identify how practitioners, especially rbts can balance compassionate care with habilitation and how effective instruction can co-exist with humane and compassionate intervention. perhaps one of the most recent examples of leading with compassionate care has been the work of hanley and colleagues as it relates to the practical functional assessment (hanley et al., 2014), skills based treatment (ghaemmaghami et al., 2015; iannaccone & jessel, 2021), and the use of universal protocols (ghaemmaghami et al., 2016; rajaraman et al., 2022). these procedures are described as ways to implement behavior analytic principles while being guided by compassion and trauma-informed care. happy, relaxed, and engaged (hre) is a concept and practice used as a way to measure the child’s enjoyment and assent with the program that is incorporated into skill-based treatment (ghaemmaghami et al., 2021). hre requires that the staff member operates in ways that produce reinforcement for the individual, thus engaging in direct actions to alleviate the suffering of that individual, which gets to the heart of compassion. there has also been an increased focus on how our interventions may impact individuals with asd. for example, rajaraman et al. (2022) outline ways in which behavior analysts may conceptualize and incorporate trauma-informed practices into the daily implementation of aba. they discussed future recommendations for bcbas to incorporate traumainformed care into practice by 1) acknowledging that potential trauma exists, 2) creating environments rich in reinforcement that promote safety and trust, 3) including potential opportunities for choice as much as possible, and 4) creating intervention programs that focus on the building of adaptive skills that will have a meaningful impact on the individual’s life (rajaraman et al., 2022). not only are these strategies helpful in the potential reduction of trauma, but they also may help foster a compassionate relationship between the therapist and the individual. additionally, ramey et al. (2022) sought to define “happiness” in individuals with asd by replicating a previously established procedure by parsons et al. (2012). ramey et al. demonstrated that while it may often be a difficult concept for behavior analysts, happiness and unhappiness can be defined and agreed upon by observers. ramey et al. defined indices of happiness and unhappiness for each participant based on questionnaires and validated those by creating happy and unhappy conditions. percentages of idiosyncratic happiness and unhappiness were measured in each condition. the results highly correlated with the results on the questionnaires. this is a wonderful step in the right direction for the operational definition and measurement of emotional states for individuals with asd. with these tools readily available in the literature, it is now time to add them to daily practice to ensure the social validity of our procedures. the circumstances view of behavior as described by friman (2021) is a description and/or explanation of how the field of behavior analysis is rooted in compassion. friman explained that behavior, as viewed through the eyes of radical behaviorism, is a function of the circumstances of the individual. often, other explanations of behavior would describe behavior as a part of the person and place blame on an individual when they engage in a behavior that is undesirable by society (e.g., commit a crime). however, behavior analysis would explain this behavior as a function the environment and the past reinforcement and punishment histories for that individual. behavior analysis does not blame individuals, but rather, explains the behavior in the context of the individual’s experience of the environment. thus, behavior analysis by its very definition exemplifies compassion. behavior analysts place a high value on the use of evidence-based treatments and are committed to effective intervention. the commitment to effective intervention may have led to some challenges when working in interdisciplinary teams, and thus behavior analysts have faced criticism regarding collaboration (leaf et al., this issue; rohrer et al., 2021). allied professions have expressed that behavior analysts can seem arrogant, dismissive of other perspectives, or rigid (e.g., gaswieski et al., 2021; gerenser & koenig, 2019; koenig & gerenser, 2019; schiebel & walting, 2016; welch & polarajko, 2016). a primary related concern has 227 leading with compassion: a discussion and steps forward for behavior analysts / melton, marchese & weiss been the use of behavioral jargon; members of allied disciplines may not understand technical terms such as the terms such as tact and intraverbals (critchfield, 2017). there are often dual uses of other terms, such as punishment. in either context, there may be a gap in understanding for nonbehavioral professionals. additionally, behavior analysts must adhere to the ethical code for behavior analysts (bacb, 2020; cox, 2019) which requires, amongst many other things, that behavior analysts support and only recommend evidence-based practices and procedures. this often leads to issues with non-behavioral colleagues, who may feel that their recommendations are dismissed because the procedures they are recommending are not yet evidence based. behavior analysts may be attempting in these interactions to adhere to the ethical code; however, they may not possess the nuanced interpersonal skill set that would enable meaningful dialogue and effective compromises. alternate certifications and allied professions also mandate adherence to ethics codes (e.g., american psychological association, american speech/language hearing association, american occupational therapy association, council for exceptional children). the complexities of autism require the expertise of multiple disciplines, and outcomes are enhanced through a multidisciplinary model (e.g., lafrance et al., 2019; slim & rueter-yuill, 2021). the importance of enhancing collaborative skills among behavior analytic practitioners has been strongly emphasized within the field, especially recently (brodhead, 2015; bowman et al., 2021). such skills are important for all collaborative relationships, including those with families, allied professionals, and clients (weiss et al., 2022). the discussion around compassionate care in aba has mainly been focused on the master’s level clinicians. given that rbts/ practitioners likely encounter families and children frequently, we will outline how the provision of compassionate aba can be extended to the rbt/ practitioner level. direct implementation of aba-based interventions in autism treatment settings rbts are most often the direct, client-facing, interventionists responsible for the implementation of aba-based interventions with individuals diagnosed with asd/autistics (bacb, n.d.). these professionals are required to complete, at minimum, 40 hr of training, pass a competency assessment, as well as pass a standardized knowledge exam (bacb, n.d.). rbts are also required to be supervised for at least 5% of working hours to ensure accurate and precise implementation of the treatment program and to maintain certification. early in the development of the field of aba, practitioners were few and the field was very small. given the extensive growth of the field, there has been a drastic and almost exponential increase in credentialed professionals (bacb, n.d.). today, the rbt credential requires a high school diploma and the previously listed criteria. it should be noted, however, that some have suggested that these minimal standards may result in insufficient competency and knowledge of our most forwardfacing interventionists responsible for implementing behavior analytic procedures (leaf et. al., 2021). the role of the rbt is multi-faceted as they are crucial to the outcomes produced for learners. they must build rapport with the individual, implement various teaching protocols (e.g., functional communication training, discrete trial teaching, naturalistic teaching), collect various forms of data (e.g., antecedentbehavior-consequence, rate, interval), navigate mealtimes, toileting procedures, and more all while responding to all of the child’s needs. when learning various protocols, it may be easiest for the staff member to implement clearly outlined procedures to increase procedural fidelity. arguably, the best implementation of all above listed procedures involves clinical judgement on the part of the rbt (leaf et al., 2016). for example, consider the application of differential reinforcement. this concept and practice require that one does not reinforce undesired behaviors or only reinforces desired behaviors (vladescu & kodak, 2010). differential reinforcement, when implemented precisely, leads to great gains in skills and improvement in overall functioning (vladescu & kodak, 2010). quality application of differential reinforcement requires that the staff member analyze the behavior of the individual and respond accordingly with varying amounts/levels of reinforcement contingent on various responses. however, without clear oversight, feedback, and a solid understanding of the implementation of differential reinforcement, the rbt may be prone to a variety of errors. for example, they might miss opportunities to reinforce better articulation or social bids that are especially clear to peers. in the context of challenging behaviors, they may exhibit “cold” reactions when the child engages in emotional behavior, which could disrupt rapport. as an illustration, when given the instruction, “do not reinforce that behavior,” the rbt may rigidly interpret that as an instruction to ignore the child completely without regard for the behavioral function or emotional state of the individual. this could lead to a rigid interpretation of the behavior plan and result in less in-the-moment assessment and responding compassionately to the child across various situations. rigid implementation of aba can potentially lead to less compassionate responses (e.g., staff simply following plans rather than responding to the child and altering their own behavior). implementation at a high degree of treatment fidelity does not need to be mutually exclusive to the application of compassionate care. the rbt is the 228 january 2023, volume 15, issue 3, 225-232 primary interventionist can respond compassionately and flexible to the child while applying the required underlying behavioral principles. compassionate care as a soft skill board certified behavior analysts (bcbas) commonly receive little instruction on soft skills and how those may interact with the relationship between stakeholders (rohrer et al., 2021). one potential reason for this may be that traditionally these skills have been viewed as challenging to operationalize and, thus, teach. no professional tools nor guidelines exist to support behavior analysts work in this area. further, coursework and other formal education in behavior analysis historically has placed great emphasis on a technical understanding of the science. the field has begun to address these gaps and to look to remedy any lack of training or behavioral deficits (leblanc et al., 2019, rohrer et al., 2021). most of this work has focused on the interactions between professionals and/or professionals and caregivers. for example, rohrer et al. (2021) suggested a checklist to evaluate the interaction between the bcba and caregiver during interactions. the compassionate collaboration tool utilized a 3-point likert scale to assess the interactions that included both compassionate interactions married with collaborative behaviors. the need for advancement in this research and skill set is vast and essential. the field of behavior analysis will be greatly improved by the development of instructional approaches that target these skills. alongside this effort, we must begin to develop measures of the direct interaction of therapists (rbts or otherwise) with the recipient of the service. as the ultimate beneficiary of service and as the primary focus of intervention, the clients served must be consistently met with humane and compassionate care. at the present time, there is a paucity of published literature that focuses on interpersonal skills and other soft skills that may be important for rbts to demonstrate. further, there are very few resources on evaluating the extent to which compassion is integrated into their training. from identification to practice it has been noted that some practitioners in the field of behavior analysis are lacking in compassionate care behaviors or skills (leblanc et al., 2019, rohrer et al., 2021; rohrer & weiss, 2022; taylor et al., 2018), especially in the context of the parent-professional relationships. it is important that this work is also extended to client-instructor interactions. practitioners must continue to lean on previous research (e.g., baker, 2013; harris & weiss, 1997) and continue to develop behavioral measures of these interactions as well as develop behavioral measures of authentic or genuine interactions. measures of authenticity will allow professionals to make better decisions regarding efficacious training techniques as well as improve our intervention quality. given that behavior analysts should evaluate compassionate interactions using a behavioral framework, they should also train rbts to use a behavioral framework when working with individuals with asd (leaf et al., 2021). this will better inform the rbt on how, when, why, and under what circumstances to flexibly respond to the individual during times of high stress or concern. compassionate behaviors are just that, behaviors. these behaviors are measurable, observable, and trainable; similarly, interventions can be evaluated to increase or decrease those behaviors. while measuring aspects of behaviors such as the extent to which a behavior is genuine or authentic may prove to be quite challenging, it is imperative that we do so to achieve best outcomes and high consumer satisfaction. rapport building between clinicians and caregivers of people with asd may be facilitated through training in compassionate care (leblanc et al., 2019; taylor et al., 2018). for example, marchese (2021), in an unpublished dissertation, developed and evaluated seven instructional modules to support training graduate students in compassionate care skills. a modified behavioral skills training package was implemented across each module that included a pre-test, online powerpoint presentation (i.e., sample of the literature, modeling on how to display various compassionate care skills), active rehearsal of target skills via homework assignment, feedback on the assignment, and a post-test. the modules were delivered via a class wide training format. results were compared to a second control group of students who did not access the training module nor the homework assignment. the control group only completed the preand posttests. overall, students in the experimental group demonstrated statistically significant increases in knowledge (i.e., post-test scores across modules) and performed well on the behavior component of the instructional modules. this study provides a preliminary demonstration that compassionate care training can be integrated within a class wide training format. assessment tools may also be used to support the application of compassionate care skills within clinical environment. marchese (2021) undertook a multistep approach to develop the parent partnership questionnaire (ppq), a tool to support collaboration between parents of children with autism and their clinician. the development included a content creation phase that involved conducting semistructured interviews with parent of children with autism, an online survey for parents to give feedback on the clarity and understanding of questions within 229 leading with compassion: a discussion and steps forward for behavior analysts / melton, marchese & weiss the draft tool, and finally procedural integrity checks and reliability testing were conducted on the final version of the tool. the ppq outlines various openended questions for clinicians to ask parents to gain an understanding of their priorities, values, and potential logistical considerations that may ultimately facilitate parent engagement. it is hypothesized that the administration of this tool allows clinicians to gather information from parents while developing rapport between both parties. the ppq may offer clinicians a structure to support the application of compassionate care skills in clinical setting. the development of tools like the ppq will offer the field more ways to better operationalize, measure, evaluate and teach skills relating to therapeutic relationships and compassionate care. we are hopeful that the bacb will expand consideration for soft skills on the next task list for rbts. the current task list outlines that the rbt adheres to all elements of the ethics code which includes behaviors like compassion and elements of soft skills. one potential consideration for expansion is to include fluency or demonstration of soft skills in the current 5th edition task list (bacb, 2021). this may facilitate the advancement of our field, the general perception of our practice and most importantly client and family outcomes. there have already been steps toward this in the research as evidenced by rohrer and weiss’ (2022) preliminary investigation. specifically, rohrer and weiss first identified empathetic and compassionate skills, then subsequently divided them into three categories that could be measured, observed, and evaluated. the skills identified were interviewing skills, interest in the family, and joining the family. using behavior skills training, the researchers taught four master’s students in behavior analysis these skills via telehealth. the results were promising in that all participants improved in engagement, social validity measures, and other standardized forms of measurement from other fields (e.g., the jefferson scale of physician empathy). all participants in this study had bachelor’s degrees and were earning master’s degrees in behavior analysis (rohrer & weiss, 2022). an important extension of this research would be to expand this research to the rbt level. specifically, such training could target areas of interaction between the rbt and the family and/or child. it is essential as the field that this concept be extended to interactions between the client and the instructor/rbt. table 1 provides a suggested process for the evaluation and analysis of compassionate interactions when incorporated into in-the-moment decision making by bcbas and rbts. when using this approach to every interaction, the bcba or rbt may be better equipped to analyze and alter their behavior given various situations. this is not a new concept, as this closely resembles the antecedent-behaviorconsequence evaluation procedures (cooper et al., 2019). this is a hallmark of aba intervention and is easily applied to situations that would improve compassionate behaviors on the part of the rbt. this simple framework could aid the rbt in evaluating their own behaviors relating to the nature of the interaction between child and therapist and in identifying how their behavior is impacting the emotional state of the child. the therapist could then more thoroughly respond to the child in a way that is intentional and leading with a compassionate approach. when using this approach, the rbt can learn how to best answer questions in times of calm interactions (e.g., the child is engaged, assenting to learning opportunities, seems content with interactions from the therapist) and when to alter responses in times of crisis (e.g., severe problematic behaviors, refusal to complete tasks, minor problematic behaviors). table 2 provides an expansion of table 1 that outlines some potential examples of situations the rbt may encounter. in the example of the calm situation, specific behaviors are listed that lead the therapist to believe the child is in a calm state. the response is outlined, as well as an analysis of how the therapist response did or did not change the behavior of the child. in the crisis, clear behavioral descriptors are present, the response is clearly described, and the analysis of the response allows the therapist to observe that the behavior was improved. often, rbts are provided only with consequence procedures and not with the analysis of how that response impacted the child’s behavior. in addition, rbts may not be trained in analyzing antecedent conditions at this level. indeed, the skill set should include understanding indices of unhappiness (ramey et al., 2022), and identifying when the client is no longer happy, relaxed, and engaged. this is not to suggest that practitioners should abandon consideration the function of the challenging behavior nor function based treatments. that is, careful consideration of the potential contraindicative intervention strategies must be included in application of compassionate care skills. effective intervention can include both the application of compassionate care and what is needed to facilitate meaningful skill building. one is not mutually exclusive of the other. this framework includes evaluation of both the response to the client within the framework of compassionate care and what is therapeutically appropriate to build skills while reducing challenging behavior. for example, the rbt can learn how to embed learning opportunities, progressively shift schedules of reinforcement, and differentially respond to the client while ensuring the client well cared for both emotionally and physically. 230 january 2023, volume 15, issue 3, 225-232 summary to date, the literature addressing the improvement and refinement of compassionate care skills in behavior analysis has largely been limited to that of parent/caregiver and professional (e.g., bcba) interactions. we strongly believe this focus must also include addressing the soft skills of direct care interventionists at the rbt level. there has been significant growth in the identification of behaviors that may comprise a definition of compassionate skills (leblanc et al., 2020; marchese, 2021; melton et al., in press; rohrer & weiss, 2022; taylor et al., 2018). further research should refine and extend these investigations to direct interventionists. the field of behavior analysis at its very core is compassionate, and there has been a resurgence of interest in how this core value can be enacted consistently in practice (e.g., baer et al., 1968; bannerman et. al., 1990; friman, 2021; foxx, 1996; rohrer et al., 2021; taylor et al., 2019; wolf, 1978). as the field progresses, the identification, refinement, and assessment of more nuanced skills is crucial to the success of our interventions. leading with a compassionate approach that balances habilitation with client happiness, assent, and engagement is of the upmost importance. a compassion-based approach will advance the partnership between professionals and clients and will ensure that we are held accountable in making socially significant differences that are validated by those we serve. references baer, d. m., wolf, m. m., & risley, t. r. (1968). some current dimensions of applied behavior analysis. journal of applied behavior analysis, 1(1), 91. baker, j. (2013). key components of social skills training. teaching social skills to people with autism: best practices in individualizing interventions. bethesda, md: woodbine house, inc. behavior analyst certification board. (n.d). bacb certificant data. retrieved from https://www. bacb.com/bacb-certificant-data. behavior analyst certification board. (2020). ethics code for behavior analysts. https://bacb. com/wp-content/ethics-code-for-behavioranalysts/ behavior analyst certification board. (2021). rbt ethics code (2.0). https://www.bacb.com/wpcontent/rbt-ethics-code bowman, k. s., suarez, v. d., & weiss, m. j. (2021). standards for interprofessional collaboration in the treatment of individuals with autism. behavior analysis in practice, 14(4), 1191–1208. https://doi.org/10.1007/s40617-021-00560-0 table 1 a process for evaluation and analysis of compassionate interactions. situation behaviors response analysis identify the situation that is potentially causing stress or enjoyment to the individual what behaviors are you seeing that lead you to believe this? what are some potential acceptable responses to this behavior provided the behavior plan and training? how does my behavior improve or worsen the condition of the individual? table 2 a process for evaluation and analysis of compassionate interactions examples situation behaviors response analysis identify the situation that is potentially causing stress or enjoyment to the individual what behaviors are you seeing that lead you to believe this? what are some potential acceptable responses to this behavior provided the behavior plan and training? how does my behavior improve or worsen the condition of the individual? examples calm: playing with the rbt steady eye contact; approaching therapist; occasional smiling continue with activity; increase learning opportunities; my response led to the continuation of calm behaviors; reassess if any behavior changes crisis: provided task demand, removed play materials crying; avoidant of task materials; flat affect preference assessment; label potential emotional state; provide additional choices my response improved the state of the child; child began approaching me calmly 231 leading with compassion: a discussion and steps forward for behavior analysts / melton, marchese & weiss brodhead, m. t. (2015). maintaining professional relationships in an interdisciplinary setting: strategies for navigating nonbehavioral treatment recommendations for individuals with autism. behavior analysis in practice, 8(1), 70–78. https://doi.org/10.1007/s40617-015-0042-7 callahan, k., foxx, r. m., swierczynski, a., aerts, x., mehta, s., mccomb, m. e., ... & sharma, r. (2019). behavioral artistry: examining the relationship between the interpersonal skills and effective practice repertoires of applied behavior analysis practitioners. journal of autism and developmental disorders, 49(9), 3557-3570. cooper, j. o., heron, t. e., & heward, w. l. (2019). applied behavior analysis (3rd edition). hoboken, nj: pearson education. cox, d. j. (2019). ethical considerations in interdisciplinary treatments. in r. d. rieske (ed.), handbook of interdisciplinary treatments for autism spectrum disorder (pp. 49–61). springer international publishing. https://doi. org/10.1007/978-3-030-13027-5_4 chadwell, m. r., sikorski, j. d., roberts, h., & allen, k. d. (2018). process versus content in delivering aba services: does process matter when you have content that works? behavior analysis: research and practice, 19(1), 14–22. http://dx.doi. org/10.1037/bar0000143 critchfield, t. s. (2017). visuwords®: a handy online tool for estimating what nonexperts may think when hearing behavior analysis jargon. behavior analysis in practice, 10(3), 318–322. https://doi.org/10.1007/s40617-017-0173-0 friman, p. c. (2021). there is no such thing as a bad boy: the circumstances view of problem behavior. journal of applied behavior analysis, 54(2), 636-653. foxx, r. m. (1996). twenty years of applied behavior analysis in treating the most severe problem behavior: lessons learned. the behavior analyst, 19(2), 225-235. gasiewski, k., weiss, m. j., leaf, j. b., & labowitz, j. (2021). collaboration between behavior analysts and occupational therapists in autism service provision: bridging the gap. behavior analysis in practice, 14(4), 1209-1222. https://doi.org/10.1007/ s40617-021-00619-y gerenser, j. e., & koenig, m.a. (2019). strategies to enhance slp-aba collaboration: working toward interprofessional practice. in j. gerenser & m. koenig (eds.), aba for slps: interprofessional collaboration for autism support teams. brookes publishing. harris, s. l., & weiss, m. j. (1998). right from the start: behavioral intervention for young children with autism. woodbine house. hanley, g. p., jin, c. s., vanselow, n. r., & hanratty, l. a. (2014). producing meaningful improvements in problem behavior of children with autism via synthesized analyses and treatments. jaba, 47, 16-36. ghaemmaghami, m., hanley, g. p., jin, s. c., & vanselow, n. r. (2015). affirming control by multiple reinforcers via progressive treatment analysis. behavioral interventions, 31(1), 70-86. ghaemmaghami, m., hanley, g. p., & jessel, j. (2016). contingencies promote delay tolerance. journal of applied behavior analysis, 49(3), 548-575. ghaemmaghami, m., hanley, g. p., & jessel, j. (2021). functional communication training: from efficacy to effectiveness. journal of applied behavior analysis, 54(1), 122-143. iannaccone, j. a., & jessel, j. (2021). a translational comparison of contingency‐based progressive delay procedures and their effects on contextually appropriate behavior. journal of applied behavior analysis, 54(1), 231-247. leblanc, l. a., taylor, b. a., & marchese, n. v. (2020). the training experiences of behavior analysts: compassionate care and therapeutic relationships with caregivers. behavior analysis in practice, 13(2), 387-393. leaf, j. b., leaf, r., mceachin, j., bondy, a., cihon, j. h., detrich, r., eshleman, j., ferguson, j. l., foxx, r. m., freeman, b. j., gerhardt, p., glenn, s. s., miller, m., milne, c. m., mountjoy, t., parker, t., pritchard, j., ross, r. k., saunders, m., & streff, t. (2021). the importance of professional discourse for the continual advancement of practice standards: the rbt® as a case in point. journal of autism and developmental disorders, 51, 1789-1801. koenig, m. a., & gerenser, j. e. (2019). interprofessional collaboration. in j. gerenser & m. koenig (eds.), aba for slps: interprofessional collaboration for autism support teams (pp. 3-27). brookes publishing. 232 january 2023, volume 15, issue 3, 225-232 lafrance, d. l., weiss, m. j., kazemi, e., gerenser, j., & dobres, j. (2019). multidisciplinary teaming: enhancing collaboration through increased understanding. behavior analysis in practice, 12(3), 709–726. https://doi.org/10.1007/s40617019-00331-y leaf, j. b., leaf, r., mceachin, j., taubman, m., ala’irosales, s., ross, r. k., ... & weiss, m. j. (2016). applied behavior analysis is a science and, therefore, progressive. journal of autism and developmental disorders, 46(2), 720-731. leaf, j. b., leaf, r., mceachin, j., bondy, a., cihon, j. h., detrich, r., ... & streff, t. (2021). the importance of professional discourse for the continual advancement of practice standards: the rbt® as a case in point. journal of autism and developmental disorders, 51(5), 1789-1801. leblanc, l. a., taylor, b. a., & marchese, n. v. (2019). the training experiences of behavior analysts: compassionate care and therapeutic relationships with caregivers. behavior analysis in practice, 13, 387–393. https://doi.org/10.1007/ s40617-019-00368marchese, n. v. (2021). supporting behavior analysts in providing compassionate care – the development of the parent partnership questionnaire (ppq) and instructional modules for graduate training [unpublished dissertation]. endicott college. melton, b., o’connell-sussman, e., lord, j., weiss, m.j. (in press). empathy and compassion as the radical behaviorist views it: a conceptual analysis. behavior analysis in practice. ramey, d., healy, o., & mcenaney, e. (2022). defining and measuring indices of happiness and unhappiness in children diagnosed with autism spectrum disorder. behavior analysis in practice. https://doi.org/10.1007/s40617-02200710-y rohrer, j. l., & weiss, m. j. (2022). teaching compassion skills to students of behavior analysis: a preliminary investigation. behavior analysis in practice, 1-20. rohrer, j., marshall, k., suzio, c., & weiss, m.j., (2021). soft skills: the case for compassionate approaches or how behavior analysis keeps finding its heart. behavior analysis in practice. 14. 10.1007/s40617021-00563-x. parsons, m. b., reid, d. h., bentley, e., inman, a., & lattimore, l. p. (2012). identifying indices of happiness and unhappiness among adults with autism: potential targets for behavioral assessment and intervention. behavior analysis in practice, 5(1), 15-25. scheibel, g., &watling, r. (2016). collaborating with behavior analysts on the autism service delivery team. ot practice, 21(7), 15–19. https://www.proquest.com/trade-journals/ collaborating-with-behavior-analysts-onautism/docview/1783039605/se-2 slim, l., & reuter-yuill, l. m. (2021). a behavior-analytic perspective on interprofessional collaboration. behavior analysis in practice, 14(4), 1238–1248. https://doi.org/10.1007/s40617-021-00602-7 taylor, b. a., leblanc, l. a., & nosik, m. r. (2019). compassionate care in behavior analytic treatment: can outcomes be enhanced by attending to relationships with caregivers?. behavior analysis in practice, 12(3), 654-666. vladescu, j. c., & kodak, t. (2010). a review of recent studies on differential reinforcement during skill acquisition in early intervention. journal of applied behavior analysis, 43(2), 351-355. welch, c. d., & polatajko, h. j. (2016). applied behavior analysis, autism, and occupational therapy: a search for understanding. american journal of occupational therapy, 70(4), 1-5. https://doi. org/10.5014/ajot.2016.018689 weiss, m. j., tereshko, l., bowman, k., marshall, k., & rose, k. (2022). effective collaboration: maximizing outcomes in autism intervention in an interdisciplinary model. in j. leaf, j.h. cihon, j.l. ferguson, & m.j. weiss (eds.), handbook of applied behavior analysis interventions for autism (pp. 125-149). springer. wolf, m. m. (1978). social validity: the case for subjective measurement or how applied behavior analysis is finding its heart 1. journal of applied behavior analysis, 11(2), 203-214. international electronic journal of elementary education, 9(3), 499-510, march, 2017 issn:1307-9298 copyright © iejee www.iejee.com change in elementary school students’ misconceptions on material systems after a theoretical-practical instruction florentina cañada cañada a * david gonzález-gómez b diego airado-rodríguez a lina viviana melo niño c maría antonia dávila acedo a a university of extremadura, badajoz, spain b university of extremadura, cáceres, spain c university of salamanca, salamanca, spain received: 21 november 2016 / revised: 18 january 2017 / accepted: 2 february 2017 abstract students get to elementary school with a series of misconceptions which are not necessarily in agreement with the scientific knowledge. misconceptions result from the student’s attempts to understand their previous experiences resulting from their interaction with their environment and they must be taken into account in educational practice to design appropriate instructional strategies that will impulse their evolution into more scientific-academic concepts. this research aims to detect the initial knowledge that twenty-one fifth grade elementary education students (ages ranged: 10-11 years old) have about material composition. a questionnaire was used as an instrument to determine initial misconceptions. based on the students’ answers, important misconceptions were identified, and an appropriate instructional strategy was design to foster the students’ conceptual change about the studied topic. a post-task questionnaire conducted after the instruction revealed the suitability of the proposed instructional design. keywords: misconceptions; experimentation in chemistry; elementary school; conceptual change. introduction teaching and learning chemistry could be considered as a complicated task mainly due to the high conceptual nature of the discipline, with a vast amount of interlinked and abstract concepts (johnstone, 2000; tümay, 2016). consequently, quite often chemistry is a priori considered as a difficult topic to learn among students and even a hard-to-teach subject among instructors. in the teaching-learning process, students construct their own knowledge as an iterative process, in which the new concepts or ideas need to make sense in the frame of the * corresponding author: florentina cañada cañada, department of science and mathematics education, faculty of education, university of extremadura (uex). av. de elvas s/n, 06006, badajoz, spain. phone: +34924289860 e-mail: flori@unex.es http://www.iejee.com/ international electronic journal of elementary education, 9(3), 499-510, march, 2017 500 conceptions that they already possessed (taber, 2009). however the real implementation of such constructivist approaches in primary school is not always easy (ari, kizilaslan tunçer & demir, 2016), and as previously stated by bedir (2015), their success has a high dependence on the teachers’ skills in the use of methods and techniques based on studentcentered activities. accordingly, detecting the ideas that students already have prior to the instruction is a key stage, since it constitutes the linkage between old and new concepts. moreover, before starting any kind of instruction, students already possess understanding about many scientific topics that form a mental framework, referred to as the scaffolding (horton, 2007; kleichmann et al., 2016). at this point it is a must to define misconceptions, which are individual constructions or mental representations of the world that students have adopted in order to understand the environment and to act accordingly. misconceptions are characterized for being firmly held by the students, who are not often willing to change (treagust & duit, 2008; dağdelen & kösterelioğlu, 2015). it is important to notice that these conceptions lead to conceptual mistakes and might be an obstacle for learning scientifically correct concepts. thus, in the building up of the learning process, students need to arrange all the new information to get it fit into the scaffolding. sometimes these arrangements do not agree with the current scientific thought, which gives rise to misconceptions (taber, 2001), and therefore, the new knowledge is constructed into a conceptually faulty base. misconceptions should not be considered as a handicap for students to incorporate new knowledge, but a necessary starting point from which student will be able to build new scientific understanding from a constructivist point of view (furió-mas, calatayud & barcenas, 2007). thus, misconceptions should be taken into account when planning and implementing instruction (taber, 2008) and must be considered by the teacher during the educational process to ease the rebuilding of the knowledge in a significant way, which would turn into meaningful learning (martins teixeira & moura bezerra sobral, 2010). when providing effective instructional approaches, to overcome misconceptions, students will be able to connect the former and new conception in order to later acquire a real meaning. the resistant nature of misconceptions may discourage teaching efforts, and research on educational science has put effort in developing strategies to induce the transformation of misconceptions into scientific conceptions, or at least, to more scientifically accurate concepts. this set of strategies or instructions was named as “conceptual change” (leach et al., 1997). the conceptual change could be defined as the modification of students’ conceptions and their substitution by other, more scientifically accepted ones, which ensures the appropriate learning (harahan, 1994). in fact, learning scientific concepts should not only consist of replacing an idea with another scientifically accepted one, but in making connections between spontaneous student theories and scientific theories. students must get to understand the superiority of accepted theories, and to achieve that situation they should be faced with conflictive situations that cannot be solved by using their own theories. the educator should drive students to situations that cannot be explained with their previous ideas. the perception of inconsistency among students’ cognitions generates psychological discomfort and that motivates them to attempt to resolve the dissonance by incorporating scientific knowledge in a natural way (posner et al., 1982). therefore, the cognitive conflict, as defined by lee & byun (2012) is a perceptual state of the discrepancy between one’s mental model and the external information recognized (internal-external conflict), or between different mental models of one’s cognitive structure (internal conflict). thus, this cognitive conflict initiates the first step in the process of the conceptual change, and therefore is imperative for achieving a conceptual change (treagust & duit, 2008). the new information must be understandable by the students, consistent with other theories and with their own experiences. change in elementary school students’ misconceptions / cañada, gómez, rodríguez, niño, acedo 501 the concept of substance is among the most important in chemistry, since it is a key issue to recognize and identify the field of the chemistry study. the concept is not only relevant from an academic point of view, but also in a daily life context since the identification of dangerous substances must be acknowledged to avoid the interaction with them (fernández-gonzález, 2013). despite the fact of its importance, the concept of substance is normally used as a synonym for material, product, object, and some others. thus, the recognition of the diversity of matter is a fundamental goal of chemistry teaching (martínez losada, garcía barros & rivadulla lópez, 2009). for this reason, material systems are introduced firstly in elementary education through science education subjects and they are studied more in-depth in secondary education. misconceptions are very persistent in chemistry at all educational levels (oliveira, gouveia & de cuadros, 2009; stains & sevian, 2014; tümay, 2016). more precisely, specific studies regarding the alternative conceptions that students of secondary education level have about the nature of matter and its changes, have been extensively reported (akgun & aydin, 2010; calik, ayas & coll, 2007; johnson, 2005; kingir, geban & gunel, 2013). in contrast, the number of studies at the primary education level, about this specific topic, is considerably more limited, mostly because authors have traditionally considered them as scientifically complex and with a sort of abstraction more suitable for secondary education level (rubio cascales, 2010). regarding the elementary school, cañada et al. (2012), noticed that students at this educational level have some difficulties to distinguish between pure substances and mixtures, especially naturally occurring homogeneous mixtures, such as milk. martín del pozo and galán martín (2012) reported similar findings in a study performed with students from the 2nd, 4th and 6th grades of elementary education (ages ranged: 8-11 years old). in this case, for instance, the majority of the students identified granite as a substance, regardless of the fact they could clearly observe the different minerals that form the granite stone. authors conclude that in this case, the natural origin overrides the student’s observation. furthermore, the authors also point out that students identify manufactured processed materials, such as iron or copper as mixtures. furthermore, durmus and bayraktar (2010) conducted a research where different instructional methodologies were applied to foster a conceptual change in 4th grade students. this report explains how the conceptual change texts are as effective as hands on laboratory experiments, and both together are effective in overcoming misconceptions regarding the matter and matter changes. pine, messer and st. john (2014) have evaluated science misconceptions of elementary school students from the teachers’ point of view by interviewing 122 elementary school science teachers in england. in this research, regarding the material systems classification, a teacher reported that a child believed that rocks could not be natural because they are dead. on the other hand, this report also points out the limitation of students understanding of the material system, since many of them considered that the term ‘material’ was restricted to cloth and fabric, and did not consider rocks and plastic to be materials. according to vogelezang (1987), elementary school students have important experiences with pure substances and mixtures, although they are not aware of the chemical sense of this problem. the same author also points to the perception as an important factor to classify material systems for children. thus, the purpose of this study is to ascertain the previous ideas of students in the 5th grade of elementary education on material systems and, more precisely, on the differences between pure substances and mixtures, and to investigate the effectiveness of conceptual change and laboratory instruction on these misconceptions. to achieve this goal, a three-step methodology is proposed. firstly, a questionnaire was designed to international electronic journal of elementary education, 9(3), 499-510, march, 2017 502 investigate the students’ initial ideas about material systems. in a second stage, these initial findings were employed to design a proper instructional strategy to in order to induce on students the overcoming misconceptions. finally, the effectiveness of the whole process was evaluated through a post-task questionnaire. the study (method) sample composition this study follows a descriptive and interpretative methodology with the aim of describing, explaining and understanding mental representations that the 5th grade of elementary school students have in relation to different material systems. the considered sample consists of 21 students of the 5th grade (ages ranged: 10-11 years old) of a public school of elementary education in badajoz (spain). methodology the experience was implemented along four weeks, in may 2014. firstly, a pre-test was conducted aimed to find out the students’ misconceptions about material systems. two weeks later, students were given a total 90-minutes instruction divided in two sessions of 30 and 60 minutes, respectively. this instruction was designed bearing in mind the detected misconceptions, as it can be observed in figure 1. sessions are described in detail below, the first session was a theoretical one and the second one consisted on lab-work. figure 1. scheme of the applied instruction in the first session, an interactive overhead presentation was used to cover the key concepts (pure substance / mixture) and students-teacher interaction was promoted throughout. in general, along primary education pure substances are very scarcely studied. the study of the inert matter is mainly focused in mixtures, without mentioning that those mixtures are actually made of pure substances. in a way, this causes that students unconsciously associate pure with natural and mixture with artificial. thus, taking these premises into account, the next items were considered in the presentation for the first session: 1) classification of the matter according to origin and composition; 2) definition of pure substance and examples; 3) definition of homogeneous / heterogeneous mixtures and examples; 4) quick test consisting on classifying like pure substance or change in elementary school students’ misconceptions / cañada, gómez, rodríguez, niño, acedo 503 homogeneous / heterogeneous mixtures everyday objects. the second session consisted on a 60-minutes lab workshop where students experienced with everyday material systems. the workshop was planned to foster the cooperative participation of the students. students were arranged in groups, and each group was provided with different materials of daily use such as salt, sand, milk, olive oil, water, sugar and ethanol. then, they were asked to classify and mix the different materials, and after that they were asked to take notes of the observations. the next list of activities was proposed, to be implemented during the practical session: 1) classify the next elements like pure substances or mixtures: sugar, salt, steel nails, olive oil, granite stone, sand and water. 2) prepare different mixtures with the elements of the list described above. students were asked to prepare at least seven mixtures, explain what happened in each case and classify the resulting mixtures like homogeneous or heterogeneous. 3) separate milk into its components. it was faced with this experiment one of the most strongly settled misconception, which is considering pure substances to all natural products. by means of this experience, students proved and saw with their eyes that milk is formed by several substances. vinegar was added to milk, which induced protein precipitation. after protein precipitation it was easy to observe a liquid supernatant and a solid precipitate. in this way, it was easy for the students to understand that there are naturally occurring substances that are not pure substances, but mixtures. the two sessions described so far, took place in the same week. after that, a post-test was conducted in the fourth week. preand post-test results were compared in order to gauge the effectiveness of the proposed methodology. instruments the instrument used in this research was a questionnaire, designed from previously published studies in relation with the same topic (martín del pozo & galán martín, 2012). the questionnaire consisted in three closed questions, where the students were asked to select the right answer in two different situations (question 1 and 2) and order different daily-used materials according to their nature (question 3). in all cases, students were asked to properly justify their answers (figure 2). in order to assess the students’ initial ideas about matter systems and the students’ performance after the instruction, they were asked to complete the questionnaire before (pre-test) and after the instruction (post-test). figure 2. preand post-test questionnaire used as instrument to assess students’ ideas about material systems after and before the instruction. international electronic journal of elementary education, 9(3), 499-510, march, 2017 504 data analysis bar plots were used to represent the answers for closed questions. the qualitative treatment of answers to the open-ended questions was done by grouping them under different categories, according to the main idea the students expressed through those answers. results and discussion firstly the pre-test and the two-session instruction strategy were completed and after that all the information was collected and analyzed. results are presented and discussed in the same order as test questions answered by the students. for each question, a detailed analysis of the responses is presented, followed by a comparison between the data obtained before the instruction (pre-test) and after the instruction (post-test). in the first question, students were asked about a mixture of water and olive oil. specifically, they had to answer whether the mixture was homogeneous or heterogeneous and they had also to provide a short explanation justifying the selected option. according to the answers provided to the closed question homogeneous/heterogeneous, 71% of the students selected the wrong option “homogeneous mixture” in the pre-test (figure 3), in other words, prior to the instruction most of students did not know how to classify properly the mixture of water and olive oil. figure 3. bars diagram representing the answers of the students to the closed question homogeneous/heterogeneous for an olive oil / water sample. the reasons provided by the students to support their selection of homogeneous or heterogeneous were analyzed. the evolution of these reasons is schematically represented in figure 4. regarding those selecting “homogeneous mixture”, 25% of them gave a right definition: “a mixture of olive oil and water is a mixture where the components are distinguished”. this was the most frequent answer between those selecting homogeneous, followed by “components have different density” in 10% of the cases, “components are undistinguished” in 5% of the cases. therefore, 25% of the students selecting homogeneous in the pre-test knew that substances remain separate in the mixture, but they were not able to match it with correct criterion homogeneous/heterogeneous, this lead us to think that the origin of this misconception could be related even with the linguistic competence. in this pre-test, only 25% of the students selected heterogeneous, and only 5% of those justified properly their selection with answers like of “the components of the mixture are distinguished”. only 4% of the students did not provide any answer for this question. change in elementary school students’ misconceptions / cañada, gómez, rodríguez, niño, acedo 505 after the didactic instruction, 95% of the students classified the mixture as heterogeneous (figure 3), which, to some extent, supports the success of the applied strategy. in this case, regarding the explanations provided to justify the selection of heterogeneous in the posttest, 40% of students used phrases related to “mixture where the components are distinguished”. however, it is worthy to mention that an important percentage of the students that selected heterogeneous in the post-test, namely 28%, justified their selection based on the different densities of the mixture components. some of the answers provided by the students in this case were “the mixture is heterogeneous because of the higher density/viscosity of the oil” (figure 4). indeed, the densities of water and olive oil are different but it is not the reason of the formation of a heterogeneous mixture, thus, the answer was considered wrong, since it proved the students’ lack of knowledge about miscibility and immiscibility. esprivalo harrell and subramaniam (2014) have recently found similar misconceptions in a study with elementary pre-service teachers. in this case, according to the authors, a number of participants equated density with buoyancy predicting that heavier objects will sink while lighter objects will float. finally in the posttest, only 5% of the students selected homogeneous, and 5 % of those, justified their selection as “components are distinguished” which supports again, a possible linguistic origin for the misconception. according to our findings, the didactic instruction had a positive effect, and the misconception evolved toward the scientifically correct concept. in fact, different authors have reported that misconceptions describe a rapid evolution in fundamental ideas about chemistry between the ages of 6 and 12, but only very slow change thereafter, in spite of intensive instruction in chemistry (ngai, sevian & talanquer, 2014). figure 4. reasons provided by students to justify their classification of the mixture oil/water as homogeneous or heterogeneous in pre-test and post-test international electronic journal of elementary education, 9(3), 499-510, march, 2017 506 in the second question, students were asked to select whether milk is a pure substance or a homogeneous mixture. before instruction (pre-test) 86% of the students selected wrongly that milk is pure substance. figure 5 shows the percentages of selected choices in the pre and post-test and the evolution of the reasons provided by students to justify their selection, before and after the instruction. within the reasons they provided supporting this selection, 43% of students identified milk as a pure substance, because of its natural origin, with answers kind of: “because milk comes from cow” or “because a mixture is to mix something and the milk is natural”. these answers show that for students the terms “natural” and “pure” have the same meaning. the origin of this mistake could be influenced by the fact that milk is normally advertised on tv as “pure milk from cow” contributing to people misunderstanding of both concepts. on the other hand, their visual experience leads the milk to be classified as a pure substance because they cannot distinguish its components, they can only see a white liquid. the second most common reason supporting the selection of pure substance in the pretest was that milk is formed by a single component, and this was the explanation provided by 14% of students selecting “pure substance” in the pretest. only 14% of the students identified milk like a mixture in the pre-test but in this case they were no able to provide a solid justification for their selection. these results are similar to the obtained by martínez losada, garcía barros and rivadulla lópez (2009). both studies highlight the difficulty for students to differentiate a pure substance from a homogeneous mixture. after instruction, 65% of students classified properly the product (figure 5). however, the most important advance was found in the justifications provided by students as support of their selection of “mixture”, because 65% of them correctly argued “milk is a mixture because is formed by several components”. figure 5. reasons provided by students to justify their classification of milk as pure substance or mixture in pre-test and post-test. thirdly, students were asked to classify twelve products as pure substances or mixtures. the products were strategically selected and they were yoghurt, salt, coke, diamond, change in elementary school students’ misconceptions / cañada, gómez, rodríguez, niño, acedo 507 water, copper, ice, iron, olive oil, granite stone, silver, and sand. results of the classification are summarized as bar diagrams in figure 6, where it is shown the percentage of election for each product before and after the didactic instruction. regarding pure substances, the obtained results showed that students had problems prior to instruction to classify iron, copper and silver as pure substances, probably because those materials are normally found industrially manufactured. when students were asked to justify their classification it was found that the origin of the products, i.e. natural or manufactured, remained versus composition. students used justifications such as “they are processed substances” or “it is artificial” to justify the election of iron, copper and silver as mixtures. namely, 19% of students based their selection of pure substances before instruction on “natural origin”, 19% based it on answers like “no manufactured”, 48% were not able to give a reason for their selection of pure substances, and 10% of students gave other sorts of reasons. salt was also wrongly classified like mixture by more than 50% of the students in the pre-test (figure 6). after instruction, percentages of the right choice for pure substances increased in almost all cases, except for water and ice that suffered a weak regression. an explanation could be found that during instruction students worked with mineral water and tap water as homogeneous mixtures. regarding the justifications that students gave to their selection of pure substances, the percentage of students using a correct reason, such as “it is formed by one component”, increased from 5% in the pre-test to 76% in the post-test, nevertheless, even after instruction 10% of students continued justifying their selections like pure substances based on the natural origin, although the percentage of students basing their selection on this reason drops from 19% in the pre-test to 10% in the post-test. in the case of mixtures, before instruction students had difficult in classifying olive oil, granite stone and sand as mixtures. the analysis of the reasons gave by the students to support their selection showed that again, the natural origin prevailed over visual assessment, especially in the cases of granite stone and sand in which different components of the mixture can be visually observed. namely, in the pre-test, 38% of students based their selection of mixtures on reasons like “manufactured”, only 10% based it on the right reason “it is formed by several components” and 43% were not able to give a reason to support their selection of mixtures. figure 6. bars diagram representing the answers of the students to the classification pure substance (left)/mixture (right). after instruction the percentage of students classifying properly mixtures increased considerably in all cases, except for olive oil, in which case the percentage of students classifying it like mixture after instruction increased very weakly with regard to the pretest. finally, it is worthy to mention that after instruction, the percentage of students international electronic journal of elementary education, 9(3), 499-510, march, 2017 508 basing their selection of mixtures on the right reason “it is formed by several components” increased up to 85%, and only 10% of students continued stuck in the reason “manufactured”. this, in a way, demonstrates the success of the developed two-session instruction strategy. similar findings and misconceptions have been previously reported by martín del pozo and galán martín (2012) and by furió-mas, calatayud and barcenas (2007). ben-zvi, eylon and silberstein (1986) tried to explain the origin of the misconceptions in this topic taking into account that students think of a compound as a random mixture of atoms because they do not have a right microscopic representation of the substance structure concept. conclusions the presence of misconceptions in students of fifth grade elementary education has been studied. according to the results shown in this research, students did not differentiate between a pure substance and a mixture, especially if the mixture is a natural product like milk. hence, students had a restricted concept of mixtures that was focused on what they perceive and it excludes products that are naturally mixed, including heterogeneous samples like granite stone or sand. in addition students had difficulties with respect to the distinction between homogeneous and heterogeneous mixtures. therefore, from this study we conclude that the concepts of pure substance and mixture are difficult to assimilate by the students. overcoming misconceptions is possible if a proper instruction is used in class. according to the results shown in this manuscript, a two-session instruction strategy, consisting of an interactive overhead presentation followed by a lab workshop session has been proved to be effective. the teacher guidance during the instruction fostered the students to confront their initial knowledge with the scientific one. as a result, students realize that different or additional conceptions were needed to explain and classify the composition of matter. finally, instruction efforts must be provided to promote the change of students’ misconceptions into more accurate scientific conceptions. this change must be addressed in elementary education because these ideas persist and are shaped as mental structures that impede the scientific learning. this is an initial study into the use of practical activities in elementary school science class, and the results encourage future studies with larger sample size and different science topics. • • • acknowledgements authors thank ministry of economy and competitiveness of spain (edu2016-77007-r) and european regional development fund (gr15009 of govern of extremadura) references akgun, a., & aydin, m. a. (2010). cross-age study on the understanding of chemical and physical change and their components. asian journal of chemistry, 22(5), 3541-3548. change in elementary school students’ misconceptions / cañada, gómez, rodríguez, niño, acedo 509 ari, e., kizilaslan tunçer, b., & demir, m. k. (2016). primary school teachers’ views on constructive classroom management. international electronic journal of elementary education, 8(3), 363378. bedir, g. (2015). perception of teaching efficacy by primary and secondary school teachers. international electronic journal of elementary education, 8(1), 41-54 ben-zvi, r., eylon, b.s., & silberstein, j. (1986). is an atom of copper malleable? journal of chemical education, 63(1), 64-66. calik, m., ayas, a., & coll, r.k. (2007). enhancing pre-service elementary teachers’ conceptual understaning of solution chemistry with conceptual change text. international journal of science and mathematics education, 5(1), 1-28. cañada, f., álvarez, r., arévalo, m.j., gil, m.v., cubero, j. & ortega, l. (2012). previous ideas on pure substances and mixtures of primary education students. iceri2012 proceedings, pp. 45984602. dağdelen, o., & kösterelioğlu, i. (2015). effect of conceptual change texts for overcoming misconceptions in “people and management” unit. international electronic journal of elementary education, 8(1), 99-112. durmus, j., & bayraktar, s. (2010). effects of conceptual change text and laboratory experiments on fourht grade students’ understanding. journal of science education and technology, 19, 498504. esprivalo harrell, p., & subramaniam, k. (2014). teachers need to be smarter than a 5th grader: what pre-service teachers know about density. electronic journal of science education, 18(6), 1-23. fernández-gonzález, m. (2013). idealization in chemistry: pure substance and laboratory product. science and education, 22, 1723-1740. furió-más, c., calatayud, m. l., & bárcenas, s. l. (2007). surveying students' conceptual and procedural knowledge of acid-base behavior of substances. journal of chemical education, 84, 1717-1724. harahan, m. (1994). students’ beliefs and learning environments: developing a survey of factors related to conceptual change. research in science education, 24(1), 156-165. horton, c. (2007). student alternative conceptions in chemistry. california journal of science education, 7(2), 18-28. johnson, p. (2005). the development of children’s concept of a substance: a longitudinal study of interaction between curriculum and learning. research in science education, 35, 41-61. johnstone, a. h. (2000). teaching of chemistry logical or psychological? chemistry education research and practice, 1(1), 9-15. kingir, s., geban, o., & gunel, m. (2013). using the science writing heuristic approach to enhance student understanding in chemical change and mixture. research in science education, 43, 1645-1663. kleickmann, t., tröbst, s., jonen, a., vehmeyer, j., & möller, k. (2016). the effects of expert scaffolding in elementary science professional development on teachers’ beliefs and motivantions, instructional practices, and students achievement. journal of education psychology, 108(1), 21-42. leach, j., driver, r., millar, r., & scott, p. (1997). a study of progression in learning about 'the nature of science': issues of conceptualisation and methodology. international journal of science education, 19(2), 147-166. lee, g., & byun, t. (2012). an explanation for the difficulty of leading conceptual change using a counterintuitive demonstration: the relationship between cognitive conflict and responses. research in science education, 42(5), 943-965. international electronic journal of elementary education, 9(3), 499-510, march, 2017 510 martín del pozo, r., & galán martín, p. (2012). los criterios de clasificación de la materia inerte en la educación primaria: concepciones de los alumnos y niveles de competencia. revista eureka sobre enseñanza y divulgación de las ciencias, 9(2), 213-320. martínez losada, c., garcía barros, s., & rivadulla lópez, j. (2009). qué saben los/as alumnos/as de primaria y secundaria sobre los sistemas materiales. cómo lo tratan los textos escolares. revista electrónica de enseñanza de las ciencias, 8, 137-155. martins teixeira, f., & moura bezerra sobral, a.c. (2010). como novos conhecimentos podem ser construídos a partir dos conhecimentos prévios: um estudo de caso. ciência & educação, 16(3), 667-677. ngai, c., sevian, h., & talanquer, v. (2014). what is this substance? what makes it different? mapping progression in students' assumptions about chemical identity. international journal of science education, 36(14), 2438-2461. oliveira, s. r., gouveia, v. p., & de cuadros, a. l. (2009). uma reflexão sobre aprendizagem escolar e o uso do conceito de solubilidade/miscibilidade em situações do cotidiano: concepções dos estudantes. química nova na escola, 31(1), 23-30. pine, k., messer, d. & st john, k. (2001). children´s misconception in primary science: a survey of teharcher´s views. research in science & technological education, 19(1), 79-90. posner, g.j., strike, k.a., hewson, p.w. & gertzog, w.a. (1982). accommodation of a scientific conception: toward a theory of conceptual change. science education, 66, 211-227. rubio cascales, j. (2010). qué sabe el alumnado que acaba la educación primaria sobre las mezclas de sustancias. in atas do xxiv encuentro de didáctica de las ciencias experimentales, (p.496506, jaén, spain). stains, m. & sevian, h. (2014) uncovering implicit assumptions: a large-scale study on students’ mental models of diffusion. research in science education, 45(6), 807-840. taber, k. s. (2001). building the structural concepts of chemistry: some considerations from educational research. chemistry education research and practice, 2(2), 123-158. taber, k. s. (2008). exploring conceptual integration in student thinking: evidence from a case study. international journal of science education, 30(14), 1915-1943. taber, k. s. (2009). challenging misconceptions in the chemistry classroom: resources to support teachers. educaciò química, 4, 13-20. treagust, d. f. & duit, r. (2008). conceptual change: a discussion of theoretical, methodological and practical challenges for science education. cultural studies of science education, 3, 297-328. tümay, h. (2016). emergence, learning difficulties, and misconceptions in chemistry undergraduate students’ conceptualizations of acid strength. science and education, 25, 21-46. vogelezang, m. j. (1987). development of the concept 'chemical substance'some thoughts and arguments. international journal of science education, 9(5), 519-528. microsoft word iejee_5_2_shtulman_checa international electronic journal of elementary education, 2012, 5(1), 27-46. issn:1307-9298 copyright © iejee www.iejee.com parent-child conversations about evolution in the context of an interactive museum display andrew shtulman ∗∗∗∗ occidental college, los angeles, usa isabel checa occidental college, los angeles, usa received: 14 august 2012 / revised: 16 october 2012 / accepted: 16 october 2012 abstract the theory of evolution by natural selection has revolutionized the biological sciences yet remains confusing and controversial to the public at large. this study explored how a particular segment of the public – visitors to a natural history museum – reason about evolution in the context of an interactive cladogram, or evolutionary tree. the participants were 49 children aged four to twelve and one accompanying parent. together, they completed five activities using a touch-screen display of the phylogenetic relations among the 19 orders of mammals. across activities, participants revealed similar misconceptions to those revealed by college undergraduates in previous studies. however, the frequency of those misconceptions was attenuated by the level of parental engagement, particularly the frequency of turn-taking between parents and children. overall, these findings suggest that evolutionary reasoning may be improved by the kinds of collaborative discussions fostered by interactive museum displays, so long as the affordances of those displays encourage multi-user interactions. keywords: conceptual development, evolution understanding, parent-child conversation, informal learning environments, science education. introduction in 1996, the u.s. national academy of sciences identified evolution as one of five “unifying concepts and processes” that should be taught in all grades, from k through 12 (national research council, 1996). the rationale behind this recommendation was that “evolution is the central organizing principle that biologists use to understand the world. to teach biology without explaining evolution deprives students of a powerful concept that brings great order and coherence to our understanding of life” (p. 3, national research council, 1998). despite the force of this recommendation, many schools continue to fail to teach evolution in any grade (griffith & brem, 2004), and many americans continue to deny the very fact of evolution, particularly human evolution (miller, scott, & okamoto, 2006; newport, 2010). ∗ andrew shtulman, department of psychology, occidental college, los angeles, ca 90041. phone: 323-259-2633. e-mail: shtulmanoxy.edu international electronic journal of elementary education vol.5, issue 1, 27-46, 2012 28 one of the problems underlying, if not exacerbating, public denial of evolution is that most people fail to understand what evolution actually is and how evolution actually occurs (shtulman & calabi, 2012). recent research has found that individuals of all ages and educational backgrounds tend to construe evolution as a kind of holistic transformation, by which organisms are predisposed to produce offspring more adapted to the environment than they were at birth (shtulman, 2006; shtulman & calabi, in press). on this erroneous theory, the sole mechanism behind species adaptation is need: if a species needs to adapt, then it will adapt (bishop & anderson, 1990; brumby, 1984; southerland, abrams, cummins, & anzelmo, 2001). selection plays no causal role on this view, which, as a result, bears more resemblance to pre-darwinian theories of evolution than post-darwinian ones (mayr, 1982). developmentally, this view appears to be an outgrowth of an early emerging “essentialist” construal of biological kinds, in which an organism’s outward appearance and behavior is determined by some kind of internal force, or “essence,” conferred from parent to child at birth (gelman, 2003; hatano & inagaki, 1994; solomon & zaitchik, 2012). while essentialism may be useful for reasoning about the properties of individual organisms (e.g., gelman & coley, 1990; waxman, medin, & ross, 2007), it has been shown to be detrimental for reasoning about the properties of entire species, as it leads students to overvalue variation between species and undervalue variation within a species (nettle, 2010; shtulman & schulz, 2008). as a consequence, students have difficulty understanding mechanisms of change that operate specifically over the variation within a species, namely, natural selection. to date, numerous studies have documented the nature of students’ essentialist, needbased views of evolution (for reviews, see gregory, 2009; sinatra, brem, & evans, 2008). less attention, however, has been paid to the ways in which alternative views of evolution manifest themselves in everyday discourse and everyday interactions. the present study attempted to explore this issue in the context of parent-child conversations at a natural history museum. in particular, we sought to elicit conversations about the phenomena represented by one of the most canonical depictions of evolutionary change: the cladogram. more commonly referred to as an “evolutionary tree,” cladograms are branching diagrams that depict patterns of common ancestry among three or more groups of organisms, or taxa. a sample cladogram, depicting the evolutionary relations among primates, can be seen in figure 1. even though cladograms are ubiquitous in biology textbooks (catley & novick, 2008) and natural history museums (torrens & barahona, 2012), they are notoriously difficult to interpret, partly because they contain unfamiliar notational conventions (novick & catley, 2007) and partly because they are amenable to inaccurate, essentialist interpretations of evolutionary change (shtulman, 2006). drawing on recent empirical investigations of “tree thinking” in introductory biology students, gregory (2008) outlined 10 such misconceptions: 1. interpreting taxa on one side of a cladogram as “higher” or “lower” than those on the other side. 2. interpreting the longest line in a cladogram as the “main line” from which other taxa have deviated or side-tracked. 3. inferring information about relatedness from the ordering of a cladogram’s terminal nodes rather than from its branches. 4. interpreting cladograms as representations of morphological similarity rather than common ancestry. 5. interpreting some taxa in a cladogram as the ancestors of other taxa rather than interpreting all taxa as “siblings” or “cousins.” parent-child conversations about evolution in the context of an interactive museum display / shtulman & checa 29 6. interpreting the length of the branches in a cladogram as measures of evolutionary change (or lack thereof). 7. inferring that the taxa on one side of a cladogram appeared, in their current form, earlier than those on the other side of the cladogram. 8. interpreting the length of the longest branch of the cladogram as a measure of time. 9. interpreting the number of intervening nodes between two taxa as a direct measure of their relatedness. 10. interpreting internal nodes in a cladogram as representing precise moments of speciation, with little to no change occurring before or after that point in time. many of the misconceptions are overlapping (e.g., 1 and 7) and some are mutually exclusive (e.g., 6 and 10), but all represent illegitimate inferences from the information at hand. as an illustration, consider the cladogram depicted in figure 1. the only information this diagram provides is information about common ancestry – namely, that humans share a common ancestor with chimpanzees more recently than with any of the other primates, that humans and chimpanzees share a common ancestor with gorillas more recently than with any of the other primates, and so forth. nevertheless, most people are prone to infer that: 1. humans are more highly evolved than other primates. 2. human evolution represents the “main line” of evolution, whereas the evolution of other primates represents sidetracks from this main line. 3. chimpanzees are related to gorillas more closely than humans are related to gorillas (because the nodes of the former pair are adjacent but the nodes of the latter pair are not). 4. humans are most similar to chimpanzees (the closest node to humans) and least similar to new world monkeys (the farthest node from humans). 5. each primate is the descendent of the primate on its left and the ancestor of the primate on its right. 6. humans have undergone the more evolutionary change than other primates (because their connection to the root node is longest). 7. each primate appeared, in its current form, earlier than the primate to its right. 8. each primate is older than the primate to its right. 9. humans are related to orangutans less closely than chimpanzees are related to orangutans (because of differences in the number of intervening nodes). 10. chimpanzees came into being instantaneously at the point denoted by the rightmost node. these inferences are not just logically unwarranted; they are also empirically incorrect. indeed, inferences like 4 and 8 are not even meaningful on a scientific understanding of speciation, let alone correct or incorrect. misconceptions of this nature have been documented both in the classroom (baum, smith, & donovan, 2005; meir, perry, herron, & kingsolver, 2007) and in carefully controlled international electronic journal of elementary education vol.5, issue 1, 27-46, 2012 30 figure 1. cladogram depicting the phylogenetic relations among seven primates (adapted from gregory, 2008). laboratory studies (catley, novick, & shade, 2010; novick & catley, 2007; novick, shade, & catley, 2010). however, all such studies have involved college undergraduates, and no studies, to our knowledge, have explored the prevalence of macroevolutionary misconceptions in a non-college population (though see berti, toneatti, & rosati, 2010, and samarapungavan & wiers, 1997, for research on children’s misconceptions about other aspects of evolution). to address this gap in the literature, we explored the prevalence of macroevolutionary misconceptions among preschooland elementary-school-aged children visiting a natural history museum with their parents. this sample served to broaden the scope of inquiry not only in terms of age but also in terms of context, as each child was interviewed as part of a dyad with his or her parent. in other words, children’s reasoning about evolution was explored within the naturalistic context of a parent-child conversation. previous research on parent-child conversation suggests that conversations of this type can be a double-edged sword, with some aspects of the conversation scaffolding learning and other aspects obscuring, or even obstructing, learning. for instance, jipson and callanan (2003) found that parents of preschool-aged children typically use the word “grow” in a literal sense, to refer to biological changes in size (e.g., “the mushroom grew taller”), but also occasionally use the word in a metaphorical sense, to refer to non-biological changes in size (e.g., “the rock grew bigger”), yielding a linguistic signal that is reliable yet noisy nonetheless. in a similar vein, rigney and callanan (2011) found that parents at a marine science center ascribed biological properties to typical animals (e.g., sharks) no more often than they ascribed biological properties to atypical animals (e.g., anemones), potentially reinforcing the inclusion of atypical animals in the category of living things, a notoriously difficult concept to acquire (see, e.g., anggoro, waxman, & medin, 2008). however, these same parents ascribed intentional states, like beliefs and desires, to both typical and atypical animals significantly more often than their children did, thus modeling a scientifically inappropriate form of reasoning. in short, parental input can serve as a source of accurate reasoning but by no means guarantees accurate reasoning (see also gleason & schauble, 2000; gunderson & levine, 2011). parent-child conversations about evolution in the context of an interactive museum display / shtulman & checa 31 in the present study, we explored how parents converse with their children about evolution, which we anticipated would be a difficult topic for both parties. we elicited these conversations by recruiting parent-child dyads from the floor of the los angeles natural history museum to complete a series of activities centered around an interactive cladogram. two questions were of primary interest. first, how well do parent-child dyads interpret the information contained in cladograms, given that they are perhaps the most prevalent representation of evolutionary change in modern culture (torrens & barahona, 2012) yet are largely misunderstood by most biology students (catley et al., 2010)? research by evans et al. (2010) and spiegel et al. (2012) suggests that museum visitors hold a variety of preconceptions about micro-evolutionary change, some consistent with the principle of natural selection (e.g., need-based reasoning) and some inconsistent with it (e.g., creationist reasoning). still, it remains an open question as to how museum visitors interpret displays representing macro-evolutionary phenomena, such as speciation, extinction, and common descent. second, what factors influence the accuracy of dyads’ reasoning? three candidate factors were identified from prior research on shared scientific thinking: (a) the child’s gender, (b) the child’s age, and (c) the dyad’s overall style of interaction. in terms of gender, we predicted that dyads with male children would outperform dyads with female children, owing to the finding that parents are more likely to explain scientific phenomena to their sons than to their daughters (crowley, callanan, tenenbaum, & allen, 2001; diamond, 1994) and might thus devote more attention to their sons in the activities at hand. in terms of age, we predicted that dyads with older children would outperform dyads with younger children, owing to the finding that older children are generally more familiar with evolutionary ideas than younger children (berti et al., 2010; legare, lane, & evans, in press; spiegel et al., 2012) and might thus comprehend the purpose of the activities more thoroughly. finally, in terms of interaction style, we predicted that dyads exhibiting higher levels of collaboration would outperform dyads exhibiting lower levels of collaboration, owing to the finding that parents at a science museum tend to hone their children’s exploration of the exhibits in conceptually constructive ways (crowley, callanan, jipson, galco, topping, & shrager, 2001; tare, french, frazier, diamond, & evans, 2011), even if parents do occasionally provide conflicting or confusing input. to preview our results, we found that parent-child dyads espoused the same kinds of misconceptions documented among college-level biology students. however, the frequency of such misconceptions varied by dyad, with dyads exhibiting low levels of collaboration espousing more misconceptions than those exhibiting higher levels of collaboration. this effect of dyad interaction was larger and more consistent than any of the other effects documented and thus has potentially important implications for both evolution education and informal science learning. method participants the participants were 49 parent-child dyads recruited from the “age of mammals” exhibit at the los angeles natural history museum (see figure 2a). we chose the age of mammals exhibit because it is thoroughly grounded in evolutionary findings and evolutionary principles and thus served as an ideal venue for eliciting conversations about evolution. all parents accompanying children between the ages of four and twelve were approached by international electronic journal of elementary education vol.5, issue 1, 27-46, 2012 32 figure 2. (a) the “age of mammals” exhibit at the los angeles natural history museum; (b) the interactive cladogram used to elicit participants’ evolutionary reasoning. the research team as they entered the exhibit and were invited to participate in a study. they were informed that the study consisted of how parents and children communicate about complex biological concepts, like evolution and common descent. fifty-five parent-child dyads consented to participate, but only 49 completed the entire study. the ages of the participating children were evenly distributed across the age range sampled; half were between the ages of four and eight (m = 6.8, sd = 1.3, n = 24) and half were between the ages of nine and twelve (m = 10.3, sd = 1.0, n = 25). as for gender, 46% of the children were female, and 46% of the accompanying parents were female. visitors who took part in the study did not receive any monetary compensation for their participation. a b parent-child conversations about evolution in the context of an interactive museum display / shtulman & checa 33 procedure the age of mammals exhibit features a total of 240 biological specimens, including both articulated skeletons of extinct species and taxidermied specimens of extant species. the specimens are organized into collections intended to illustrate three key principles: “continents move. climates change. mammals evolve.” these principles are also illustrated in the form of various touch-screen, computerized displays. one such display, an interactive cladogram, comprised the focus of the present study (see figure 2b). all parent-child dyads who consented to participate were taken directly to the interactive cladogram, briefly familiarized with its features and functions, and then led through a series of five activities exploring the information contained within. technically, the display was not a “cladogram,” in the biologist’s sense of the word, because it attempted to represent more than just patterns of common ancestry; it also represented the times that taxa diverged and whether the taxa are extant or extinct. nevertheless, we will refer to the display as a cladogram because its primary function was to depict patterns of common ancestry. five display-based activities were designed to probe participants’ understanding of macroevolution and the ways in which macroevolutionary relations are represented in cladograms. these activities, which were derived from previous research on students’ macroevolutionary misconceptions (e.g., catley et al., 2010; gregory, 2008), are described below in relation to participants’ actual responses. all responses were video recorded and transcribed at a later date. at the completion of the five-activity interview, participants were encouraged to explore other areas of the exhibit, but their conversations at those other areas were not recorded. coding measures of engagement. each interview was transcribed by two independent coders, the first producing a written record of all utterances and actions and the second editing or embellishing that record to account for details the first coder appeared to have missed. the coders then summed the number of distinct utterances and actions for each dyad member and each activity, resulting in 490 engagement scores (5 scores per participant for 98 participants). any self-contained thought, question, or response was coded as a distinct utterance, even if that thought did not take the form of a complete sentence (e.g., “yes” or “ok” were counted as distinct utterances). likewise, any attempt to deliberately manipulate the display (via tapping or scrolling), or manipulate a partner’s view of the display (via pointing or waving), was coded as a distinct action. extended sequences of dialogue or activity were broken into distinct utterances and actions, with the exception of experimenter requested actions (e.g., “point to the monkey, the tree shrew, and the flying lemur”), which were counted simply as one action so as not to inflate engagement scores for some activities relative to others. coders agreed on their tabulation of distinct utterances and actions 93% of the time, and all disagreements were resolved through discussion. measures of accuracy. participants’ responses to each activity were assigned a score that ranged from -1 to +2. responses that revealed a positive misconception about the material at hand received a score of -1. responses that were vague, ambiguous, or equivocal (including “don’t know” responses) received a score of 0. responses that revealed a partial understanding of the material at hand received a score of +1. and responses that revealed a full understanding of the material at hand received a score of +2. what constituted a correct response or an incorrect response is discussed below in relation to the corresponding activity. it should be noted that each dyad received a single score per activity, rather than separate scores for each dyad member, as almost all dyads offered a single response by the international electronic journal of elementary education vol.5, issue 1, 27-46, 2012 34 conclusion of almost all activities. two coders independently assigned scores to all 245 responses (5 responses per dyad for 49 dyads). overall agreement was 82% (cohen’s κ = .76), and all disagreements were resolved through discussion. results how well do parent-child dyads reason about evolutionary phenomena? below we describe each activity used to elicit participants’ evolutionary reasoning and the nature of their responses, followed by an analysis of participants’ engagement with each activity and how that engagement was related to response accuracy per activity (summed across dyads). we address the question of whether, and how, engagement influenced response accuracy per dyad (summed across activities) in the following section. activity 1: ordering. the introductory screen of the interactive cladogram featured all 19 orders of mammals, arranged in a semicircle with primates in the center (see figure 2b). the first activity was designed to elicit participants’ beliefs about the necessity, and potential flexibility, of this particular ordering. as mentioned previously, the ordering of the taxa in a cladogram is, to a large extent, arbitrary. while taxa that share a most recent common ancestor must be adjacent (e.g., chimpanzees and humans), their ordering relative to one another is arbitrary (i.e., chimpanzees can be on the left and humans on the right or humans can be on the left and chimpanzees on the right). the representation of nested taxa is governed by the same constraint, meaning that entire groups of taxa can be swapped with one another so long as the underlying branching relations are preserved. thus, any one taxon could appear at any point in the row of terminal nodes, and any cluster of taxa could be reordered in many different ways. we attempted to elicit participants’ understanding of the ordinal relations among taxa by asking them to locate three particular taxa – the monkey, the tree shrew, and the flying lemur – and reflect on the ordering of those taxa. specifically, we asked, “does it matter that the flying lemur is on the left, the tree shrew is in the middle, and the monkey is on the right? or could they be reordered so that the monkey is on the left, the flying lemur is in the middle, and the tree shrew is on the right? why or why not?” because lemurs and shrews share a common ancestor with each other more recently than either shares with monkeys (as depicted by the relevant branching relations), the only constraint on ordering was that the lemur and the shrew had to be adjacent, which was true of the hypothetical ordering we asked participants to consider. nevertheless, 17 dyads claimed that the taxa could not be reordered, justifying their judgment with an affirmation that ordering matters (scored -1). most dyads (n = 23) were unsure whether or not the taxa could be reordered (scored 0), and only a few dyads (n = 9) claimed the taxa could be reordered but were unable to provide an adequate justification for their judgment (scored +1). none of the dyads provided both a correct judgment (that the taxa could be reordered) and a correct justification (that only the branching relations matter), and thus none received a score of +2. on the contrary, dyads that received a score of 0 or 1 seemed genuinely unsure of whether, and how, the ordinal properties of the display reflected information about the species’ evolutionary origins. activity 2: branching. participants’ understanding of the branching relations in a cladogram was elicited more directly in the second activity. participants were asked to locate two nonplacental mammals – the kangaroo and the platypus – and to read about their features in a pop-up window that appeared upon touching each. we chose non-placental mammals as our target taxa for this activity because their divergence from the other mammals occurred earliest (around 170 million years ago for egg-laying mammals and 130 million years ago for parent-child conversations about evolution in the context of an interactive museum display / shtulman & checa 35 marsupials) and was thus highly salient. after participants had read about the features of kangaroos and platypuses, they were asked, “how are these two mammals different from the other mammals in the tree? is this difference reflected in the tree itself somehow?” most dyads were able to identify a genuine morphological or geographic difference between the non-placental mammals and the other mammals (e.g., “only kangaroos have pouches and only platypuses lay eggs,” “they’re the only ones that live in australia”), but very few were able to identify how that difference was reflected in the tree. six dyads claimed the difference was not reflected in the tree (scored -1); 15 dyads claimed they were unsure whether or not the difference was reflected in the tree (scored 0); and 23 dyads claimed the difference was reflected in the tree but did not refer to the branching relations, e.g., “they [the labels] are different colors” or “they [the animals] just look different” (scored +1). only 5 dyads correctly identified the branching relations as the relevant form of representation, e.g., “this one’s branching out, completely separate from these” (scored +2). activity 3: speciation. one of the unique features of the interactive cladogram was a slider at the bottom of the screen for manipulating the timeline, allowing users to scroll between the beginning of the divergence of the 19 orders of mammals (65 million years ago) and the present day. moving back in time “shrank” the cladogram such that branching events that occurred after that time no longer appeared on the screen. we used this feature of the display to elicit participants’ beliefs about the origin of species. specifically, we asked participants to move the slider to 40 million years, which caused the taxon representing an extinct, hippo-like creature – the paleoparadoxiid – to disappear from the screen. we then asked, “did you see that the paleoparadoxiid disappeared from the tree? why do you think that happened? what might have occurred between 40 million years ago and 30 million years ago that led to the appearance of paleoparadoxiid?” of interest was whether participants could identify a biologically plausible cause of divergence – i.e., geographic isolation, reproductive isolation, or unique selection pressures. five dyads did, in fact, cite such a factor, e.g., “maybe the climate changed” (+2). fourteen dyads noted that the paleoparadoxiid must have evolved during the time period of interest but were unclear on what factors may have driven its divergence from its closest relative (scored +1). the majority of dyads (n = 21) were unsure of what the appearance and disappearance of the paleoparadoxiid icon was supposed to represent biologically (scored 0), and the remaining dyads (n = 9) acknowledged that the appearance and disappearance of the paleoparadoxiid icon was intended to represent a speciation event but cited biological implausible means of speciation, e.g., “the sea cow and the hyrax had a baby” or “it came from the ground” (scored -2). activity 4: relatedness. another unique feature of the interactive cladogram was that users could “launch the story” of a particular taxon, which opened a window containing detailed information about the taxon’s habitat, diet, and morphology. also contained in these windows were additional cladograms, depicting the relations among other species within the taxa not explicitly depicted in the main display. we used this feature as a vehicle for eliciting participants’ understanding of the relatedness of species whose morphological features seemingly belie their evolutionary origins: camels, pigs, and whales. participants were asked to “launch the story” for the camels, which then brought up a cladogram depicting pigs on the left, camels in the middle, and whales on the right. the branching relations among these taxa indicated that camels are more closely related to whales than to pigs; the outward behavior and morphology of these animals, however, would suggest that camels are more closely related to pigs. faced with this tension, participants were asked to determine whether camels were more closely related to pigs or to whales and to explain how international electronic journal of elementary education vol.5, issue 1, 27-46, 2012 36 they were able to discern that fact from the tree. the story itself, we should note, did not explicitly reference this tension, nor did it give participants any clues as to which of the three species were most closely related. the vast majority of dyads (n = 39) claimed that the tree indicated that camels are more closely related to pigs than to whales (scored -1). three dyads claimed not to be able to discern what the tree indicated about relatedness (scored 0); four claimed that the tree indicated that camels are more closely related to whales but were unable to explain how it indicated that (scored +1); and three claimed the tree indicated that camels are more closely related to whales and were also able to justify that inference by reference to the branching relations (scored +2). activity 5: extinction. the interactive cladogram contained another feature not represented in standard cladograms: it allowed participants to select images of the specimens on view next to the display and learn about those specimens in the context of the phylogenic relations and historical timeline contained within. we used this feature to elicit participants’ reasoning about the relation between extinct species and extant species. participants were asked to navigate to the “mammals on display” tab, select the “entelodont” (an extinct, pig-like creature), and decide (a) whether it still exists and (b) where it might fit in the cladogram in the main display. the entelodont’s status as an extinct species could be discerned from the sliding timeline at the bottom of the screen, which made the entelodont non-selectable if moved to a point past its estimated date of extinction. it could also be discerned from the information contained in the pop-up window describing the species. thus, the question of whether or not the entelodont is extinct was of less interest than the question of where it might fit in the cladogram, which depicted 19 extant species but only one extinct species (the paleoparadoxiid). as expected, virtually all dyads were able to discern that the entelodont is extinct. only 19 dyads, however, were able to discern that it would be located among the ungulates (scored +2). twenty-two dyads indicated that the entelodont would be located in some other part of the display, typically the bottom, near the cladogram’s root (scored +1); 6 dyads were unsure of where the entelodont would be located (scored 0); and 2 dyads claimed that the entelodont would not fit into the cladogram because it was extinct and extinct species cannot be represented in cladograms (scored -1). patterns of engagement. participants’ engagement with each activity was operationalized as the number of distinct utterances and actions produced during that activity. engagement scores were calculated separately for each dyad member and each activity. to determine whether engagement scores varied by activity or by dyad composition, we ran a repeatedmeasures analysis of variance (anova) of the engagement scores in which activity (ordering, branching, speciation, relatedness, extinction) and dyad member (parent, child) were treated as within-participants variables and parent’s gender (male, female), child’s gender (male, female), and child’s age (4-8, 9-12) were treated as between-participants variables. this analysis revealed significant effects of activity (f(4,164) = 5.08, p < .01) and dyad member (f(1,41) = 30.74, p < .001), but no significant effects of parent’s gender, child’s gender, or child’s age and no significant interactions either. the effects of activity and dyad member on engagement scores are illustrated in figure 3a. while dyads produced more utterances and actions for some activities (e.g., branching) than for others (e.g., extinction), this effect was small in comparison to the effect of dyad member. across activities, the mean engagement score for children was 4.6 (sd = 1.4, range = 1.2 to 8.8), whereas the mean engagement score for parents was only 1.8 (sd = 2.5, range = 0 to 11.6). in fact, 27 of the 49 parents produced fewer than one utterance or action per activity, parent-child conversations about evolution in the context of an interactive museum display / shtulman & checa 37 despite the fact that the interview was framed, in both verbal and written communications, as “a study of how parents and children converse about complex biological topics, like evolution and common descent.” the remaining 22 parents, however, tended to produce as many utterances and actions as their children, if not more. this variance, while unexpected, ultimately proved fruitful in determining how engagement scores related to response accuracy at the level of the dyad (discussed subsequently). figure 3. (a) mean number of utterances/actions per activity per dyad member; (b) mean accuracy of dyad responses per activity (+ se). ext = extinction, spe = speciation, rel = relatedness, ord = ordering, bra = branching. patterns of reasoning. participants’ mean accuracy on each activity is displayed in figure 3b, ordered by the mean level of engagement with those activities. a repeated-measures parent child ext spe rel ord bra m e a n u tt e ra n c e s /a c ti o n s 2 0 6 4 10 8 a b activity ext spe rel ord bra m e a n r e s p o n s e a c c u ra c y 0.0 -0.5 1.0 0.5 2.0 1.5 -1.0 international electronic journal of elementary education vol.5, issue 1, 27-46, 2012 38 anova revealed not only that accuracy scores differed by activity (f(4,192) = 34.43, p < .01) but also that accuracy scores were quadratically related to mean engagement scores (f(1,48) = 96.41, p < .001, for the quadratic contrast). the source of this quadratic relation is not entirely clear, though one possibility is that the association between accuracy scores and engagement scores may have been moderated by a third variable – task difficulty – such that engagement was negatively correlated with response accuracy for easy tasks but positively correlated with response accuracy for harder tasks. put differently, increased engagement may have been productive for the more difficult tasks but counterproductive for the easier ones. we did not, however, collect any independent measures of task difficulty, so this speculation requires further verification. nevertheless, the two activities that differed most in engagement – activity 5 (extinction) and activity 2 (branching) – yielded opposite correlations between accuracy scores and engagement scores at the level of the dyad. dyads who exhibited more engagement with the branching activity tended to produce more accurate responses (r = .27), whereas dyads who exhibited more engagement with the extinction activity tended to produce less accurate responses (r = -.20). these correlations were not significantly different from zero (given the small sample size), but they were still significantly different from one another (z = 3.25, p < .01), suggesting that the relation between engagement and accuracy may differ depending on the conceptual demands of the task. what factors influenced accuracy of reasoning? dyads differed substantially in the accuracy of their reasoning, from providing responses scored -1 on four of the five activities to providing responses scored +2 on four of the five activities. to determine which factors, if any, were associated with accuracy, we regressed each dyad’s accuracy score against five dyad-specific measures: the child’s age (in years), the child’s gender (0 = female, 1 = male), the parent’s gender (0 = female, 1 = male), the parent’s mean engagement score, and the child’s mean engagement score. only two measures emerged as significant predictors of response accuracy: the child’s age (β = .32, t = 2.39, p < .05) and the parent’s mean engagement score (β = .36, t = 2.61, p < .05). these effects are displayed in figure 4, with child’s age dichotomized as “younger” (4-8) and “older” (9-12) and parent’s engagement scores dichotomized as “low” (m < 1 utterance or action per activity) and “high” (m > 1 utterance or action per activity). dyads with older children produced more accurate responses than dyads with younger children, and dyads with high parental engagement scores produced more accurate responses than dyads with low parental engagement scores. a univariate anova revealed no interaction between child’s age and parental engagement (f(1,45) < 1, ns), though it did reveal that the effect of parental engagement (η2 = .13) was nearly three times as large as the effect of child’s age (η2 = .05). to explore the effect of parental engagement further, we computed an additional measure of engagement: the number of alternations between the child’s contribution to the conversation (utterance or action) and the parent’s contribution. while this measure was strongly correlated with our initial measure of parental engagement (r = .74), it was not entirely redundant with that measure. figure 5 illustrates how the two measures diverged. the transcript shown in 5a contains multiple alternations between parent and child, with parent a actively eliciting information from the child, either from the display or from memory. the transcript shown in 5b, on the other hand, contains far fewer alternations, with parent b generally dominating the conversation. while parents in both dyads produced a parent-child conversations about evolution in the context of an interactive museum display / shtulman & checa 39 figure 4. mean accuracy of dyad responses (+ se) as a function of parental engagement (low vs. high) and child’s age (younger vs. older). similar number of utterances or actions per activity (m for parent a = 5.0, m for parent b = 4.8), the two dyads received very different alteration scores (m for dyad a = 4.8, m for dyad b = 1.8). it should be noted that dyads a and b were two of only 33 dyads for which alternation scores could be computed at all. the remaining 16 dyads included too little parental engagement, typically because the parents in these dyads resigned themselves to simply watching their children complete the activities. it should also be noted that the overall paucity of parental involvement rendered the use of more detailed, content-specific coding schemes (like those used by crowley et al., 2001, or evans et al., 2010) impractical. among the 33 dyads for which alternations could be computed, the mean number of alternations was 14.2 (sd = 10.7, range = 2 to 44), and alternations were strongly correlated with response accuracy (r = .55). alternations actually proved to be a stronger predictor of response accuracy than parental engagement in general (r = .35). but were alternations a unique predictor of response accuracy? we addressed this question by submitting dyads’ mean accuracy scores to a hierarchical regression in which alternation scores were entered subsequent to parental engagement scores. whereas parental engagement scores explained 14% of the variance in response accuracy in the initial model (f(1,31) = 5.00, p < .05), alternation scores explained an additional 17% of variance in the subsequent model (fchange(2,30) = 6.69, p < .01). in fact, the partial correlation between alternation scores and response accuracy, controlling for parental engagement scores, was nearly as large as the zero-order correlation between these two variables (r = .44, p < .01), whereas the partial correlation between parental engagement scores and response accuracy, controlling for alternation scores, was no longer significant (r = .06, ns). the nature of parental engagement thus appeared to be more important than the mere act of engagement. older younger m e a n r e s p o n s e a c c u ra c y 0.0 -0.5 1.0 0.5 2.0 1.5 -1.0 low high parental engagement international electronic journal of elementary education vol.5, issue 1, 27-46, 2012 40 figure 5: examples of high parental engagement involving (a) frequent alternations and (b) infrequent alternations between parent and child contributions. e = experimenter, p = parent, and c = child. actions are denoted in brackets. conclusions evolutionary concepts like speciation, extinction, and common descent are notoriously difficult to understand (gregory, 2008; shtulman, 2006), and canonical representations of those concepts are notoriously difficult to interpret (catley et al., 2010; novick et al., 2010). the a e: select the kangaroo and the platypus and read about their features. p: can you find the kangaroo? c: [hits the kangaroo] p: [reads it to her] ok, now the platypus. you keep looking over it. c: [hits the kangaroo] p: [reads it to her] e: how are these two mammals different from the other mammals in the tree? p: where do their babies grow? c: in their pouch. p: right, what about the platypus? c: i don’t know. p: remember it lays eggs, now where do they come from? c: australia and new guinea. e: is this difference reflected in the tree itself somehow? p: by the colors. b e: select the kangaroo and the platypus and read about their features. p: kangaroo and platypus. c: [leans on display looks around] p: ok, here’s the kangaroo and the platypus. c: [reads them] p: ok, now this one, most other mammals don’t lay eggs, but the platypus is a mammal that lays eggs. that’s a big deal, will you remember that? e: how are these two mammals different from the other mammals in the tree? p: well, the platypus lays eggs and live in the water, and the kangaroo has a pouch and keeps their young in the pouch. e: is this difference reflected in the tree itself somehow? p: well, they are all mammals, but [points to tree, sweeping motion]. hmm. it’s obvious about the platypus... parent-child conversations about evolution in the context of an interactive museum display / shtulman & checa 41 present study sought to determine how parents and children discuss such concepts and interpret such representations in the context of an interactive museum display. we found that parents and children, like the college-aged students tested in previous research, exhibited significant difficulty in interpreting the core feature of a cladogram – namely, the branching relations among its taxa. parents and children also revealed the same kinds of evolutionary misconceptions documented in previous research, including (a) that the ordering of a species in a cladogram carries biologically relevant information, (b) that the morphological overlap between species is a reliable indicator of shared ancestry, and (c) that speciation occurs through a kind of instantaneous transformation or emergence. nevertheless, the frequency of those misconceptions was negatively correlated with the degree to which parents were involved in generating a response, particularly the degree of turn-taking between parents and children. the more often the two collaborated (by this measure), the more often they generated accurate interpretations and explanations of the phenomena at hand. these findings have implications for both the study of evolutionary reasoning and the design of informal learning environments. with respect to evolutionary reasoning, they suggest that collaboration may be an effective means of reducing, or even eliminating, evolutionary misconceptions. previous research by asterhan and schwartz (2007, 2008) found that undergraduates who answered evolution-based problems on their own learned less about evolution, in both the short-term and the long-term, than those who worked in pairs. they also found that the style of a dyad’s interaction influenced learning such that dyads who engaged in argumentation exhibited greater learning gains than those who merely shared information. assuming that turn-taking between parents and children served as a proxy for argumentation, the present study not only replicates asterhan and schwartz’s findings but also extends those findings to (a) populations of different ages and (b) conversations elicited in more naturalistic contexts. that said, the studies by asterhan and schwartz employed controls absent from the present study, including the provision of instruction on how to conduct a critical discussion and the administration of preand post-intervention measures of evolution understanding. controls like these are difficult to implement in a museum setting, where participants’ time and attention are limited resources. it may thus be beneficial to test parent-child dyads in a more controlled (laboratory) setting, particularly for the purpose of assessing learning. while we did, in fact, document a strong relation between the level of interaction between dyad members and the accuracy of their responses, it is unclear whether greater dyadic interaction was a cause, or merely a symptom, of more accurate responding. dyads who entered the museum with better evolution understanding may have engaged in more discussion simply because they were more familiar, or more comfortable, with the topics at hand. at least two considerations militate against this interpretation, however. first, not all forms of engagement were associated with higher response accuracy. children’s engagement scores, for instance, were uncorrelated with response accuracy, as were parental engagement scores after controlling for parent-child alternations. second, engagement was positively correlated with response accuracy for some activities but negatively correlated with others. still, future research should explore the effects of parent-child interaction on learning more directly by including both pretests and posttests and by administering them (separately) to both parents and children. with respect to informal learning environments, our findings suggest that environments that encourage dyadic interaction – particularly parent-child interaction – may be more effective international electronic journal of elementary education vol.5, issue 1, 27-46, 2012 42 at fostering learning than those that encourage more monadic forms of exploration. these findings echo many other findings in the science education literature. van schijndel, franse, and raijmakers (2010), for instance, found that children who were actively coached by their parents during a museum visit engaged more effectively with the exhibits than children who were not coached in this way. tenenbaum, prior, dowling, and frost’s (2010) found that parent-child dyads who explored a museum with a booklet of activities spent more time exploring and discussing the exhibits than dyads who explored the museum in a less structured way. and tare et al. (2011) found that parents’ explanatory behavior at a museum exhibit was positively correlated with their children’s explanatory behavior. two aspects of our findings, however, stand in contrast to these earlier findings. first, the relation between dyads’ level of engagement and accuracy of reasoning was not entirely straightforward. dyads reasoned most accurately for activities in which they were either most engaged or least engaged; intermediate levels of engagement were, on the other hand, associated with the poorest performance (see figure 3). whatever the cause of this effect may be, the effect itself implies that increased engagement is not always productive or beneficial and that learners may need additional guidance in how to allocate their attention to the displays at hand (see, e.g., allen & gutwill, 2004). second, we did not explicitly instruct parents on how to engage with their children during the activities, leaving open the option that parents might chose not to engage at all, and, to our surprise, approximately half adopted that option. anecdotal evidence suggests that what led to such high levels of parental attrition was not confusion about the purpose of the study but the nature of the touch-screen display, which was clearly designed for a single user. if two users touched the screen at the same time, then either the cursor on the display would toggle between the two points of contact or the display would reset itself altogether, closing all pop-up windows and restoring all default settings (e.g., restoring the sliding timescale to “present day”). dyads quickly became aware of this contingency and delegated the task of manipulating the display to a single individual, typically the child. as a result, many parents seemed to become increasingly disengaged over the course of the interview, an observation backed by a significant drop in parental engagement scores from the first activity to the last (m = 1.9, 2.6, 2.0, 1.4, 1.0; linear contrast: f(1,48) = 12.33, p < .01). this unexpected finding highlights a paradox in the use of touch-screen displays increasingly populating the halls of science museums today: such displays are typically designed for a single user, yet, in our study, dyads in which a single member was actively engaged with the display profited less from the display than those in which both members were actively engaged. the conceptual benefits of collaborative activity over solitary activity have been documented in a variety of studies (e.g., craig, chi, & vanlehn, 2009; okada & simon, 1997, schwarz, neuman, & biezuner, 2000). the mechanism behind this effect appears to be the addition of a “social impetus” to explain and justify one’s reasoning. museum displays that can capture or create this kind of impetus would thus seem to be more efficacious than those that cannot. that said, the design of such displays is constrained by additional, pragmatic factors that might effectively limit interactivity, including how long visitors can be expected to use the display, how well visitors can discern the purpose of the display’s affordances, and how constructively visitors can intervene on the phenomena of interest (allen & gutwill, 2004). which considerations to privilege over others is a question that likely merits a different answer for every display. still, our research suggests that, at a minimum, touch-screen displays should be tolerant of multiple points of contact so that parents and children may jointly interact with the display without accidentally erasing one another’s paths of exploration. parent-child conversations about evolution in the context of an interactive museum display / shtulman & checa 43 in conclusion, visitors to a science museum are no more immune to evolutionary misconceptions than other populations. science museums, however, provide unique opportunities for collaboratively discussing evolutionary phenomena, and such discussions appear to help attenuate evolutionary misconceptions. while further research needs to be done on how parent-child conversations foster accurate evolutionary reasoning and whether such conversations lead to long-term learning, the current findings point to a promising new method for increasing evolution understanding among the general public. • • • acknowledgements this research was supported by national science foundation grant drl-0953384 awarded to andrew shtulman. we would like to thank emi yoshimura and the staff of the los angeles natural history museum for collaborating with us on the design and implementation of the testing procedure, as well as katherine abelson, elahhe afkhamnejad, catherine kiang, and madalyn long for their assistance with data collection and data analysis. andrew shtulman. received a ph.d. in psychology from harvard university in 2006. he has been a member of the department of psychology at occidental college since 2007. his research focuses on conceptual development and conceptual change, particularly as they relate to science education. isabel checa. is an undergraduate at occidental college, anticipated to graduate in 2013. she has been a member of dr. shtulman’s lab since 2010 and is interested in early childhood education. references allen, s., & gutwill, j. p. (2004). designing with multiple interactives: five common pitfalls. curator, 47, 199-212. anggoro, f. k., waxman, s. r., & medin, d. l. (2008). naming practices and the acquisition of key biological concepts: evidence from english and indonesian. psychological science 19, 314-319. asterhan, c. s. c., & schwartz, b. b. (2007). the effects of monological and dialogical argumentation on concept learning in evolutionary theory. journal of educational psychology, 99, 626-639. asterhan, c. s. c., & schwartz, b. b. (2009). argumentation and explanation in conceptual change: indications from protocol analyses of peer-to-peer dialog. cognitive science, 33, 374-400. baum, d. a., smith, d. s., & donovan, s. s. s. (2005). the tree-thinking challenge. science, 310, 979-980. berti, a. e., toneatti, l., & rosati, v. (2010). children’s conceptions about the origin of species: a study of italian children’s conceptions with and without instruction. journal of the learning sciences, 19, 506-538. bishop, b. & anderson, c.a. (1990). student conceptions of natural selection and its role in evolution. journal of research in science teaching, 27, 415-427. brumby, m. n. (1984). misconceptions about the concept of natural selection by medical biology students. science education, 68, 493-503. catley, k. m., & novick, l. r. (2008). seeing the wood for the trees: an analysis of evolutionary diagrams in biology textbooks. bioscience, 58, 976-987. catley, k. m., novick, l. r., & shade, c. k. (2010). interpreting evolutionary diagrams: when topology and process conflict. journal of research in science teaching, 47, 861–882. international electronic journal of elementary education vol.5, issue 1, 27-46, 2012 44 craig, s. d., chi, m. t. h., & vanlehn, k. (2009). improving classroom learning by collaboratively observing human tutoring videos while problem solving. journal of educational psychology, 101, 779-789. crowley, k., callanan, m., jipson, j., galco, j., topping, k., & shrager, j. (2001). shared scientific thinking in everyday parent-child activity. science education, 85, 712-732. crowley, k., callanan, m.a., tenenbaum, h. r., & allen, e. (2001). parents explain more often to boys than to girls during shared scientific thinking. psychological science, 12, 258-261. diamond, j. (1994). sex differences in science museums: a review. curator, 37, 17-24. jipson, j. l., & callanan, m. (2003). mother-child conversation and children's understanding of biological and non-biological changes in size. child development, 74, 629-644. evans, e. m., spiegel, a., gram, w., frazier, b. f., tare, m., thompson, s., & diamond, j. (2010). a conceptual guide to natural history museum visitors’ understanding of evolution. journal of research in science teaching, 47, 326-353. gelman, s. a. (2003). the essential child. oxford, uk: oxford university press. gelman, s. a., & coley, j. d. (1990). the importance of knowing a dodo is a bird: categories and inferences in 2-year-old children. developmental psychology, 26, 796-804. gleason, m. e., & schauble, l. (2000). parents’ assistance of their children’s scientific reasoning. cognition and instruction, 17, 343-378. gunderson, e. a., & levine, s. c. (2011). some types of parent number talk count more than others: relations between parents’ input and children’s cardinal-number knowledge. developmental science, 14, 1021-1032. gregory, t. r. (2008). understanding evolutionary trees. evolution: education and outreach, 1, 121137. gregory, t. r. (2009). understanding natural selection: essential concepts and common misconceptions. evolution: education and outreach, 2, 156-175. griffith, j. a., & brem, s. k. (2004). teaching evolutionary biology: pressures, stress, and coping. journal of research in science teaching, 41, 791-809. hatano, g., & inagaki, k. (1994). young children’s naïve theory of biology. cognition, 50, 171-188. legare, c. h., lane, j., & evans, e. m. (in press). anthropomorphizing science: how does it affect the development of evolutionary concepts? merrill-palmer quarterly. mayr, e. (1982). the growth of biological thought: diversity, evolution, and inheritance. cambridge: harvard university press. meir, e., perry, j, herron, j. c., & kingsolver, j. (2007). college students’ misconceptions about evolutionary trees. american biology teacher, 69, 71-76. miller, j. d., scott, e. c., & okamoto, s. (2006). public acceptance of evolution. science, 313, 765-766. national research council (1996). national science education standards. washington, dc: national academy press. national research council (1998). teaching about evolution and the nature of science. washington, dc: national academy press. nettle, d. (2010). understanding of evolution may be improved by thinking about people. evolutionary psychology, 8, 205-228. newport, f. (2010). four in 10 americans believe in strict creationism. gallup organization. novick, l. r., & catley, k. m. (2007). understanding phylogenies in biology: the influence of a gestalt perceptual principle. journal of experimental psychology: applied, 13, 197-223. parent-child conversations about evolution in the context of an interactive museum display / shtulman & checa 45 novick, l. r., shade, c. k., & catley, k. m. (2010). linear versus branching depictions of evolutionary history: implications for design. topics in cognitive science, 3, 536-559. okada, t., & simon, h. (1997). collaborative discovery in a scientific domain. cognitive science, 21, 109-146. rigney, j. c., & callanan, m. a. (2011). patterns in parent-child conversations about animals at a marine science center. cognitive development, 26, 155-171. samarapungavan, a. & wiers, r. w. (1997). children’s thoughts on the origin of species: a study of explanatory coherence. cognitive science, 21, 147-177. schwarz, b. b., neuman, y., & biezuner, s. (2000). two wrongs may make a right...if they argue together! cognition & instruction, 18, 461-494. shtulman, a. (2006). qualitative differences between naive and scientific theories of evolution. cognitive psychology, 52, 170-194. shtulman, a., & calabi, p. (2012). cognitive constraints on the understanding and acceptance of evolution. in k. s. rosengren, s. brem, e. m. evans, & g. sinatra (eds.), evolution challenges: integrating research and practice in teaching and learning about evolution (pp. 47-65). cambridge, uk: oxford university press. shtulman, a., & calabi, p. (in press). tuition vs. intuition: effects of instruction on naive theories of evolution. merrill-palmer quarterly. shtulman, a., & schulz, l. (2008). the relation between essentialist beliefs and evolutionary reasoning. cognitive science, 32, 1049-1062. sinatra, g. m., brem, s. k., & evans, e. m. (2008). changing minds? implications of conceptual change for teaching and learning about biological evolution. evolution: education and outreach, 1, 189195. solomon, g. e. a., & zaitchik, d. (2012). folkbiology. wires cognitive science, 3, 105-115. southerland, s. a., abrams, e., cummins, c. l., & anzelmo, j. (2001). understanding students’ explanations of biological phenomena: conceptual frameworks or p-prims? science education, 85, 328-348. spiegel, a., evans, e. m., frazier, b. f., hazel, a., tare, m., gram, w., & diamond, j. (2012). changing museum visitors’ concepts of evolution. evolution: education and outreach, 5, 43-61. tare, m., french, j., frazier, b., diamond, j., & evans, e. m. (2011). explanatory parent-child conversation predominates at an evolution exhibit. science education, 95, 720-744. tenenbaum, h. r., prior, j., dowling, c. l., & frost, r. e. (2010) supporting parent-child conversations in a history museum. british journal of educational psychology, 80, 241-254. torrens, e., & barahona, a. (2012). why are some evolutionary trees in natural history museums prone to being misinterpreted? evolution: education and outreach, 5, 76-100. van schijndel, t. j. p., franse, r. k., & raijmakers, m. e. j. (2010). the exploratory behavior scale: assessing young visitors hands-on behavior in science museums. science education, 94, 794-809. waxman, s., medin, d., & ross, n. (2007). folkbiological reasoning from a cross-cultural developmental perspective: early essentialist notions are shaped by cultural beliefs. developmental psychology, 43, 294-308. international electronic journal of elementary education vol.5, issue 1, 27-46, 2012 46 www.iejee.com this page is intentionally left blank 257 copyright © www.iejee.com issn: 1307-9298 © 2022 published by kura education & publishing. this is an open access article under the cc by ncnd license. (https://creativecommons.org/ licenses/by/4.0/) international electronic journal of elementary education january 2023, volume 15, issue 3, 257-265 is accreditation, like a colonoscopy, good for you? ellie kazemi* abstract introduction accreditation is typically a voluntary process that involves a thorough evaluation of an organization’s policies, procedures, and practices. much like a colonoscopy, the evaluation process probes deep and can be uncomfortable. with the discomfort, time, cost, and effort it takes to undergo evaluation for accreditation, the natural question is whether it is worth doing. in this paper, i will review the history of accreditation and the results of systematic literature reviews focused on the impact of accreditation. i will also discuss how accreditation may help provide quality control in behavior analysis and safeguard against service providers’ behaviors being solely shaped by funding sources, such as insurance providers. lastly, i will provide critical questions consumers can ask to assess accrediting bodies’ transparency, objectivity, and fairness when they are seeking accreditation. accreditation is usually a voluntary process that involves a thorough evaluation of an organization’s policies, procedures, and practices against a set of pre-established standards. once standards are established, typically, trained, and objective external peer reviewers evaluate an organization’s compliance by comparing what they review to the pre-established standards. the process is methodical and reiterative such that as a profession matures, the standards and evaluation process are revised to keep up with the changes in the profession. accreditation programs can be developed nationally, by the government, by independent agencies authorized to do so by governments, or by independent (for profit, nonprofit, or not-for-profit) national or international agencies contracted by health care organizations (world health organization [who], 2022). accreditation differs from certification and/or licensure even though the aim of all these professional and regulatory organizations is consumer protection (litvak & sush, 2023). certification and licensure hold individual practitioners accountable to a code of ethical conduct. they also set minimum criteria for competency to practice, usually through an examination for entry into the profession and keywords: accreditation, healthcare quality, behavior analysis accreditation, applied behavior analysis quality, aba quality, behavior analysis standards, aba standards received : 22 february 2023 revised : 10 march 2023 accepted : 23 march 2023 doi : 10.26822/iejee.2023.298 * correspondance details: ellie kazemi, behavioral health center of excellence, california state university, northridge, usa. e-mail: ellie@bhcoe.org orcid: https://orcid.org/0000-0001-8316-4112 258 january 2023, volume 15, issue 3, 257-265 supervised experience hours. once certified and/or licensed, the individual practitioner must update their status by providing evidence of continued education in the required subject areas and continued adherence to legal and ethical guidelines. however, licensure and certification do not manage the behaviors of organizations. from a behavioral system perspective, the direct environmental contingencies surrounding the behavior of organizations have the most influence on organizational behaviors. with behavior analysis services, the most immediate contingencies contacted by service providers are what the insurance companies approve or deny and the corresponding insurance requirements (e.g., assessments, reporting). additionally, many funders have compliance requirements specific to aspects of clinical practice. it then makes sense that clinical practice may be shaped by funder contingencies, which may or may not be aligned with best practice. service providers may focus on aspects of their service delivery that influence the livelihood of their organization, such as the likelihood of future patient referrals, compliance with insurance requirements to avoid audits, and compliance with authorizations to ensure payment for services. unfortunately, the requirements set forth by insurance providers are not necessarily in line with best practice recommendations and quality behavior analytic services. therefore, it makes sense to have a specific set of standards and a process in place for shaping organizational behavior to adhere to best practice recommendations. accreditation arranges contingencies for service providers to demonstrate that they adhere to standards of excellence and best practices in behavior analysis. for example, service providers are not specifically paid by insurance and funding entities to provide ongoing training to their clinical staff. what is worse is that investment in training staff may inadvertently be punished by the high staff turnover. however, the efforts and investment of organizations that continue to provide sufficient training to their staff are acknowledged through the accreditation process. for another example, see the section on ethics, integrity, and professionalism in the standards of excellence (behavioral health center of excellence [bhcoe], 2022). the concept of accreditation in the united states is more than a hundred years old and emerged from concerns to protect public health and safety. accreditation is carried out by private, not-for-profit, or nonprofit organizations designed for this specific purpose. in 1917, the "minimum standard for hospitals" was developed by the american college of surgeons (acs). it was a set of guidelines establishing minimum standards for hospitals in the united states. to develop the guidelines, a committee of 21 surgeons, hospital administrators, laboratory workers, statisticians, and leading hospital superintendents met for two days in chicago to formulate a set of questions that would enable them to obtain hospital data to consider a “minimum standard.” these standards were designed to ensure that hospitals were equipped and staffed to provide safe and effective care to patients. the "minimum standard for hospitals" included guidelines for hospital size, equipment, and staffing levels, as well as recommendations for the types of services that hospitals should provide. the standards also established requirements for the training and education of hospital staff, including doctors, nurses, and other healthcare professionals (for more information regarding acs’s efforts, the minimum standards, and historical images of the notes, see wright, 2017). the "minimum standard for hospitals" was a groundbreaking effort to improve healthcare quality in the united states, laying the foundation for developing more comprehensive accreditation standards. the acs was pleased to find an immediate interest in compliance with the standards, even though compliance was entirely voluntary and sending out college staff to conduct evaluations and provide consultations was labor-intensive and costly. the immediate adoption of accreditation was likely because hospitals had an opportunity to help formulate the standards, and their competitors were doing it (wright, 2017). today, hospitals in the united states must meet a wide range of standards to be accredited, including standards related to patient safety, quality of care, and healthcare outcomes. the acs hospital standardization project was an essential framework for hospitals for three decades before evolving into the joint commission on accreditation of hospitals in 1951, which was renamed the joint commission on accreditation of healthcare organizations in 1987, and the joint commission in 2007. in the decades since its establishment, the joint commission has become one of the country's most widely recognized accrediting bodies for healthcare organizations. other accrediting organizations have also been established over the years, including the commission on accreditation of rehabilitation facilities (carf), utilization review accreditation commission (urac), the national committee for quality assurance (ncqa), and council on accreditation (coa). in healthcare, the movement toward standardization and accreditation was to question the status quo and to improve patient care (lenaway et al., 2007). today, accreditation for healthcare services is most often required by payers, such as insurance providers, as a condition for reimbursement. in cases when it is not required, some healthcare organizations seek accreditation to renegotiate or obtain higher reimbursement rates from funding entities. also, some organizations choose to seek accreditation voluntarily to demonstrate their commitment to quality and to improve their own performance. in 259 is accreditation, like a colonoscopy, good for you? / kazemi line with this movement in healthcare, educational institutions, licensure boards, and certification bodies have all moved toward requiring accreditation as a means of quality control (eaton, 2015; ibrahim, 2014). the accreditation process, however, is costly, and it involves much time and effort from the accrediting body, the evaluators, and the organizations seeking accreditation. it is no wonder that accreditation evaluation can be compared to a colonoscopy because an organization must allow an objective third party to look at all its intimate parts, including its procedures, policies, and practices. the question is, does this cumbersome and costly process increase the quality of care? first, what is healthcare quality? healthcare quality is a broad concept that has been defined by the national academy of medicine as “the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge” (centers for medicare & medicaid services [cms], 2021, what is quality improvement? section). like the seven dimensions that define applied behavior analysis proposed by baer et al. (1968, 1987), a physician by the name of donabedian (1990) proposed the seven attributes of health care that define healthcare quality. these dimensions were later adapted by the institute of medicine (2001) and included safety, patient-centeredness, timeliness, equity, access, efficiency, and effectiveness. accurate assessment of healthcare quality is challenging, but the cms has proposed various frameworks for improving outcomes and conducting quality measurements (see cms, 2022;). in the meantime, as efforts to improve measurement continues, hospital accreditation which incorporates adherence to the seven dimensions has become adopted worldwide to assess and improve healthcare service quality (lam et al., 2018). what has been the impact of accreditation on healthcare quality? accreditation methodologies vary across accrediting bodies and typically rely on the organization seeking accreditation to provide documentation of procedures and policies. therefore, the research evidence in the published literature evaluating the effectiveness of accreditation for patient outcomes is mixed, and the results should be interpreted with some caution (araujo et al., 2020). additionally, accreditation takes a bird’s view of an organization; thus, it does not guarantee that best practices will be followed daily (hinchcliff et al., 2012). one long-standing challenge to producing robust research evidence on the impact of accreditation has been the absence of patient-level data on both accreditation status and patient outcomes (bracewell & winchester, 2021). a more appropriate mechanism for evaluating patient outcomes seems to be value-based care (vbc) and payment models whereby service providers are incentivized to submit patient-level data for quality measurement and patient-level analyses (see litvak, 2023). despite the challenges in evaluating the impact of accreditation, the results of systematic literature reviews suggest that healthcare accreditation may have a positive impact on several important aspects of healthcare (see araujo et al., 2020). in their literature review, araujo et al. (2020) initially reviewed 943 citations from eight different databases. araujo et al. only included 36 studies in their final review that used quantitative methods to compare accredited vs. nonaccredited hospitals on the seven healthcare quality dimensions. they found that accreditation had a positive impact on five of the seven dimensions, including efficiency, safety, effectiveness, timeliness, and patient-centeredness. some earlier systematic literature reviews have also found that accreditation programs improve the process of care and clinical outcomes of a wide spectrum of clinical conditions (alkhenizan & shaw, 2011). in the field of education, the literature also suggests that accreditation of educational institutions may have a positive impact on educational services by cultivating accountability, encouraging continuous self-reflection and improvement, and increasing access to educational opportunities (e.g., ülker, 2020). in a recent survey conducted by norc at the university of chicago for the public health accreditation board (phab), 98% of applicant health departments expected the accreditation process to increase quality improvement processes and 76% reported continued engagement in quality improvement activities four years after accreditation (gonick et al., 2020). the world health organization (who) suggests that healthcare accreditation should be recognized as an ongoing process of quality improvement vs. simply a status (e.g., whether an organization is accredited) and thus supportive of the perpetual process to improve the quality of care provided. furthermore, it is recommended that accreditation standards and subsequent reaccreditation processes systematically measure improvement over time (who, 2022). in general, the results of literature reviews and recommendations of organizations such as the who suggest that accreditation impacts health quality in a positive way. however, it is unclear how accreditation directly impacts the quality of care and achieves improved outcomes for patients and organizations. some of this may be easy to infer. for example, in their review, cabana et al. (1999) found that there were about 300 potential barriers physicians reported when asked about adherence to clinical practice guidelines. 260 january 2023, volume 15, issue 3, 257-265 bracewell and winchester (2021) suggested that one of the ways accreditation may improve outcomes is by reducing some of these types of barriers. that is, physicians’ lack of awareness of newer practice guidelines because of a fast pace evolving literature may be overcome through discussions about accreditation standards and requirements to pass. additionally, the accreditation evaluation process itself may set the occasion for open discussions and change the focus to improvement (hovlid et al., 2020). outside of patient data, however, there is evidence that accreditation is valuable and pays off both financially and non-financially in terms of higher quality care, improved client outcomes, and increased compliance. in 2021, the joint commission contracted with the roi institute (see https:// roiinstitute.net/) to conduct an evaluation of a sample of behavioral health organizations to report on the roi of accreditation. the key findings, based on 180 behavioral health organizations, were that the roi was 623%, which is a benefit-cost ratio of 7.23. in other words, for every dollar spent, the dollar is returned plus an additional $6.23. the main areas of impact included improved competencies of staff and supervisors, reduced staff turnover, improved reimbursements, increased revenue, risk reduction, and improvements in operational efficiencies (roi institute, 2022). what are some pros and cons of accreditation? accreditation can be valuable in several ways. it is a process through which an organization is evaluated against a set of standards to ensure that it meets certain criteria and is adhering to best practice standards. this process is designed to help ensure that the organization is operating at a certain level of quality and demonstrates that the organization is open to being reviewed and receiving and acting upon external feedback from a third party. one of the possible benefits of accreditation is that it provides assurance to patients, parents, and other stakeholders that the accredited organization meets certain standards of quality. second, accreditation can help an organization to improve continuously by providing a framework for self-assessment and the identification of areas for improvement. third, accreditation can help reduce the risk of errors and improve patient safety by ensuring that organizations have processes in place to identify and mitigate potential risks. fourth, accreditation often sets the occasion for conversations that lead to the standardization of processes, procedures, terms, and titles, which reduces redundancies and ambiguity and increases efficiency and transparency in a profession. fifth, accreditation can promote professionalism within an organization by encouraging staff to adhere to best practices and guidelines, just as the organization sets an example by adhering to them for accreditation. lastly, it can increase the credibility and reputation of an organization by demonstrating that it has met certain standards of quality. if the organization is an internship site or offers supervised experiences for students, accreditation can provide greater recognition that the organization meets certain standards for training and supervision, which can improve the quality of the fieldwork supervised experience for individuals accruing hours to sit for certification and/or licensure. this can be especially important for students who are seeking to gain valuable clinical experience and build their professional careers. accreditation can also have some possible downsides. first, one of the disadvantages of accreditation is that it can be a costly process. a thorough evaluation of an organization’s policies, procedures, and practices takes time and resources from the accrediting body. therefore, accreditation involves application fees for the organization seeking an accreditation evaluation. additionally, the organization must invest time and resources to prepare for the accreditation process and respond to the evaluation feedback. second, the accreditation process, like a colonoscopy, is thorough and may be uncomfortable as a third-party observes some of the most intimate parts of an organization, including internal documents, correspondences, grievance procedures, training logs, supervision materials, leadership guidelines, and more. third, accreditation does not guarantee quality because it is a snapshot in a moment in time and may miss day-in and day-out activities for specific clients or staff. fourth, accreditation is not always recognized by all organizations or may not be required yet, which minimizes the potential of accreditation to help establish credibility. lastly, there is a price to standardization, especially if the standards and the accreditation evaluation process do not leave room for flexibility and clinical judgment. there is a fine balance to strike with standardization to ensure appropriate customization and person-centered, individualized care. if that balance is off, accreditation may place barriers for small and innovative organizations and tip the balance toward a one-size-fits-all approach in patient care. just as with anything else in life, including consenting to a colonoscopy, there are always pros and cons. it is crucial that service providers and stakeholders interested in accreditation are aware of the pros and cons to help shape the accrediting bodies’ standards and evaluation process to minimize the cons and strengthen the pros. is accreditation right for applied behavior analysis? as previously discussed, accreditation is a process by which organizations demonstrate that they meet certain standards of quality and safety. therefore, 261 is accreditation, like a colonoscopy, good for you? / kazemi it makes sense for each healthcare profession to have specialized accreditation to ensure that the organizations seeking accreditation adhere to quality care as per the specific industry regulations and the profession’s best practices. for example, for colonoscopy, the primary accreditation bodies are the america college of gastroenterology (acg) and the american society for gastrointestinal endoscopy (asge). both accrediting bodies have the same goal of ensuring that providers meet specific standards for patient care and safety, as well as for the technical quality of the procedures. although a general healthcare accreditation would ensure patient safety, it would not include specific best practice guidelines and the technical aspects of the field of gastroenterology. similarly, organizations providing behavioral healthcare can pursue general healthcare accreditation, which would help assess and improve the organization’s general patient safety and organizational policies. however, an accreditation specific to behavior analysis would also include profession-specific evaluations of best practices (e.g., use of non-harmful reinforcers, supervision caseload, care coordination, collaboration). one of the advantages of a profession-specific accreditation for behavior analysis is that it would set the occasion for discussions and agreements about profession-specific titles and terminology. another benefit of professionspecific accreditation for behavior analysis is that stakeholders such as patients, parents/caregivers of patients, and patient advocates would be able to have a source to refer to when they are trying to identify quality service providers. usually, accrediting bodies provide information to the public that also promotes awareness of quality service provision (e.g., https://www.bhcoe.org/parent-autism-qualityaba-providers/; bhcoe, n.d.). lastly, stakeholders interested in evaluating best practices in behavior analytic care can review published standards, which have been developed specifically for behavior analytic services; for example, see bhcoe/ansi 201: standards of excellence for applied behavior analysis services, which have been adopted by ansi as an american national standard (bhcoe, 2022). also, see the autism commission on quality accreditation program standards and guide (version 1; https:// autismcommission.org/standards/) and standards for interprofessional collaboration in treatment of individuals with autism (bowman, suarez, & weiss, 2021). what are some things to look for in an accreditor? overall, it seems clear that accreditation in behavior analysis would provide oversight of behavior analytic services at the organizational level and improve patient care. however, it is important to note that not all accrediting bodies are equal. below, i have outlined some important questions consumers should ask as they consider accreditation (for a brief summary of these questions, see https://www.bhcoe. org/2022/08/top-questions-for-aba-providers-toask-when-choosing-an-accreditation-program/, or https://accreditationguru.com/10-steps-to-selectingan-accrediting-body-2/) (ansi, 2022; bhcoe, 2022). does the accrediting body hold accreditation or receive feedback from an independent body that reviews its performance? from a behavioral perspective, every organization that potentially has control over contingencies that shape service providers’ behaviors is at risk for abuse of power. it is important to have a checks and balances system in place to ensure that power is shared and there is oversight. for a balanced approach, it is important that the accrediting body is overseen by independent parties informed of the accreditation process and best practices in standard development and evaluations. for example, a hallmark of a credible accreditation program is that they hold accreditation by the american national standards institute (ansi), which promotes transparency in how standards are developed and how public comments and feedback are incorporated (litvak & sush, 2023). ansi is a private organization that administers and coordinates the u.s. system of voluntary professional standards and evaluation. although ansi itself is not a standard developing organization, it provides a framework for fair standard development and quality evaluation systems. ansi safeguards the integrity of organizations that develop standards and is a neutral venue for coordinating standards and promoting collaborative efforts in standard development. in addition to including stakeholders whose lived experiences are invaluable insight for developing standards, the accrediting body must guide standard development by providing the commission with results of thorough literature reviews and best practice recommendations. is the evaluation methodology valid? large-scale evaluation of an organization’s procedures, processes, and practices is not an easy feat. thankfully, there is a literature base to guide large-scale program evaluation to ensure reliability and validity regarding measurement and accreditation decisions. in fact, there are empirical journals solely dedicated to quality, comparability, and evaluation for accreditation, but many of these journals are profession specific. for example, the journal of accreditation and quality assurance provides information on all aspects of quality, transparency, and reliability of measurement results in chemical and biological sciences. the journal also includes fields such as nutrition, consumer protection, pharmacy, forensics, and laboratory medicine. science and engineering ethics focuses on education, research, and practice in engineering. quality assurance in education focuses on education 262 january 2023, volume 15, issue 3, 257-265 at all levels (e.g., primary, higher, professional). joint commission journal on quality and patient safety focuses on the quality and safety of healthcare. taken together across the literature, commonalities exist in what is considered good measurement science and program evaluation. some best practice recommendations include employing leadership services with experience in quality assurance and evaluation methodology. with the evaluation process itself, it is important to involve all stakeholders (e.g., patients, technicians, supervisors, and leadership) and to incorporate all voices using a multi-informant approach. furthermore, a multi-dimensional approach to evaluation helps increase assessment validity (cumming & miller, 2019; shryock & reed, 2009). for example, an accreditation evaluation based on self-assessment and self-report is much more limited than an evaluation based on self-assessment, direct observations of therapy/treatment, open-ended interviews, and surveys. the multi-dimensional approach to assessment increases the acceptability of the accreditation decision, thereby increasing the validity of the results when there is reliability across dimensions and informants. at a minimum, it is best to include both direct and indirect assessment methods for evaluating if an organization’s practices, policies, and procedures adhere to the profession’s standards. lastly, there are many key players within accrediting bodies, it is essential that the individual(s) responsible for overseeing the accreditation program, including but not limited to standard development and maintenance, evaluation methodology, compliance, procedures, and processes, have educational and professional experience in quality assurance, quality measurement, and evaluation methodology. is the accreditor an independent neutral entity, which helps make the accreditation decision fair? in any profession, there are a limited number of professionals, and it is difficult to develop neutrality and independence when the limited number of professionals shift and change positions in their careers. however, it is imperative for accrediting bodies to be neutral, objective third parties that do not personally benefit from setting the professions’ standards and accreditation evaluation methodology. to have credibility, the accrediting body should not be owned or operated by an organization representing a specific group of stakeholders such as payors, service providers, or patients. furthermore, the evaluators cannot be employees from other organizations who provide services to the same population or work for a competitor. such practices protect the evaluation process and limit any influence of potential biases and conflicts of interest. lastly, it is crucial for the accrediting body not to have dual relationships or organizational biases toward certain trade organizations, certification or licensing bodies, or professional associations. how long has the accreditor been evaluating organizations? it is important for the accrediting body to have demonstrated a history of viability for continued successful operation and stability. any reader who has taken some time to develop tools for measuring behavior can relate to the time and experience it takes to develop a behavior measurement system that is reliable and valid. developing evaluation tools for accreditation, similarly, requires time and experience in large-scale evaluations. additionally, developing tools that enable trained evaluators to reliably make high stake decisions of pass/not pass requires piloting and obtaining a representative sample, several revisions to tools based on the results, and revisions to the tools and/or methodology based on stakeholder feedback. it can take years to develop reliable evaluation tools that are informative, fair, and socially acceptable. organizations should ensure that accrediting bodies have spent years developing and self-assessing and revising their tools. does the accreditor have an independent disciplinary review or compliance committee? when a credentialed clinician does not adhere to the credentialing board’s code of ethics, the witness of the violation can submit a formal complaint outlining the clinician’s behaviors alleged to violate the code of ethics, documentation for the alleged violation, and documentation of attempts to bring the issue to the clinician (for example see https://www.bacb.com/ ethics-information/reporting-to-ethics-department/; reporting to the ethics department, 2023). what about when the violation is conducted by individuals who are not clinicians or credentialed? who oversees grievances or complaints against service organizations, sometimes owned or operated by leadership who are not certified or licensed behavior analysts? the role of the disciplinary review or compliance committee of an accrediting body is to process compliance concerns received from the general public, patients, parents/caregivers, and staff to assist accredited organizations to remedy any problem areas as well as provide continued support to allow for growth and quality improvement. although it may not be at the forefront when choosing an accreditation program, it is essential that the accrediting body has a committee to oversee organizations’ adherence to the standards under which they have been evaluated. as noted before, accreditation evaluation only provides a birdseye view from a mere snapshot in time; therefore, a compliance review department enables the accrediting body to promote accountability and adherence to the standards across time (for example, see https://www.bhcoe.org/become-a-bhcoe/ report-a-compliance-concern/; bhcoe, 2022). 263 is accreditation, like a colonoscopy, good for you? / kazemi does the accreditor demonstrate good customer service? it may not seem important initially, but good customer service is crucial. the organization undergoing the accreditation process may have questions or concerns throughout the evaluation. it is important that they can receive answers promptly. it is also important for customer service personnel to consist of standards experts, a support team, resources, specific examples, and someone who oversees their account. another key component to an accrediting body demonstrating good customer service is a modality for organizations to be able to provide ongoing feedback throughout and following the evaluation process. furthermore, it is crucial that the accreditor not only solicits and encourages feedback but also can demonstrate how the feedback has been utilized to promote improvements. has the accreditor been considered or approved by federal and/or local/state authorities? lastly, organizations should seek out an accreditation body whose standards align with their applicable federal, state, and local mandates in the regions that they operate to ensure compliance and consistency with both mandates and best practices. summary and conclusions the answer to the question of, “is accreditation, like a colonoscopy, good for you?”, is yes, absolutely! accreditation will provide a contingency for organizational behavior and allow the profession of behavior analysis to shape the quality-ofservice delivery instead of funding organizations. accreditation will also catapult discussions between behavior analysts to standardize terms and come to some consensus regarding how they reference common procedures, much like the bcba task list from behavior analyst certification board (behavior analyst certification board [bacb], 2017). accreditation will also provide transparency into the profession and what the profession itself considers best practice. ultimately, transparency regarding best practices in service delivery will influence patient outcomes and quality of care. currently, there is no way for a consumer, or a funding agency, to differentiate between service providers outside of personal relationships and anecdotal data (e.g., testimonials, word of mouth). however, just as an accrediting body can hold service organizations accountable, it is equally important that service providers hold the accrediting bodies to standards of excellence to mitigate potential negligence and abuse of power. in conclusion, like a colonoscopy, accreditation allows for the assessment and identification of problems, which can prevent the widespread growth of problems through intervention. most organizations providing behavior-analytic care are highly patientcentered and focused on delivering excellent treatment. however, the rare bad seeds influence the public’s perception of behavior analysis and cultivate widespread criticism. unfortunately, bad news and reports of poor behavior spread quickly, like cancer, undermining the efforts of organizations that provide excellent services. although undergoing accreditation may not feel pleasant, it can prevent the spread of cancer, and i would argue that it is good for the profession of behavior analysis and its future vitality. author note statements and declarations: ellie kazemi serves as the chief science officer at the behavioral health center of excellence (bhcoe), which has provided financial support for writing this manuscript. references alkhenizan, a., & shaw, c. (2011). impact of accreditation on the quality of healthcare services: a systematic review of the literature. annals of saudi medicine, 31(4), 407-416. https:// doi.org/10.4103/0256-4947.83204 ansi national accreditation board. (2022). tips for selecting a certification body. https://anab. ansi.org/management-systems-accreditation/ certification-bodies/how-to-select araujo, c. a. s., siqueira, m. m., & malik, a. m. (2020). hospital accreditation impact on healthcare quality dimensions: a systematic review. international journal for quality in health care, 32(8), 531-544. https://doi.org/10.1093/intqhc/ mzaa090 baer, d. m., wolf, m. m., & risley, t. r. (1968). some current dimensions of applied behavior analysis. journal of applied behavior analysis, 1(1), 91-97. https://doi.org/10.1901/jaba.1968.1-91 baer, d. m., wolf, m. m., & risley, t. (1987). some stillcurrent dimensions of applied behavior analysis. journal of applied behavior analysis, 20(4), 313327. https://doi.org/10.1901/jaba.1987.20-313 behavior analyst certification board. (2017). bcba task list (5th ed.). littleton, co: author. behavioral health center of excellence. (2022, august 2). re: top questions for aba providers to ask when choosing an accreditation program. bhcoe accreditation. https://www.bhcoe. org/2022/08/top-questions-for-aba-providersto-ask-when-choosing-an-accreditationprogram/ 264 january 2023, volume 15, issue 3, 257-265 behavioral health center of excellence (2022). bhcoe/ ansi 201: standards of excellence for applied behavior analysis services. los angeles, ca: publisher. behavioral health center of excellence. (n.d.). how bhcoe helps ensure quality aba care and parental peace of mind. bhcoe accreditation. h t t p s : // w w w. b h c o e . o rg / p a re n ta u t i s m quality-aba-providers/ bracewell, n., & winchester, d. e. (2021). accreditation in health care: does it make any difference to patient outcomes?. bmj quality & safety, 30(11), 845-847. http://dx.doi.org/10.1136/ bmjqs-2020-012533 bowman, k. s., suarez, v. d., & weiss, m. j. (2021). standards for interprofessional collaboration in the treatment of individuals with autism. behavior analysis in practice, 14(4), 1191-1208. https://doi.org/10.1007/s40617-021-00560-0 cabana, m. d., rand, c. s., powe, n. r., wu, a. w., wilson, m. h., abboud, p. a., & rubin, h. r. (1999). why don't physicians follow clinical practice guidelines? a framework for improvement. jama, 282(15), 1458-1465. https://doi.org/10.1001/ jama.282.15.1458 centers for medicare & medicaid services. (2021). quality measurement and quality improvement. htt p s : // w w w.c m s .g ov/ m e d i c a re/q u a l i t yinitiatives-patient-assessment-instruments/ m m s / q u a l i t y m e a s u r e a n d q u a l i t y improvement centers for medicare & medicaid services. (2022). quality measures. https://www.cms.gov/ m e d i c a r e / q u a l i t y i n i t i a t i v e s p a t i e n t assessment-instruments/qualitymeasures centers for medicare & medicaid services. (2022). quality measures: how they are developed, used, & maintained. https://mmshub.cms.gov/ sites/default/files/guide-quality-measureshow-they-are-developed-used-maintained. pdf cumming, t., & miller, m. d. (2019). academic assessment: best practices for successful outcomes with accreditation evaluation teams. new directions for community colleges, 2019(186), 81–93. https://doi.org//10.1002/cc.20359 donabedian a. (1990). the seven pillars of quality. archives of pathology & laboratory medicine, 114(11), 1115–1118. eaton, j. s. (2015). an overview of us accreditation. revised november 2015. council for higher education accreditation. gershfeld litvak, s., & sush, d. j. (2023). ethics and legal issues. in j. matson (ed.), handbook of applied behavior analysis: integrating research into practice. springer. gonick, s., heffernan, m., siegfried, a.l., kennedy, m., & meit, m.b. (2020). assessing accreditation outcomes: quality improvement and performance management findings. bethesda, md: norc. hinchcliff, r., greenfield, d., moldovan, m., westbrook, j. i., pawsey, m., mumford, v., & braithwaite, j. (2012). narrative synthesis of health service accreditation literature. bmj quality & safety, 21(12), 979-991. https://doi.org/10.1136/ bmjqs-2012-000852 hovlid, e., braut, g. s., hannisdal, e., walshe, k., bukve, o., flottorp, s., ... & frich, j. c. (2020). mediators of change in healthcare organisations subject to external assessment: a systematic review with narrative synthesis. bmj open, 10(8), e038850. http://dx.doi.org/10.1136/bmjopen-2020-038850 ibrahim, h. a.-h. (2014). quality assurance and accreditation in education. open journal of education, 2(2), 106-110. https://doi.org/10.12966/ oje.06.06.2014 institute of medicine (us) committee to design a strategy for quality review and assurance in medicare. medicare: a strategy for quality assurance, vol. 1. washington, dc: national academies press, 1990. http://www.ncbi.nlm. nih.gov/books/nbk235462/ institute of medicine (us) committee on quality of health care in america. crossing the quality chasm: a new health system for the twentyfirst century. washington, dc: national academies press, 2001. joint commission. (2022). wikipedia. retrieved february 2, 2023 from https://en.wikipedia. org/wiki/joint_commission kazemi, e. & morford, z. (2022, may 12). the value of bhcoe accreditation: comparing accredited to non-accredited organizations & review of customer surveys [powerpoint slides]. behavioral health center of excellence. 265 is accreditation, like a colonoscopy, good for you? / kazemi lam, m. b., figueroa, j. f., feyman, y., reimold, k. e., orav, e. j., & jha, a. k. (2018). association between patient outcomes and accreditation in us hospitals: observational study. bmj, 363. https://doi.org/10.1136/bmj.k4011 lenaway, d., corso, l., &amp; bailey, s. (2007). accreditation as an opportunity to strengthenpublic health. journal of public health management and practice, 13(4), 332–333. https://doi.org/10.1097/01. phh.0000278023.26326.25 reporting to the ethics department. (2023). behavior analyst certification board. retrieved february 3, 2023 from https://www.bacb.com/ethicsinformation/reporting-to-ethics-department/ roi institute. (2022). the impact and roi of joint commission accreditation for behavioral health. [white paper]. https://www.jointcommission. org /-/media/tjc/documents/accred-andcert/bhc/20230208-tjc-bhc-white-paper.pdf shryock, k., & reed, h. (2009, june). abet accreditation: best practices for assessment. in 2009 annual conference & exposition (pp. 14-148). ülker, n. (2020). an analysis of current trends in higher education: the place and importance of students' learning experiences in quality assurance. journal of higher education & science/yüksekögretim ve bilim dergisi, 10(3). world health organization. (2022). health care accreditation and quality of care: exploring the role of accreditation and external evaluation of health care facilities and organizations. geneva: world health organization. license: cc by-nc-sa 3.0 igo. wright jr, j. r. (2017). the american college of surgeons, minimum standards for hospitals, and the provision of high-quality laboratory services. archives of pathology & laboratory medicine, 141(5), 704-717. https://doi.org/10.5858/ arpa.2016-0348-hp international electronic journal of elementary education, 2015, 8(1), 147-160. issn:1307-9298 copyright © iejee www.iejee.com from research to practice: the effect of multicomponent vocabulary instruction on increasing vocabulary and comprehension performance in social studies lori graham texas a&m university, usa anna graham texas a&m university, usa courtney west  texas a&m university, usa received: september, 2014 / revised: july, 2015 / accepted: august, 2015 abstract this study was designed to demonstrate the effect of implementing multi-component vocabulary strategy instruction in fourth grade social studies. curriculum was designed for a six-week period and was intended to actively engage students and reinforce retention of word meanings in isolation and in context. teachers were randomly chosen for assignment to the intervention and/or to the comparison group. the study included 375 fourth-grade students from 3 different districts and 5 schools. the student population consisted of 29 classes taught by 23 different teachers. two different vocabulary and comprehension measures were administered, and results were analyzed using difference score analyses and repeated measures anovas. outcomes were consistent across both administered measures. although student scores improved in both the group receiving the intervention and the group receiving regular classroom instruction, findings indicated that the group receiving the intervention showed greater gains and persisted longer than in the comparison classrooms. keywords: vocabulary, social studies, comprehension, explicit instruction, semantic feature analysis introduction the 2011 report by the national assessment of educational progress (naep, 2011) highlights the following points: (a) among fourth graders nationally, 33% are reading below basic level, and 24% of eighth graders are reading below basic level, which means that these students cannot perform at minimum academic expectations. (b) equally  courtney west, director, office of medical education college of medicine, texas a&m health science center 8441 highway 47, bryan, texas 77807, phone: 979.436.0230, e-mail: west@medicine.tamhsc.edu http://www.iejee.com/ international electronic journal of elementary education vol.8, issue 1, 147-160, 2015 148 distressing is the observation that the percentage of children showing proficiency in handling demanding material is only 34% at both the fourth grade and eighth grade levels. snow, burns, and griffin (1998) stressed the importance of reading because it is essential to succeed in our society. further, they stated that, “in a technological society, the demands for higher literacy are ever increasing, creating more grievous consequences for those who fall short” (p. 10). this must be addressed beginning at a very early age and continuing through school with high standards for students. perie, grigg, and donahue (2005), report an alarming number of students who do not demonstrate even partial mastery of prerequisite knowledge and skills considered fundamental for proficient work at each grade. lyon (as cited in u.s. government printing office, 2001), in a report to the subcommittee on education reform, reminded us, “thirtyeight percent of fourth graders can’t read well enough to understand a basic paragraph” (p. 13). according to armbruster, lehr, and osborn (2003), comprehension is described as “the reason for reading. if readers can read the words but do not understand what they are reading, they are not really reading” (p. 48). in the executive summary regarding comprehension, the national reading panel (nrp, 2000), described several themes that emerged when examining the research. one of the themes was that “reading comprehension is a cognitive process that integrates complex skills and cannot be understood without examining the critical role of vocabulary learning and instruction and its development” (nrp, 2000, p. 41). increasing the number of words in a person’s vocabulary was determined to be a strong predictor of students’ ability to comprehend text as students must work to construct meaning through a combination of the text and the reader (anderson & freebody, 1981; durkin, 1993; yildirim, yildiz, & ates, 2011). according to phythian-sence and wagner (2007), “acquiring the vocabulary we use for thinking and communicating is a linguistic achievement of nearly incomprehensible importance and complexity” (p.1). multiple studies have demonstrated effective methods for teaching vocabulary words in classroom settings (baker et. al., 1998; blachowicz & fisher, 2000; coyne, mccoach, & kapp, 2007; foil & alber, 2002; nagy & scott, 2000; vitale & romance, 2008). foil and alber (2002) described proficient reading as dependent on the development and synthesis of a complex array of critical sub-skills, in which understanding word meanings and their connection to other concepts is a critical component. to avoid the cycle of poor reading leading to limited vocabulary knowledge, perpetuating further lack of reading and development of vocabulary knowledge, they described strategies for building vocabulary (foil & alber, 2002). cunningham and zibulsky (2009) supported the need for teachers to gain knowledge and utilize effective strategies to further literacy development for all children. knowledge of successful instruction is important for classroom knowledge and application. aaron, joshi, and quatroche (2008) described the reciprocal relationship between vocabulary and comprehension and further noted that repetition and meaning promote the retention of vocabulary. carreker and birsh (2005) used extensive multisensory activities for teaching basic language skills. included in these skills was work on specific approaches to the teaching of vocabulary. strategies are important so that a child can recognize words and/or decode, but the goal of reading is not accomplished if students are unable to connect meaning to the words (beck, mckeown, & kucan, 2008). real comprehension is described by zimmerman and hutchins (2003) as thinking, learning, and expanding a reader’s knowledge and horizons. many students today have a clearer understanding and are able to more accurately conceptualize the meaning of a term after repeated exposure through several means (blachowicz & fisher, 2002). multi-component strategies are essential in providing repeated exposure in different contexts, and successfully keeping students engaged in the learning material (simmons et al., 2005). the the effect of multi-component vocabulary instruction / graham, graham & west 149 complexity of the knowledge of a word is further explained by nagy (2009) as “knowing a word includes knowing how it can function in a sentence, what other words it is commonly used with, how it is related in meaning and form to other words, and what styles of language for which it is appropriate” (p. 48). neuman and dwyer (2009) emphasize this notion that connection between vocabulary and better reading ability is that vocabulary is more than words; it represents knowledge. nation (2008) stated that “the ability to deal with words is at the very heart of reading: if an individual fails to read words, if they are slow to read words, or if they are unable to appreciate the meanings of words, comprehension will be seriously hampered” (p. 1122). the importance of vocabulary as a critical determinant of comprehension success is further explicated by joshi (2005), “a well-developed meaning vocabulary is a prerequisite for fluent reading, a critical link between decoding and comprehension. however, the role of vocabulary in fluent reading has received much less attention in both research and theory than have decoding and comprehension strategies” (p. 209). thus, a number of researchers have indicated a strong need for further study on the role of vocabulary in comprehension, particularly in the content areas (beck & carpenter, 1986; beck, mckeown, & gromoll, 1989; beck, mckeown, sinatra & loxterman, 1991; hall, 2004; harmon & hedrick, 2005; nrp, 2000; williams, 2005). therefore, the researchers sought to address two primary research questions: what is the effect of multi-component social studies vocabulary instruction on comprehension, and is that difference sustained? method to address these two research questions, the authors designed this study to determine the effect of multi-component vocabulary instruction in social studies in grade 4. social studies was selected because content area texts for this subject contain vocabulary that must be learned to comprehend the material. the opportunity to create curriculum that met the needs of the schools, teachers, and students, while addressing a much-needed area of research, opened the door for exploration of explicit instruction and the vocabularycomprehension connection. the findings from the literature review demonstrated the gap in research for much-needed studies to address vocabulary instruction, its effect on comprehension, and active learning within the confines of real schools and authentic settings. the focus was on direct, explicit instruction of vocabulary strategies and how it impacts comprehension. explicit instruction was utilized because it was identified in the nrp (2000) report as one of the most important methods of teaching vocabulary and would enable teachers to focus on key words that were common and recurrent (juel & deffes, 2004) in their social studies materials. this type of direct, explicit instruction paves the way for students to identify words and subsequently retain their meanings through repeated application of word learning strategies. participants for the purpose of this study, the authors focused on three separate districts in the southwestern part of the united states. the research was conducted on five different campuses located in five different cities within approximately 30 miles of each other. prior to the beginning of the school year, researchers met with superintendents and school administrators to engender their support. the administrators provided the names of all fourth-grade social-studies teachers on each campus. the names were then anonymized and randomly assigned to either the treatment group or comparison group. one campus had two teachers who taught all sections of social studies. the first teacher’s name drawn served as the treatment group teacher for her four sections of students and the other teacher as the comparison group for her four sections of students. there were a total of 23 teachers and 29 sections of students who participated. a total of 375 fourth-grade social international electronic journal of elementary education vol.8, issue 1, 147-160, 2015 150 studies students participated in this study, comprising 15 treatment groups and 14 comparison groups. intervention at the time, the district’s curriculum consisted of 300 minutes of language arts instruction per week, with no structured vocabulary component. the authors designed a six-week intervention in which treatment group teachers were asked to spend 90 minutes per week of language arts time focused on social studies vocabulary instruction. the teachers were asked to structure the additional instruction time in thirty-minute segments, three times per week, as classroom time and schedules permitted. the authors surveyed all teachers to obtain basic information regarding teaching experience, degrees, certification, ethnicity, and gender. also included in the survey was information related to the current use of different instructional strategies in the classroom and the teacher’s perception of his or her familiarity with different strategies. the researchers designed curriculum notebooks for the six weeks according to district curriculum and provided them for each of the teachers in the treatment group. a comprehensive set of materials were reviewed together and all of the teachers in the treatment group were trained in the expectations for vocabulary instructional methods prior to implementation. the training included the use of each of the following: explicit instruction; student study teams; active engagement in learning tasks; vocabulary maps; connections webs; semantic feature analysis. students were actively involved in the learning process and the teacher facilitated activities such as games like ready, set, go; vocabulary memory; or jeopardy, in addition to their instruction. while students were expected to actively participate, there were multiple opportunities for different types of participation. these activities allowed for repetition and use of words in multiple contexts, which allowed students to grasp word meaning and required them to use them in more than one context. therefore, students were able to expand their knowledge, apply the knowledge to the content area materials, and extend beyond the content with various activities. each of the multi-component vocabulary instructional strategies was included within the curriculum guides provided for all treatment group teachers. additionally, treatment group teachers were provided with supplemental materials, games, and activities for implementation of the curriculum. the first two weeks of materials were also copied and placed in student folders for every student in their classes. explanation of materials and their appropriate use were provided in the training sessions prior to implementation of the curriculum. the teachers in the treatment groups followed the lesson plans created for this study to implement the vocabulary strategies. the teachers were asked to spend a minimum of 90 minutes per week, preferably in a 30 minute three times per week format, if time and schedules permitted, to implement this intervention. the acquisition and maintenance of the content learned through vocabulary strategies was assessed through pretests, six-week posttests, and additional testing six weeks post-intervention. the types of materials that were utilized are described above and an example of a connections web is included in the appendix. researchers met with the participating teachers four times during the study. the first meeting was an overview of the study, and at the request of the school administrators, included teachers in both the treatment and comparison groups. following a brief introduction, the comparison teachers were dismissed, and the treatment teachers were trained regarding the implementation of the vocabulary intervention strategies. no specific information regarding the actual study and/or implementation of strategies was discussed with the group as a whole. a second and third training session occurred with both the treatment and comparison groups to discuss testing procedures. the second session was for the first set of tests and the third was held before administration of the effect of multi-component vocabulary instruction / graham, graham & west 151 posttests. one final meeting prior to implementation was also held to review, answer questions, and ensure that all teachers were ready to begin at the inception of the second six weeks of the school year. these meetings were held separately in each of the three districts. also, as the tests were delivered and picked up, individual visits with teachers were held as needed. testing procedures were planned in each case so that each classroom was assigned a particular format for administration to vary the order in which tests were administered. fidelity of implementation observation of the treatment and comparison group teachers and their classrooms took place beginning the sixth week of the school year -before, during, and after the study period. observation was conducted by one of the authors and four trained data collectors. the data collectors had a number of years of experience in visiting classrooms. an original training session was held to explain the process and describe the research. each data collector was provided with a notebook that included basic information about the study and the materials they would need as they entered each classroom. the forms were discussed in great detail and bi-weekly meetings were held to discuss any questions or concerns. the senior author also observed each of the four data collectors on two separate occasions. regular contact with the data collectors through meetings, email, and phone calls ensured continuity of the data collection. the data collectors observed each classroom a minimum of once each week during the social studies instructional period, using a checklist based from materials developed by the teacher quality grant (simmons et al., 2005), containing six sections. the first section included the beginning and ending times of the observation, the name of the district and school, treatment vs. comparison group, maximum number of students in the classroom, and maximum number of adults in the classroom during the observation. in the second section, observers were asked to look for seven different comprehension strategies and note the level of implementation, i.e. whether a strategy was modeled, it was explained, or students practiced it. thirdly, observers were asked to look for seven vocabulary strategies and tally how often they occurred during the class period. observation of the teacher providing explanations, definitions, or examples of vocabulary, and/or extension to include paraphrasing, and/or multiple meaning words and the use of visuals, facial expressions, demonstrations, the use of word learning strategies, demonstrated knowledge of words by the students with teacher responses and specific application of word learning strategies were all included. fourthly, grouping arrangements (teacher working with: whole class, large group, small groups, pairs, individual student, no direct student contact) and text reading (supported oral reading, independent silent reading, independent oral reading, teacher reads aloud, teacher reads aloud with students following, text not used for comprehension instruction) were coded every 15 minutes during the observation. in the fifth section, observers were asked to check thirteen different possibilities of materials used during the observation, including visuals, textbooks, computers, workbook pages, chalkboards, videos, and audio tapes among others. finally, the observers noted the implementation of intervention instruction. it was broken down into thirteen different categories, and coded as to the level of implementation being: none of the time, part of the time, or full time. the quality of implementation was rated on a 0-2 scale associated with unacceptable, acceptable, and excellent. teachers were cooperative about allowing visits at different times and on different days. school schedules for special days and/or activities necessitated an occasional change in observation schedules. international electronic journal of elementary education vol.8, issue 1, 147-160, 2015 152 measures three different measures were administered to all the students to address the two research questions examined in this study. the test of silent contextual reading fluency (toscrf) was administered at the beginning of the study to determine students’ reading ability, and assess if it was a confounding factor. to address the first research question, “what is the effect of multi-component vocabulary instruction on comprehension?,” the researchers administered two tests: a curriculum-based measures (cbm) test, and a checkpoints for content (checkpoints) test. the cbm was strictly a vocabulary test that served as a baseline to determine if students’ vocabulary performance improved during the intervention. the checkpoints test was similar to a unit test and was designed to measure students’ performance in comprehension. to address the second research question, “is any difference sustained?,” both measures were administered three times: before the intervention, at completion of the intervention, and six weeks following the conclusion of the intervention. test of silent contextual reading fluency (toscrf, 2006). according to hammill, wiederholt, and allen (2006), this test: measures the speed with which students can recognize the individual words in a series of printed passages that become progressively more difficult in their content, vocabulary, and grammar. the passages that the students are given to read are adapted from passages in the gray oral reading tests-fourth edition (cited in wiederholt & bryant, 2001) and the gray silent reading tests (cited in wiederholt & blalock, 2000). (p. 1) the test has a two-minute practice section, and then a three-minute exam section. the test requires knowledge of word identification, meaning, sentence structure, and comprehension. it was intended for the purposes of this study to serve as a measure of the students’ reading ability. it was administered only in the beginning with the pretest measures. the toscrf is a timed measure in which students must recognize individual words in a series of printed passages. the passages get progressively more difficult with regard to content and vocabulary. toscrf was normed using a national representative sample of 1,898 individuals in 23 states and for stated purposes, it was demonstrated to be both valid and reliable. curriculum-based measure (cbm). the vocabulary matching curriculum-based measure was administered as a pretest, posttest 1 after six weeks, and posttest 2 six weeks postintervention. the cbm was used as a fluency measure for vocabulary. the measure was timed for five minutes. it was in a matching format with 20 social studies words and their definitions. the cbm was adapted from the teacher quality grant (simmons, rupley, hairrell, byrns, vaughn, & edmonds, 2005). it was patterned after the work of espin, shin, and busch (2005), who discussed the importance of measuring change in students. typically, measurement is at a single point in time, which is evident with achievement testing and other standardized tests measures. curriculum-based measurement provides an ongoing data collection system that provides teachers with information on student progress, and in this case, on the progress of the intervention (espin et al., 2005). espin et al. (2005) completed a study to determine whether or not vocabulary-matching probes could be used as an indicator to determine student learning in social studies. their research supported the use of these measures. the cbm created for this study was formed following their model of five-minute, group-administered, vocabulary matching probes. this measure has been supported by other researchers as well (deno, 1985; francis, shaywitz, stuebing, shaywitz, & fletcher, 1994; fuchs & fuchs, 1998). checkpoints for content (checkpoints). checkpoints for content was a teacher and researcher created multiple choice exam similar to a unit test. the checkpoints were the effect of multi-component vocabulary instruction / graham, graham & west 153 adapted from the teacher quality grant materials and existing measures from the individual districts. two fourth-grade teachers from two different districts not associated with the study assisted in preparation of the questions. the intention was to measure for comprehension of specific expository text material. there were 20 multiple choice questions derived specifically from the districts’ curriculum. the checkpoints were administered as pretests, posttests 1, and posttests 2. analysis demographic information was collected and tabulated for both students and teachers participating in this study. the dependent variable in this study was the student outcomes including scores on the pretests, posttest 1 measures, and posttest 2 measures. descriptive statistics were performed on the fluency, cbm, and checkpoints measures. difference score analysis was also performed on the cbm and checkpoints measures. additionally, results of one-way repeated-measures analyses of variance (anova) were computed using the effect for group, effect for time, and effect for group by time or interaction effect on both test measures. results demographic information on the study participants and their respective campuses is presented in table 1. overall, the study population very closely approximated the reference population in terms of ethnic and socioeconomic characteristics. no significant differences were observed for gender or ethnicity between the experimental and control groups. table 1. gender and ethnicity of 4th grade students in experimental and control groups. experimental (n= 207) control (n= 168) n % n % gender female 118 57.0 90 53.6 male 89 43.0 78 46.4 ethnicity asian 1 0.5 1 0.6 african-american 41 19.8 26 15.5 hispanic 11 5.3 12 7.1 other 2 1.0 3 1.8 white 152 73.4 126 75.0 descriptive statistics for the fluency measure are presented in table 2. the experimental group scored slightly higher than the control group, but the difference was not statistically significant. table 2. results of test of silent contextual reading fluency group n m* sd p control 168 74.71 50.77 0.268 experimental 207 79.86 39.05 *standardized toscrf scores. the first research question addressed in this study was, “what is the effect of multicomponent vocabulary strategy on comprehension?” two measures were used to address that question, the cbm and checkpoints tests. descriptive statistics for these two measures are presented in table 3. on the checkpoints measure, the control group had a mean of 11.00 for the pre-test and 13.38 for post-test 1. the experimental group had a international electronic journal of elementary education vol.8, issue 1, 147-160, 2015 154 mean of 11.37 for the pre-test and 14.13 for post-test 1. on the cbm, the control group had a mean of 4.53 for the pre-test and 8.63 for post-test 1. the experimental group had a mean of 4.14 for the pre-test and 13.27 for post-test 1. table 3. means and standard deviations for checkpoints and cbm pre-, post-test 1, and posttest 2. test measures group n mc sd checkpointsa pretest control experimental entire sample 134 196 330 10.99 11.32 11.18 3.604 3.528 3.557 posttest 1 control experimental entire sample 134 196 330 13.28 14.10 13.76 3.923 3.960 3.959 posttest 2 control experimental entire sample 134 196 330 12.86 14.35 13.75 4.276 3.690 4.000 cbmb pretest control experimental entire sample 143 193 336 4.59 4.11 4.32 3.349 3.222 3.280 posttest 1 control experimental entire sample 143 193 336 8.50 13.20 11.20 5.403 5.476 5.914 posttest 2 control experimental entire sample 143 193 336 9.02 13.15 11.39 5.795 5.669 6.069 a. 34 in the control group and 11 in the experimental group excluded because of incomplete checkpoints test scores. b. 25 in the control group and 14 in the experimental group excluded because of incomplete cbm test scores. c. number of items correct on each test, out of 20 items. difference score analysis demonstrates a significant difference between post-test 1 and pre-test for the intervention group on the cbm measure (table 4). the mean difference on the checkpoints measure was very similar between the two groups, and was not statistically significant. table 4. difference score analysis for preand post-test 1on cbm and checkpoints measures. test measure group n mean difference sd p checkpoints control 157 2.36 3.36 0.263 experimental 201 2.75 3.09 total 358a 2.56 3.21 cbm control 151 3.86 4.47 <0.001 experimental 196 9.03 4.38 total 347b 6.78 5.11 a. 11 in the control group and 6 in the experimental group excluded because of incomplete test scores. b. 17 in the control group and 11 in the experimental group excluded because of incomplete test scores. the effect of multi-component vocabulary instruction / graham, graham & west 155 research question 2 the second research question addressed in this study was, “is any difference in comprehension sustained?” to address this question, researchers administered both the cbm and checkpoints measures a third time, six weeks after the intervention had concluded (table 3). repeated measures anovas were conducted on each test measure, and the results are presented in table 5. for the checkpoints test, the wilks’ lambda for the effect of time was 0.590, p< 0.001, with a partial 2 of 0.410. the effect for group by time produced a value of 0.970, p= 0.006, with a partial 2 of 0.030. for the cbm test, the wilks’ lambda for the effect of time was 0.297, p< 0.001, with a partial 2 of 0.703. the effect for group by time produced a value of 0.744, p< 0.001, with a partial 2 of 0.256 (table 5). table 5. results of repeated measures anovas for cbm and checkpoints measures. effect wilks’ lambda p partial 2 cbma time 0.297 <0.001 0.703 group*time 0.744 <0.001 0.256 checkpointsb time 0.590 <0.001 0.410 group*time 0.970 0.006 0.030 a. 25 in the control group and 14 in the experimental group excluded because of incomplete cbm test scores. b. 34 in the control group and 11 in the experimental group excluded because of incomplete checkpoints test scores. conclusions this study focused on the need for research in the area of vocabulary instruction. in particular, fourth grade students were selected for multi-component strategy instruction in their social studies classrooms. according to bromley (2007), teaching vocabulary well is a key aspect of developing engaged and successful readers. additionally, nagy and scott (2000) described word meanings as making up as much as 70-80% of comprehension. the importance of vocabulary was well-documented; the existence of intervention studies to support the importance of intense vocabulary instruction was not. the nrp (2000) emphasized that vocabulary learning was effective if students were actively engaged in their learning. the curriculum for this study was designed to actively engage students and to reinforce retention of word meanings in isolation as well as in context. the first question to be addressed in this study was: “what is the effect of multicomponent vocabulary instruction on fourth grade students’ social studies vocabulary and comprehension performance during a six-week period?” the cbm test, consisting of 20 items (words and definitions) in a matching format, strictly measured vocabulary performance. a sample question from the cbm required students to match the term “livestock” with the definition of “animals that are kept or raised for use and profit.” the checkpoints test, however, was designed as a multiple-choice unit test covering the same material the students were learning, but sourced from a separate curriculum. a sample checkpoints question was “if you went on a field trip to view livestock, you would probably see ______,” and required the students to select from the following four options: “a. corn, wheat, grain; b. hay, fertilizer, irrigation; c. cows, sheep, turkeys; and d. tractors, barbed wire, wheel barrows.” international electronic journal of elementary education vol.8, issue 1, 147-160, 2015 156 it would be expected that the increased vocabulary instruction in the experimental group would result in improved scores on the cbm, which is a strict vocabulary measure. however, if vocabulary instruction also impacts comprehension, then scores on the checkpoints measure should improve as well. table 4 shows the mean difference between the pre-test and post-test 1 was greater for the experimental group in both measures than for the control group. however, the difference for the cbm was statistically significant, while the difference for the checkpoints measure failed to achieve statistical significance. in contrast, when looking at the results from all three pre-test, post-test 1, and post-test 2 scores to answer the second research question, if any difference is maintained, both cbm and checkpoints measures show a statistically significant interaction for group by time. however, the partial 2 for the cbm measure is much larger (0.256) than for the checkpoints measure (0.030). this would indicate that the vocabulary intervention had a greater impact on the cbm measure (strict vocabulary measure), while a smaller, but still significant impact on the checkpoints measure (comprehension measure). it is interesting that no significant difference for the checkpoints measure was observed in the first six weeks, but a difference did emerge at the 12-week follow-up point. this may be due to the fact that both groups scored higher initially on the checkpoints pre-test (overall mean = 11.18 / 20) as compared to the cbm pre-test (overall mean = 4.32 / 20). the higher pre-test scores on the checkpoints test would mean there is less room for improvement as compared to the cbm test. additionally, after the intervention, scores on the checkpoints measure fell slightly for the control group, while they continued to improve for the experimental group. this may indicate that the effect of vocabulary instruction on improved comprehension performance may be prolonged past the duration of the intervention. finally, the fluency scores for the experimental group were slightly higher than those of the control group. while the difference was not statistically significant, we would expect such a result to bias the results of the checkpoints measure toward the null. one of the challenges of this study design was the implementation of the vocabulary intervention. while teachers were receptive to participating in the study, they did have two primary concerns. one was associated with the lack of time to complete the instructional strategies provided, and the other concern was whether or not the focus on vocabulary was allowing them to be inclusive enough with the content in the textbook. the study was intended to combine vocabulary and content and demonstrate that teaching content with students who do not understand the vocabulary could be futile and certainly would not contribute to long-term retention of content knowledge. however, the teachers wanted more time to read and focus on the material in the text, in addition to the vocabulary focus. therefore, time was a factor. additionally, no assessment was made as to the level of baseline vocabulary instruction that occurred during the study period among the control teachers. while teachers in the experimental group were asked to spend ninety minutes each week on the vocabulary strategies, it is possible that some of the control teachers were independently covering the same material as well. however, since the teachers were randomly assigned to either the control group or experimental group, we would not expect any difference in teaching methodology between the two groups. furthermore, if the control teachers were independently using similar vocabulary strategies, we would expect the results to be biased towards the null. one of the strengths of this study was the positive reception from the teachers involved. on a post-study survey, almost all of teachers agreed that their personal instructional practices had changed as a result of participating. additionally, they also the effect of multi-component vocabulary instruction / graham, graham & west 157 agreed that their students were gaining knowledge as a result of the intervention. teachers in the classroom face many varied demands for their time and attention. implementing an intervention that not only demonstrated statistically significant improvement in comprehension, but also was of practical significance and value to the teachers themselves increases the potential benefit to other teachers. finally, the instruments used in this study were demonstrated to have high levels of internal reliability. reliability analyses of the checkpoints and cbm measures were conducted, with both measures revealing an alpha greater than 0.7, the standard for internal consistency set out by pallant (2005). the teachers from this study initially self-reported that they were unable to spend the time on vocabulary. however, careful instruction requires that in order to convey content, students must have an understanding of the vocabulary (anderson & freebody, 1981; durkin, 1993; yildirim, yildiz, & ates, 2011). experimenting with practices to determine their effectiveness is critical for improving our classroom instruction (cunningham and zibulsky, 2009). as demonstrated in this study, multi-component vocabulary instruction in 4th grade social studies improved performance in both vocabulary and comprehension areas. retention of that improvement was demonstrated as well. while addressing vocabulary can be a time-consuming process, its contribution to success in comprehension of content cannot be underestimated, particularly in helping those who might otherwise fall into the group of those who fall short in terms of literacy development (nation, 2008). further research studies should be conducted in classrooms over other content areas to broaden our understanding of vocabulary instruction and the resulting effect on comprehension for all students. several teachers and administrators also noted that this instruction would be beneficial in esl classrooms. regardless of the setting or content area, literacy development is crucial, and multi-component vocabulary strategies have been shown to be a key part of that process (simmons et al., 2005). references aaron, p., joshi, r. & quatroche, d. (2008). becoming a professional reading teacher. baltimore: paul h. brookes publishing co. anderson, r., & freebody, p. (1981). vocabulary knowledge. in j. t. guthrie (ed.), comprehension and teaching: research reviews (pp. 77-117). newark, de: international reading association. armbruster, b., lehr, f., & osborn, j. (2003). put reading first: the research building blocks for teaching children to read. washington, dc: the partnership for reading, national institute for literacy, national institute of child health and human development, & the u.s. department of education. beck, i., & carpenter, p. (1986). cognitive approaches to understanding reading implications for instructional practice. american psychologist, 41, 1098-1105. beck, i., mckeown, m., & gromoll, e. (1989). learning from social studies texts. cognition and instruction, 6, 99-158. beck, i., mckeown, m. & kucan, l. (2008). creating robust vocabulary. the guilford press: new york. beck, i., mckeown, m., sinatra, g., & loxterman, j. (1991). revising social studies text from a textprocessing perspective: evidence of improved comprehensibility. reading research quarterly, 26, 251-276. international electronic journal of elementary education vol.8, issue 1, 147-160, 2015 158 biemiller, a. (1999). language and reading success. brookline, ma: brookline books. blachowicz, c., & fisher, p. (2000). vocabulary instruction. in m.l. kamil, p.b. mosenthal, p.d. pearson, & r. barr (eds.), handbook of reading research (vol. 3, pp. 503–524). mahwah, nj: erlbaum. blachowicz, c. & fisher, p. (2002). teaching vocabulary in all classrooms. upper saddle river, new jersey: merrill prentice hall. bromley, k. (2007). nine things every teacher should know about words and vocabulary instruction. journal of adolescent and adult literacy, 50, 528-537. brown, v., hammill, d. & wiederholt, j. (1986). test of reading comprehension: a method for understanding written comprehension of language. austin, texas: pro-ed. carlisle, j.f., & katz, l.a. (2005). word learning and vocabulary instruction. in j. birsch (eds.), multisensory teaching of basic language skills, second edition. baltimore: paul h brookes. carreker, s. & birsh, j. (2005). multisensory teaching of basic language skills activity book. baltimore: paul h. brookes publishing co. chall, j., & jacobs, v. (2003). the classic study on poor children’s fourth-grade slump. american educator, 27, 14-15. coyne, m., mccoach, b., & kapp, s. (2007). vocabulary intervention for kindergarten students: comparing extended instruction to embedded instruction and incidental exposure. learning disability quarterly, 30, 74-88. cunningham, a. & zibulsky, j. (2009). introduction to the special issue about perspectives on teachers’ disciplinary knowledge of reading processes, development and pedagogy. reading and writing, 22, 375-378. deno, s. (1985). curriculum-based measurement: the emerging alternative. exceptional children, 52, 219-232. durkin, d. (1993). teaching them to read. boston: simon & schuster. espin, c., shin, j., & busch, t. (2005). curriculum-based measurement in the content areas: vocabulary matching as an indicator of progress in social studies learning. journal of learning disabilities, 38, 353-363. foil, c. & alber, s. (2002). fun and effective ways to build your students’ vocabulary. intervention in school and clinic, 37, 131-139. francis, d., shaywitz, s., stuebing, k., shaywitz, b., & fletcher, j. (1994). measurement of change: assessing behavior over time and within developmental context. in g. r. lyon (ed.), frames of reference for the assessment of learning disabilities: new views on measurement issues (pp. 2958). baltimore: paul brookes. fuchs, l., & fuchs, d. (1998). treatment validity: a unifying concept for reconceptualizing the identification of learning disabilities. learning disabilities research and practice, 13, 204-219. hall, l. (2004). comprehending expository text: promising strategies for struggling readers and students with reading disabilities? reading research and instruction, 44, 75-95. hammill, d., wiederholt, j., & allen, e. (2006). test of silent contextual reading fluency examiner’s manual. austin, tx: pro-ed. harmon, j.m. & hedrick, w.b. (2005). research on vocabulary instruction in the content areas: implications for struggling readers. reading & writing quarterly, 21: 261–280. joshi, r. (2005). vocabulary: a critical component of comprehension. reading and writing quarterly, 21, 209-219. juel, c. & deffes, r. (2004). making words stick. what research says about reading, 61, 6.association for supervision and curriculum development: alexandria, va. the effect of multi-component vocabulary instruction / graham, graham & west 159 nagy, w. (2009). understanding words and word learning: putting research on vocabulary into classroom practice. in rosenfield, s., & berninger, v. (eds.) handbook on implementing evidenced based academic interventions. new york: oxford university press. nagy, w., & scott, j. (2000). vocabulary processes. in m. kamil, p. mosenthal, & p. pearson (eds.), handbook of reading research (vol. 3, pp. 269-284). hillsdale, nj: erlbaum. nation, f. (2008). learning to read words. the quarterly journal of experimental psychology, 61, 1121-1133. national center for educational statistics (nces). (2011). reading report card. washington, dc: u.s. department of education, office of educational research and reform. retrieved from http://nationsreportcard.gov/reading_2011/reading_2011_report/ national reading panel (nrp). (2000). teaching children to read: an evidence-based assessment of the scientific research literature and its implications for reading instruction. washington, dc: national institute of child health and human development. neuman, s. & dwyer, j. (2009). missing in action: vocabulary instruction in pre-k. the reading teacher, 62, 384-392. pallant, j. (2005). spss survival manual. new york: open university press. perie, m., grigg, w., & donahue, p. (2005). the nation’s report card: reading 2005 (nces 2006451). washington, dc: u.s. government printing office, u.s. department of education, institute of education sciences, national center for education statistics. phythian-sence, c., & wagner, r. (2007). vocabulary acquisition: a primer. in r. wagner, a. muse, & k. tannebaum (eds.), vocabulary acquisition implications for reading comprehension (pp. 1-14). new york: guilford press. simmons, d., rupley, w., hairrell, a., byrns, g., vaughn, s., & edmonds, m. (2005). enhancing the quality of expository text instruction and comprehension through content and case-situated professional development (grant contract no. r305mo50121a). washington, dc: u.s. department of education’s institute of educational sciences. snow, c., burns, m., & griffin, p. (eds.). (1998). preventing reading difficulties in young children. washington, dc: national academy press. u.s. government printing office. (2001, march 8). measuring success: using assessments and accountability to raise student achievement (hearing before the subcommittee on education reform of the committee on education and the workforce, house of representatives, one hundred seventh congress, first session). washington, dc: author. vaughn, s., & linan-thompson, s. (2004). research-based methods of reading instruction grades k-3. alexandria, va: association for supervision and curriculum development. vitale, m. & romance, n. (2008). broadening perspectives about vocabulary instruction: implications for classroom practice. the nera journal, 44, 15-22. wiederholt, j. & blalock, g. (2000). gray silent reading tests. austin, tx: pro-ed. wiederholt, j. l., & bryant, b. r. (2001). gort 4 gray oral reading tests examiner’s manual. austin, tx: pro-ed. williams, jp; hall, km; lauer, kd; stafford, b; desisto, la; and decani, j. (2005). expository text comprehension in the primary grade classroom. journal of educational psychology, 97(4), 538550. yildirim, k., yildiz, m., & ates, s. (2011). is vocabulary a strong variable predicting reading comprehension and does the prediction degree of vocabulary vary according to text types. educational sciences: theory into practice, 11(3), 1541-1547, zimmerman, s. & hutchins, c. (2003). 7 keys to comprehension. three rivers press: new york. international electronic journal of elementary education vol.8, issue 1, 147-160, 2015 160 appendix a microsoft word iejee_4_2_wood international electronic journal of elementary education, 2012, 4(2), 317-345. issn:1307-9298 copyright © iejee www.iejee.com teacher perceptions of gender-based differences among elementary school teachers ∗∗∗∗ tracy darrin wood ∗∗∗∗∗∗∗∗ saint louis university, united states received: september 2011 / revised: february 2012 / accepted: february 2012 abstract the purpose of this study was to examine teacher perceptions of gender-based differences among elementary school teachers. in this mixed-methods study, the researcher utilized an online survey to collect data. quantitative analysis revealed no statistically significant differences; however, qualitative analysis showed that there were more negative responses to survey statements pertaining to males. more than half of the participants perceived differences between female and male elementary teachers, including beliefs that female teachers are more nurturing, that male teachers are more laid back, and that male teachers are more dominant and commanding with students. four conclusions were drawn: elementary teachers perceive differences between female and male teachers, male elementary teachers are perceived more negatively than female colleagues, a vast majority of participants feel that more male elementary teachers are needed in the elementary schools, and differences between female and male elementary teachers remain unclear. keywords: teacher gender, differences, elementary education, perceptions, male teacher introduction gender differences provide content for discussion in a variety of media including movies, television, books, magazines, and research; however, differences between female and male elementary teachers have not been fully researched and analyzed. according to the national center for education statistics, women outnumber men in the teaching profession by approximately three to one (2006). in 2001, the national education association reported that only 9% of america’s elementary school teachers were men. more than a decade later, the situation has improved little; for example, the bureau of labor and statistics (as cited by the ∗ this article is written from the author’s doctoral dissertation research. ∗∗ tracy darrin wood, saint louis university, college of education and public service, 3500 lindell blvd., saint louis, missouri, 63103, united states, phone: (636) 891-6150, e-mail: drtracywood@gmail.com international electronic journal of elementary education, 2012, 4(2), 317-345. 318 men teach organization) reported that 18.8% of elementary and middle school teachers in america were men. jones (2003) cites female teachers’ concerns about the comparatively low number of men in the teaching profession and their hope of having more male elementary colleagues to offset the disproportionate number of female teachers. one might conclude that this deficit would result in an unacceptable balance in elementary teacher gender demographics. the lack of balance could lead to a difference in perceptions about female and male teachers. in addition, this lack of balance could be caused by a difference in perceptions. this research study uses a phenomenological approach, which glesne (2006) defines as the description of an individual’s consciousness and experience of a phenomenon. phenomenological research analyzes the research participants’ thoughts, beliefs, and perceptions; for example, this study has gathered qualitative and quantitative data regarding female and male elementary teachers’ perceptions of their colleagues. the purpose of this research was to examine the perceptions of teachers related to gender differences in elementary teachers. the data from this research may underscore the need to increase the male teacher population at the elementary school level. educators may also use this data to improve teacher education programs and professional development. the main research question was: what are teacher perceptions of gender-based differences among elementary teachers? for the purpose of this study, the researcher will examine the issue by grouping previous literature into three domains: teacher as self, teacher to student, and teacher to teacher. it is through these domains that the researcher implemented his own study. teacher as self in the teacher as self domain, literature was analyzed that discussed teachers’ classroom management style. previous research was also analyzed that focused on teachers’ content knowledge and instructional effectiveness. regarding classroom management, chudgar and sankar (2008) noted that male teachers were more focused on maintaining classroom authority by enforcing strict discipline. furthermore, green et al. (2008) identified one possible difference between female and male teachers’ classroom management; female teachers perceive behavior challenges to be more severe than do their male colleagues. however, yavuz (2009) found no difference between female and male teachers’ attitudes about classroom management or their instructional management. carrington et al. (2008) found that teacher gender had no effect on student achievement. previously, alghazo (2005) had noted that there was no difference in teachers’ perceptions of the delivery and evaluation of instruction based on gender, but added that there were statistically significant differences in the amount of time that female and male special education teachers spent on planning instruction and how they rated the effectiveness of their instructional management. in alghazo’s study, male teachers rated themselves as spending more time planning instruction than did their female colleagues. in addition, male teachers rated themselves higher than did female teachers in instructional management, contradicting yavuz’s later findings. dee (2006) states that “a teacher’s gender does have large effects on student test performance, teacher perceptions of students, and students’ engagement with academic material” (p. 71). the argument, dee asserts, is that boys perform better when taught by male teachers and girls perform better when taught by female teachers. by contrast, mullola et al. (2011) stated that teacher gender had no effect on student grades. driessen (2007) notes two differences between female and male teachers. one difference is that, in year 6, male teachers in the netherlands tested their students more often than did female teachers. second, in year 8, male teachers emphasized cognitive gender-based differences among elementary school teachers / t. d. wood 319 objectives more often in the classroom. according to driessen, the difference in teacher education between female and male teachers was statistically significant, suggesting that because males have more education, they also may have more content knowledge than their female colleagues. teacher to student the teacher to student domain focused on various aspects of the teacher–student dynamic. it focused on teachers as role models and their ability to be nurturing, patient, fair, and sensitive to their students’ needs. it addressed teachers’ ability to motivate and engage their students. finally, it discussed teachers’ general desire to work with children and their ability to develop relationships with their students. according to studies by martin and marsh (2005), driessen (2007), and martin et al. (2008), neither boys nor girls are motivated any more or less by female or male teachers; they found no differences in this regard between the abilities of female and male teachers. martin and marsh believe that motivation was more of a student factor than a teacher factor, and that an individualized approach would yield the greatest results. in her study, jones (2003) found that female teachers overwhelmingly responded that male teachers would be better at motivating boys than would female teachers. however, her research also demonstrates that the longer the interview process lasted with female teachers, the more they would discuss limitations in male teachers’ ability to motivate male students. in the mid-1900s, females were thought of as being perfectly capable of serving as role models for all students. wiest (2003) notes that women were “deemed better suited to serve as role models and teachers of moral behavior” and that they possessed “emotional qualities to work with youth” (p. 63). even though driessen’s (2007) research emphatically shows that there is no statistically significant difference between men and women serving as role models, he raises probing questions about teachers. for example, did students even look up to teachers as role models? he goes on to state: it is also the case that just as male teachers do not always constitute a suitable role model for boys, female teachers do not always constitute a suitable role model for girls. furthermore, such matching according to sex may simply reinforce and strengthen stereotypes at times. to be viewed as a role model by their pupils, teachers must earn respect and admiration (p. 186). men are considered good possible role models not only for regular education students, but also for students in special education. rice and goessling (2005) state that schools need more male special education teachers to provide a “balanced educational experience for their students” (p. 354). even if a male teacher can be a positive role model for his students, martino and kehler (2006) argue that schools cannot solve “boys’ diverse educational and social problems” by simply providing male teachers in the building (p. 125). cooney and bittner (2001) raise the concern of some teachers “that a poor male role model could do a lot of damage in the classroom and it is important to be aware that good teaching is critical” (p. 81). noted author and researcher carol gilligan (1982) proposed that females have a unique ability to make moral decisions pertaining to personal and caring aspects of development. cooney and bittner discuss the stereotype that males do not nurture, commenting that this was a “barrier to recruiting males to the field” (p. 80). males are not stereotypically associated with nurturing and sensitivity. data collected by jones (2003) and cushman (2005) suggest that males who display a combination of sensitivity and masculinity are the most desired in elementary education. in their research, carrington et al. (2008) states that “it was clear that students who had female teachers had more positive attitudes” toward school (p. 321). williams-johnson et al. (2008) explain that teachers can engage students with their own international electronic journal of elementary education, 2012, 4(2), 317-345. 320 brand of nurturing, adding that teachers will nurture according to their own pre-existing beliefs about the most important aspects of the educational process. this would indicate that, unless male teachers identify the importance of nurturing in the classroom, they would not be effective at nurturing their students. according to tatar and emmanuel’s research (2001), male elementary teachers “tended to give more egalitarian responses” (p. 221) to students in certain situations. the authors explain this trend by commenting that male elementary teachers were working against the stereotype that they were not as fair as female teachers. huang and fraser (2009) found that even after controlling specific variables, such as background and characteristics of schools, male teachers perceived better relationships with their students than did female teachers. teacher to teacher the third domain focused on teachers’ perceptions of their colleagues’ ability to develop effective relationships. this domain also discussed the expected jobs of men and women in the elementary education setting. huang and fraser (2009), wiest (2003), and cooney and bittner (2001) suggest that collegiality may be of greater importance to male elementary teachers because of their lower numbers. weist also claimed female teachers wish males fit in better and made more of an effort to be team players. as suggested earlier, prospective male teachers often do not enter the profession because of a lack of male peers and the societal perception of the profession being feminine. cooney and bittner found, through interviewing teachers, that isolation was a major issue for male teachers. one interviewee stated that he did not feel comfortable sharing some of his concerns or successes with the female teachers in his building. this research was extended by smith (2004), who concluded that loneliness and lack of opportunities for socializing were detriments to attracting more men into the field. through their qualitative research studies, jones (2003) and wiest found that female teachers expressed a desire to have male colleagues who would be good listeners, team workers with a sense of humor, and not arrogant. they wanted a male teacher who could “fit in” and contribute to the overall team atmosphere of the school. rice and goessling (2005) assert that “contact with men in schools, particularly elementary schools, is usually limited to interactions with the principal, janitor, or physical education teacher or coach” (p. 348). in her native country of the netherlands, driessen (2007) found that only 23% of headmasters, or principals, were women, whereas 84% of teachers were women. in addition, 77% of the support staff were women. these statistics illustrate a trend common to many countries: although most administrative positions are held by men, the majority of teachers are women. piper and collamer (2001) identify the difficulties that men have in typically female jobs (e.g., librarian). the male librarians they interviewed felt that at least occasionally, females in the building asked them to perform traditionally male jobs such as lifting and carrying heavy objects. however, the male librarians felt that although they were treated as “special because of their maleness, they never were left out because of it” (p. 408). in addition to the role of the librarian, rice and goessling found that special education also attracts few men. librarians and special educators are predominantly women. this study explored teachers’ perceptions of female and male colleagues’ effectiveness in the three domains previously discussed. understanding teachers’ perceptions of gender-based differences among elementary school teachers can lead to improvements in teacher education and professional development programs. it also may lead to the recruitment of more men into the teaching profession. in addition, collecting teachers’ perceptions of their colleagues and fully analyzing such data may lead educators to a better understanding of why there are so few male elementary teachers. gender-based differences among elementary school teachers / t. d. wood 321 method participants it was encouraging to note that the majority of respondents to the online survey were experienced teachers and, therefore, potentially able to provide more comprehensive information. an impressive 43% of participants had 13 or more years teaching experience. also, a combined 39% of participants had 5 to 12 years of teaching experience. a combined 80% of survey respondents had attained master’s degrees or higher. the researcher assumes that the varied teacher experience provides rich, descriptive, and meaningful data that inform the findings of this study. survey respondents’ experience in their own classrooms and their education play an important role in this study’s contribution to the field of education. although not essential to the purpose of this study, it is interesting to note that of the 215 respondents who answered the item regarding race/ethnicity, 206 were caucasian. of the remaining nine, five were african american, two were hispanic, one was native american, and one was asian american. data collection method: the online survey in this study, the researcher used a mixed-methods approach with an online survey (appendix a) to gather data. according to patton (2003), a survey or questionnaire that solicits responses by asking fixed choice, or closed-ended, questions as well as open-ended questions is an example of how qualitative inquiry and quantitative measurement are combined in a single research study. glesne (2006) suggests that open-ended questions focusing on teachers’ perceptions of the past or present “tend to be richer ground” for descriptions (p. 82). participants also had the opportunity to add comments describing their experiences and clarifying their responses to closed-ended questions. johnson and christenson (2008) state that using closed-ended items “exposes all participants to the same response categories and allows standardized quantitative statistical analysis” (p. 177). this research collected quantitative information through closed-ended items from the survey and allowed the researcher to use numerical data to explore differences between female and male teachers. as dillman (2007) points out, the principle of standardization applies because the researcher is providing repeated stimuli such as item stems, response categories, and additional information to the participants. furthermore, johnson and christenson state that the principle of standardization is utilized to ensure maximum response comparability. this researcher compared the responses of female and male participants. although only 25 of the 217 participants were male teachers, a thorough analysis of teachers’ perceptions was still conducted. development of the survey the primary instrument for data collection was an online survey questionnaire that was developed by the researcher. the researcher piloted the survey items with a group consisting of former elementary teachers currently teaching at the middle school level, along with elementary teachers working at elementary schools other than his own. the group consisted of both women and men; the researcher believed their experiences would provide valuable feedback to aid in the development of the online survey. the researcher further developed the online survey questionnaire for this study in collaboration with dr. rina m. chittooran, educational studies faculty at saint louis university. a third party vendor, key survey, was chosen by the researcher to administer and distribute the survey and to collect feedback from the participants. sue and ritter (2007) and czaja and blair (2005) identify essential components to an internet survey, including a brief message stating the purpose, encouraging participation, and providing clear directions for beginning and completing the survey. advantages of an internet survey include low cost and speed of data collection (czaja international electronic journal of elementary education, 2012, 4(2), 317-345. 322 & blair 2005; sue & ritter 2007). czaja and blair have found that response time for internet surveys ranges from 10 to 20 days and explain that the “ability to obtain reasonably complete and detailed answers to open-ended questions may prove to be an advantage of internet surveys…” (p. 41). the geographic distribution of the sample and sample size has no effect on the cost of an internet survey (czaja & blair 2005; sue & ritter 2007). although research has confirmed that the internet survey is a viable means of data collection, there are some disadvantages to note. a low response rate is one such disadvantage (czaja & blair 2005; gilligan, 2007; sue & ritter 2007). the response rate with this study was 19%, falling short of sheehan’s (2001) recommended rate of 36.83%, and is one limitation to the generalizability of this study. this could be due to participant disinterest or technical difficulty, such as slow telecommunications speed, unreliable internet connections, and lowend browsers. response bias is another disadvantage to internet surveys. teachers may respond to the items in a way that will appease the researcher, instead of answering truthfully. czaja and blair suggest that another disadvantage to an internet survey is that the researcher has “no control over who actually completes the questionnaire” (p. 44). the first section was designed to gather both quantitative and qualitative information pertaining to participants’ experiences of working with female and male teachers. the survey questionnaire includes items in a likert format to which participants were asked to respond as follows: 5 – strongly agree, 4 – agree, 3 – neither agree nor disagree, 2 – disagree, or 1 – strongly disagree. there was space provided after each item for participants to offer additional information. the second section was the qualitative portion of the study consisting of open-ended items designed to gather information about participants’ in-depth experiences. the third section contained demographic items for participants to provide basic information about themselves and their work settings. reliability and validity survey questionnaires are inherently weak with validity but strong in reliability (colorado state, 2007). lincoln and guba (1985) explain that validity is approached differently in a qualitative study than in a quantitative one. some qualitative researchers refer to validity as trustworthiness. glesne (2006) asserts that trustworthiness is an issue that should be addressed in research design and throughout data collection. in this study, peer review was used to augment trustworthiness. glesne describes peer review as external reflection and input on one’s work. the researcher utilized the suggestions of peers from saint louis university throughout this process. another method used to strengthen validity was clarification of researcher bias, a process defined by cresswell (2003) as continuous reflection on the part of the researcher in order to minimize interference from his attitudes, opinions, or perceptions. because the researcher is himself a male elementary teacher, the potential for researcher bias in this study was significant. glesne describes the effectiveness of researchers’ reflection regarding their own subjectivity and how they use and monitor such reflection. furthermore, the researcher used evidence based on content; this is defined by johnson and christenson (2008) as a “judgment on the degree to which the evidence suggests that the items, tasks, or questions” appropriately represent the researcher’s topic of interest (p. 152). data collection procedures the survey questionnaire development process was continuous up until the point of administration, thus allowing the researcher to maximize the tool’s effectiveness. a district recruitment letter was sent to four school districts asking for permission to recruit teachers at their elementary schools. after the district officials agreed to participate, they documented their approval on notification of assent to participate letters. once approval was granted, the gender-based differences among elementary school teachers / t. d. wood 323 school districts provided the researcher with the elementary teachers’ e-mail addresses. the researcher uploaded the e-mail addresses into key survey, which then distributed invitations to teachers requesting that they participate in the study. each invitation to participate included a link to the survey website, along with the purpose of the study and instructions for completing and submitting the internet survey. the e-mailed invitation also outlined the intent of the survey to the potential participants and stressed that the feedback would be strictly confidential. it also explained that although no complete guarantee could be provided, comprehensive measures were taken to minimize risk of data loss or compromise. the recruitment letter explained that completion of the survey implied consent to participate in the study. key survey e-mailed approximately 1 170 surveys to teachers working in four public school districts around a major metropolitan city in the midwestern united states. two hundred seventeen teachers responded to the survey questionnaire. key survey ensured that each participant responded only once to the survey. it was not necessary to answer the survey items in order; the participant could skip questions, and later return to provide a response. all survey items provided adequate space for the participants to submit written comments and feedback. key survey provided contact information in case of technical difficulties completing or submitting the survey. data security was ensured by various measures used by key survey, including ensuring the security of its website, data encryption, secured socket layers (ssl), and firewall protection. fourteen days after the first invitation to respond, key survey sent a follow-up email to thank participants for completing the survey; it also encouraged any teachers who had not yet completed the survey to do so. finally, key survey collected data for analysis and the final report. the use of key survey allowed the researcher to maximize the advantages of the internet survey while minimizing the disadvantages associated with online surveys. data analysis once the data had been collected from the survey, the researcher analyzed the data for trends and variance. the vendor provided the total sum of responses, the response percentages, and charts and graphs that represented the collected data. the researcher thoroughly analyzed the survey responses for qualitative data that would provide insight into how female and male elementary teachers perceive one another. it is important to note that while there were no statistically significant findings in respondents’ perceptions of gender-based differences among their elementary school colleagues, there were qualitative differences. these have been reported and interpreted in subsequent sections. indeed, it may be that clinical findings such as these have greater implications for what actually happens in the elementary school setting. qualitative data were printed and coded for analysis of trends and variance. data were used to comprehensively identify the participants’ perceptions of their colleagues regarding gender. results and discussion the three domains in each of the following subsections, the survey results are grouped and discussed according to the three domains created by the researcher: teacher as self, teacher to student, and teacher to teacher. fixed-choice survey items are discussed first, followed by a discussion of the qualitative responses to open-ended items. it is important to note that some of the survey participants did not respond to every item, so the number of responding participants is also noted. teacher as self. this domain focuses on teachers’ individual classroom management style, content knowledge, and instructional effectiveness. regarding teachers having adequate international electronic journal of elementary education, 2012, 4(2), 317-345. 324 content knowledge, a combined 9.21%, or 21 participants, selected neutral (neither agree nor disagree) or negative (disagree and strongly disagree) responses for male elementary teachers, compared with 4.63%, or 10 participants, for female teachers. thus, some participants perceive their male colleagues as having an inadequate amount of content knowledge to teach their classes. table 1 summarizes the responses for the survey item pertaining to teachers’ content knowledge. table 1. teacher content knowledge the female teachers i have worked with have adequate content knowledge to teach their classes. the male teachers i have worked with have adequate content knowledge to teach their classes. (n) (%) (n) (%) strongly agree 125 57.87 108 49.77 agree 81 37.50 88 40.55 neither agree nor disagree 10 4.63 17 7.83 disagree 0 0.00 3 1.38 several comments were made by teachers indicating that “it always depends on the teacher” and that “some did and some didn’t” have adequate content knowledge. the second survey item regarding male elementary teachers’ content knowledge resulted in similar comments, such as, “depends on the teacher” and “male teachers also have the knowledge and experience in their areas.” item responses to male elementary teachers’ content knowledge, however, also provided several references to the limited number of male elementary teachers. there were three respondents who did not perceive their male colleagues as having enough content knowledge to teach their classes. it could be that in this suburb of a major metropolitan city in the midwest, both female and male elementary teachers have adequate content knowledge. however, the multiple responses regarding the low number of male elementary teachers could indicate that it is difficult to compare teacher content knowledge when there are so few male elementary teachers. regarding teachers having effective teaching styles, results from this pair of survey items show a perceived difference between the teaching effectiveness of female and male elementary teachers. although the percentage of teachers who responded positively (strongly agree or agree) for both female and male teachers is nearly identical, there are differences with almost every other response choice. female teachers obtained a mean score of 4.42 and male teachers obtained a score of 4.18 on a scale of 1 to 5. the range of the responses for male teachers having an effective teaching style was greater than that for female teachers. table 2 shows the mean and standard deviations of responses. table 2. effective teaching style the female teachers i have worked with have an effective teaching style. the male teachers i have worked with have an effective teaching style. mean 4.42 4.18 standard deviation .641 .855 the comments included with these two survey items were positive for female teachers; however, there were multiple negative comments regarding the effectiveness of male teachers. two comments by female teachers pertaining to male elementary teachers having gender-based differences among elementary school teachers / t. d. wood 325 effective teaching styles were “not as often as the female teachers” and “at times [they] can be overly strict.” one comment suggested a caveat that males’ effective teaching style is “partly from their physical presence.” this could suggest that the respondent perceived that teaching style effectiveness was due to gender-related factors, and not because of experience, educational attainment, or skill. there were also several comments stating that the lack of male elementary teachers made it difficult to accurately assess their teaching style. regarding teachers using effective classroom strategies, respondents perceive that the effectiveness of classroom management is not determined by gender. one comment read, “i have not gotten the sense that effective classroom management is a reflection of gender as much as it is the core beliefs that individual teachers hold about children…” multiple respondents again stated that the effectiveness of classroom management techniques depended on the teacher. four respondents felt that the female teachers they had worked with did not use effective classroom management strategies, yet 16 respondents felt this way about the male teachers. one comment from a female teacher read, “my male colleague tends to manage his class in a military style, and some students do not respond well to it.” this sentiment reflects a previous comment from a female teacher about male teachers’ teaching styles sometimes being “overly strict.” it could be that male teachers’ “strictness,” as perceived by the participant, helps them manage disruptive classroom behaviors more effectively than female teachers. however, an interesting contradiction from another female teacher also stood out. she stated, “i feel most males have better management skills… or the students tend not to pull as much for a male teacher.” this respondent perceived that male teachers either have better classroom management skills or their students behave better simply because of their teacher’s gender. teacher to student. the second domain pertains to the various aspects of the teacher–student dynamic. this relationship includes the ability to motivate and engage students, serve as a role model, and be nurturing, patient, fair, and sensitive to students’ needs. it also includes teachers’ general desire to work with children and to develop relationships with their students. regarding teachers’ abilities to motivate their students, a combined 93.52%, or 202 participants, felt that female elementary teachers are effective at motivating their students and keeping them engaged, whereas only 85.18%, or 184 participants, felt that male elementary teachers are effective in this area. a greater percentage of participants felt neutral or negative about male elementary teachers’ abilities to motivate and engage their students (14.82%) compared with female teachers (6.49%). more survey participants selected neutral or negative choices for male teachers’ classroom management than for female teachers. multiple comments by many participants were provided about both genders, such as “depends on the teacher,” “individual exceptions apply,” and “it’s the individual person.” as with the survey items regarding effective teaching styles, respondents selected neither agree nor disagree and disagree for male teachers more often than for female teachers. one respondent commented about two of her male colleagues that “one has [motivated his students effectively] and one has not [motivated his students effectively].” because of the low number of male elementary teachers, one male teacher who ineffectively motivates his students might artificially inflate negative stereotypes about male teachers; on the other hand, such inflation may not occur with the relatively large population of female teachers. regarding teachers serving as role models, although positive results were very similar for both genders, two differences emerged in the neutral and negative categories. one difference was that a greater number of respondents were neutral about the differences between the effectiveness of male teachers and female teachers as role models. another difference was that more respondents disagreed with this statement for male teachers than international electronic journal of elementary education, 2012, 4(2), 317-345. 326 for female teachers. these differences do not emerge in any other survey items in this study. table 3 lists the number and percentage of each response to these survey items. table 3. role models the female teachers i have worked with effectively serve their students as role models. the male teachers i have worked with effectively serve their students as role models. (n) (%) (n) (%) strongly agree 97 44.91 100 46.30 agree 97 44.91 95 43.98 neither agree nor disagree 22 10.19 14 6.48 disagree 0 0.00 7 3.24 eighteen of the participants commented that it depended on the teacher, not their gender. three separate comments about male teachers stand out: “good male teachers can be hard to find, and students can tell when they have a good male teacher.” “i think it is especially important to have males in elementary school, since so many children come from single parent homes.” “it has been my observation that a good male teacher in the building has a strong influence on kids.” it appears that a male teacher who can effectively serve students as a role model can have an important positive impact on students. conversely, a male teacher ineffectively serving as a role model is often perceived negatively by his colleagues. one participant commented about female teachers serving as role models, stating that “other than white teachers not being a role model for some city children,” most females were serving as positive role models. although this comment addresses the role that race and ethnicity play in teachers serving as role models for their students, it does not address any perceived differences between female and male elementary teachers. regarding teachers’ ability to effectively nurture their students, the survey revealed interesting results regarding perceived differences between the genders. although the majority of respondents feel that both female and male elementary teachers are sufficiently nurturing and sensitive to their students, there was a large difference between the numbers of strongly agree and agree responses for male teachers. fifty percent, or 107 participants, responded that they strongly agree that female teachers are nurturing and sensitive with their students, whereas only 15%, or 32 participants, selected strongly agree in response to male elementary teachers. furthermore, 23%, or 49 participants, selected neither agree nor disagree or disagree in response to whether male teachers are nurturing and sensitive to their students, whereas only 6%, or 12 participants, selected those same responses for female teachers. a comment made regarding female teachers’ nurturing and sensitivity was that “the elementary setting brings this out in teachers (hopefully!).” this comment was absent from the male nurturing and sensitivity survey item, although three participants’ comments stand out. “i think women are naturally more nurturing than men…” “i have observed both nurturing and ‘professional’ styles in male teachers.” “some male teachers are not as sensitive as female teachers; however the[y] still are concerned about their students.” thus, the perception of some elementary teachers is that male teachers are not as nurturing or sensitive as their female colleagues, which could support carol gilligan’s work (1982) describing the unique nurturing and sensitivity possessed by females. perhaps the schools surveyed are not hiring male teachers who have these characteristics. it also could be that the respondents made these comments because they perceive, much like the societal stereotype, that males do not inherently possess the abilities to nurture and be sensitive to gender-based differences among elementary school teachers / t. d. wood 327 their students. this aligns with cooney and bittner’s research (2001) that this perception can be a barrier to hiring more male elementary teachers. in response to survey items regarding teachers’ effectiveness in the area of patience with their students, most participants selected strongly agree or agree for both female and male teachers, yet there were differences in the neither agree nor disagree and disagree responses. whereas 9%, or 19 survey participants, responded with these two choices for female teachers, 18%, or 40 participants, responded with these two choices for male elementary teachers. while not statistically significant, more teachers responded negatively toward male elementary teachers. data collected from these two survey items continue to show that in almost all cases, respondents state that it depends on the individual teacher. a closer look at the comments, however, reveals a slightly different perspective made by one respondent about male elementary teachers. this individual stated, “[males] can be less patient than female teachers.” even though it is only one comment, there were not any statements made in response to the survey item addressing patience among female teachers. data from this research reveal that more respondents perceive males as not having enough patience to be effective in the classroom; thus, it could be a challenge for males to be effective classroom teachers, which may explain why there are fewer male candidates entering the field of elementary education. data from the two survey items addressing teacher fairness are very similar across genders. means and standard deviations were nearly identical regarding female and male teachers treating their students fairly. although the range was greater regarding male teachers, the results were not statistically significant. gender was not perceived as playing a role in fairness according to this study. several interesting comments were made addressing the topic of fairness. for example, “what is fair? that used to be a ‘trick’ question on our district’s phone interviews.” another respondent said, “the ‘correct’ answer was giving children what they need at the time. it may look different depending on the child and the particular circumstances.” and yet another stated, “there are very likely teachers who consider themselves to be fair because they treat all kids equally—it’s one of those things upon which we must really reflect continually.” regarding teachers having a general desire to work with children, 95%, or 204 participants, felt that female teachers have a desire to work with children and 92%, or 199 participants, felt that male teachers have a desire to work with children. table 4 shows the number of participants responding with each of the options to teachers’ desire to work with children. table 4. desire to work with children the female teachers i have worked with have a desire to work with children. the male teachers i have worked with have a desire to work with children. (n) (%) (n) (%) strongly agree 124 57.94 100 46.08 agree 80 37.98 99 45.62 neither agree nor disagree 10 4.67 14 6.45 disagree 0 0.00 3 1.38 strongly disagree 0 0.00 1 0.46 one respondent commented that in her generation, it was expected that she become a nurse, mother, or teacher. another said, “i don’t see the same passion with the males in my building.” other comments referred to the trend of males entering secondary education international electronic journal of elementary education, 2012, 4(2), 317-345. 328 because of the opportunities for involvement in clubs and sports and indicated that many of the males in the elementary setting were headed toward administration. teacher to teacher. the third domain focuses on the perceptions that teachers have of their colleagues’ ability to develop effective collegial relationships and have positive attitudes and their expected jobs regarding gender in the elementary education setting. regarding teachers being collegial with each other, these survey items elicited a higher percentage of negative responses regarding male teachers (7.44%) than female teachers (2.33%). respondents raised several intriguing points. one comment was made that female teachers are collegial “most of the time, but there is a lot of nit-picking that goes with working with mostly women.” another participant commented that “i find female teachers are either very collegial, or jealous and do not want to share.” that comment was supported by another respondent who described two different working environments with females, one that was highly positive and productive, and one that was not. participants stated that “male teachers tend to share more than female teachers” and that they “do not let a lot of things get to them.” this survey item pertaining to female teachers’ collegiality was the first in the survey to engender negative qualitative responses. regarding teachers having positive attitudes about their profession, overall, more participants responded negatively to the idea that male elementary teachers (4.47%) have positive attitudes than to that for female teachers (0.93%). table 5 categorizes response frequencies and percentages for female and male teachers’ positive attitudes. table 5. positive attitudes the female teachers i have worked with have generally positive attitudes about their profession. the male teachers i have worked with have generally positive attitudes about their profession. (n) (%) (n) (%) positive (strongly agree and agree) 194 90.66 179 82.87 neutral 18 8.41 28 12.96 negative (disagree and strongly disagree) 2 0.93 9 4.47 again, there were several comments made pertaining to teacher attitude. for example, regarding female teachers’ attitudes, one respondent said, “at times it can become difficult to move forward and let things go.” furthermore, three participants’ statements regarding male elementary teachers support this. one said, “males don’t take everything so seriously and tend to keep the women from getting ‘bitchy.’” another respondent commented, “…they are more positive because it (the profession) was their choice. women often hit a glass ceiling.” yet another participant stated, “they seem to not let things bother them as much.” several trends emerged regarding expected jobs for both women and men within the elementary setting. fifty-six survey participants stated that all of the jobs described in the item were expected jobs for women at the elementary setting. one comment summarizes most responses, “i think all positions in the elementary building are open for both male and female teachers. i do not think that there is an expected job for a certain sex anymore in the elementary building.” the second most common response—with 44 responses—was that the office staff was expected to be composed of women. one respondent stated that “office staff is the only job that would be seen as a female job.” with 33 responses, the third most gender-based differences among elementary school teachers / t. d. wood 329 common choice was classroom teachers. the vast majority of respondents made comments such as “all,” “any of them,” and “all of the ones you mentioned” to expected jobs for men in the elementary setting. the second most common response for expected jobs for men was administration, which was listed 40 times. several participants stated that the only administrator they had ever worked with had been a man. others, however, suggested that more women were becoming administrators every year. pe teacher was the third most expected job for men (indicated by 37 of those surveyed.) one respondent even stated, “my current principal actually voiced a desire to hire a male pe teacher, hmmm.” custodians and classroom teachers tied as the fourth most common expected job for men, with 22 responses. four of the 22 respondents identifying that classroom teacher was an expected job for a man specifically stated that “they would be upper elementary” and “you rarely see a [male] kindergarten or first or second grade teacher.” the fifth most common response, with 14 selections, was that all of the jobs were expected to be filled by men, except for that of office staff. this comment occurred so frequently that the researcher was compelled to create a separate category titled all except office staff. several teachers comments that they had never worked with a male secretary. three respondents commented that although they had never worked with a male office staff member, they would be more than willing to do so without bias. the researcher also asked the participants what, if any, differences existed between female and male elementary teachers. of the 217 initial participants, 164 responded to this item. sixty-nine participants commented that either there were no differences or that there were too few male elementary teachers to discuss such differences. some of these participants made comments such as “there are none that i recognize,” “i think gender does not make a difference, but personality does,” “there are definitely differences in teachers but i do not feel it is due to their gender,” and “in elementary school there is no difference.” the vast majority of the participants stated that they had worked with many more female teachers than males, making it difficult to generalize what they know about male teachers. one comment sums up this perception nicely, “i really don’t think of this [teacher differences] along gender lines. my experience with male elementary teachers is somewhat limited because i haven’t worked with many.” the remaining 95 survey participants who responded to this item perceived differences between female and male elementary teachers. after coding the data, the researcher categorized the explanation of differences into four groupings: males–just because, the mellow male, the female who mothers, and commander male. out of the five categories, 30 comments fit into the female who mothers category. noted author and researcher carol gilligan (1982) describes women as having the unique ability to nurture and demonstrate caring to their students. several comments made in this research support gilligan’s work. one comment was, “female teachers are more nurturing than males.” another respondent stated, “female teachers are more sympathetic to students than male teachers.” yet another participant commented, “female teachers are more giving of their personal time and show more empathy to their students.” all comments that the researcher categorized in the female who mothers group were not equally positive. multiple comments were made describing the possible negative aspects. one comment was, “female teachers are more motherly, smotherly towards their students.” another comment was made that “female teachers seem to have difficulty with letting things go when there is a difference in opinion.” a third respondent stated that “female teachers are far more emotional and clicky (sic) than their male counterparts.” the mellow male had the next highest frequency, with 24 responses. these respondents commented that they perceive male teachers to be laid-back, relaxed, independent, and international electronic journal of elementary education, 2012, 4(2), 317-345. 330 have a better sense of humor than their female colleagues. some of the specific comments made were “males seem more laid back,” “the ‘cool guy’ mentality—that is an innate difference in general between women and men,” “they tend to go with the flow,” and “men seem to be more positive than women.” several comments show that this perception of males being laid back can also be negative. one such comment was, “other males are much more laid back and don’t mind letting the kids goof off.” a female teacher stated, “male teachers tend to be more lenient when it comes to management.” another female teacher’s comment was, “male teachers seem to be a little less organized.” again, because of the low number of male teachers it is difficult to generalize perceptions of the male teacher population. however, these responses raise several interesting issues about male teachers. do they seem more independent because there are fewer male colleagues with whom they can be comrades? why are male teachers perceived to be more laid back? since they know they are in the minority could it be that they do not want to “rock the boat?” perhaps it is because they are intimidated by their female colleagues and, therefore, cautious about voicing their opinions. twelve respondents viewed their male teaching colleagues as “lazy.” these respondents said that the male colleagues they knew put in far less work, left school early, and were less passionate about their jobs. two respondents shared negative comments that “many of the male teachers expect the female teachers to do the planning and organizational work,” and “in my building the males are followers of their female teammates. they do very little initiating and are told what to do and sometimes how to do it.” it could be that coupled with the female teaching trait of being motherly, male teachers’ attempts to fit in and be a part of the team are perceived as laziness and apathy. twenty-two respondents said that males achieve a certain level of success simply because of their gender. this researcher labeled this category as males–just because, since several respondents commented on male teachers’ “maleness.” one female respondent stated, “male teachers tend to get more respect from students simply because they are men…” another female participant commented that “males tended to be more liked by a larger majority of students, especially other male students.” respondents commented that male teachers are counted on to discipline students, even when the students are not their own. several respondents also stated that men in their building garnered more respect because of their gender. it could be that this perception is true, that men are respected more because of their gender. it could also be that because they are the minority, they do specific tasks in the building that female teachers are not willing to do or are not asked to do. eighteen respondents commented on men being loud, outgoing, verbally direct, and businessoriented; thus, the researcher organized these comments into the commander male category. one comment was, “male teachers tend to be very structured and organized, while female teachers can be disorganized.” another female respondent stated, “i find that male teachers tend to have firm boundaries without bending in special situations.” yet another female participant said, “male teachers are better disciplinarians with challenging students.” most of the comments were positive in nature, yet a few respondents perceived these qualities to be negative. two female survey participants stated: “some male teachers are less accommodating to individual needs and learning styles,” and “in general, my experience is that some male teachers are extremely strict and yell more often than their female counterparts.” again, these results could indicate that because of the low number of male teachers, it only takes a few ineffective ones to have a major impact on the stereotype of male teachers. the researcher also surveyed participants regarding the need for more male elementary teachers. one hundred and eighty-four respondents perceived a need for more male elementary teachers; their responses were grouped by the researcher as follows: male role models, balanced staff, opportunity for varying styles, and the male presence. seventy-six gender-based differences among elementary school teachers / t. d. wood 331 percent, or 138 participants, who perceived a need for more male elementary teachers stated that more males were needed as role models. a large number of participants said that students, both boys and girls, needed male role models in elementary school because so many students’ homes did not have a positive male influence. forty-seven percent, or 87 respondents, stated that students needed a “strong” and “good” male role model. nineteen percent (35) of the respondents who felt that more men were needed in elementary education believed that men provided a “balance” to the staff and school environment. multiple comments addressed the fact that since the greater population is made up of almost half women and half men, our schools should represent those figures accurately. missing from this reasoning was a need to hire a more diverse staff, including teachers of varying races and ethnicities. eight percent, or 15 of the respondents, stated that more male teachers were needed because they offer a different approach to teaching; two of the comments are as follows: “to add another point of view and demonstrate that logic can be as effective as well as emotions.” “[more men are needed] to offer diverse perspectives and styles in school that would benefit everyone.” two percent, or four of the respondents, stated that students enjoy and respond well to male teachers, and therefore schools need more of them. respondents shared that male teachers are able to connect with students in a unique way and “get something different from them.” less than one percent (.05%), or nine of the respondents, commented that more male teachers are not needed in elementary education. these respondents explained that gender is not an issue and that no one should get a job because they are male, female, or of a certain race. two of these nine respondents stated that none were needed at the elementary level, but more were needed at the middle and secondary levels of education. most research highlights the low number of male elementary teachers, yet these particular respondents perceived that there was no need for more male elementary teachers. however, the majority of survey participants felt that elementary education needs more male teachers. finally, the researcher asked for additional comments. twenty-one percent, or 45 of the initial 217 participants, responded to this survey item. twelve respondents commented on the importance of hiring more male teachers of high quality and stated that it was an important issue. several respondents stated that this study was a “good idea” and that this study was “important for education.” one female respondent stated “i really wish there could be some kind of push to get more male teachers in elementary school. [i] would like to see society communicate support for men entering the profession with importance and not just a role for women.” this comment was supported by another female participant, who commented “i think that men would make excellent elementary teachers, but are often steered into higher grade levels.” there were only two negative comments about the research topic itself. one female respondent stated her frustration this way: it really irks me that they [males] are many times singled out for glamorization— movies made, books written, teacher of the year, etc. i think a lot of the attention they get is that they start out as a novelty/minority in the first place, so they are easier to focus on. i think that many female teachers get looked over because they are not a novelty, but an expectation. fifteen respondents commented on the challenge of discussing possible differences between female and male elementary teachers, because of a myriad of factors including the low number itself. many comments included the fact that the respondents had never worked with a male teacher before, thus making it difficult for them to respond to the survey. one individual communicated that it was not a concern, because teachers should not be hired because of their gender, among other reasons. another participant shared the opinion that international electronic journal of elementary education, 2012, 4(2), 317-345. 332 teacher gender has more effect on the grade level team dynamics than it has on the students. two other comments were made regarding team dynamics. one comment was that “male elementary teachers tend to spend time more productively by staying focused on tasks to be accomplished during meetings and making decisions more quickly without discussing non-essential information unrelated to the decision.” a second comment was simply put, in regard to staff personnel, “male teachers are allowed to be more laid back.” these comments support previous statements to other survey items relating to the emotional or attitudinal differences between female and male teachers. conclusions a comprehensive review of the data and an analysis and synthesis of findings suggest the following conclusions regarding teachers’ perceptions of gender-based differences among their elementary school colleagues: 1. male elementary teachers are, overall, perceived more negatively than their female colleagues. the reason that men received more negative responses than female teachers is unclear; it could be that men who work in elementary education are less effective than their female colleagues or that women working within the profession simply do not perceive their male colleagues as effective because of their own biases. further, with the low number of male elementary teachers, it could be that a few ineffective men significantly impact negative perceptions related to their effectiveness, whereas similar numbers of ineffective female teachers are overshadowed by the majority of their colleagues who may be highly effective. 2. the majority of participants feel that there are differences between female and male teachers, particularly when it comes to their ability to serve as role models and the jobs they are expected to fill in the elementary setting. more than half of the teachers surveyed felt that female and male elementary teachers were different in important ways. respondents felt that women are more nurturing and sensitive to their students’ needs than are their male colleagues, despite the fact that some of these character traits might be perceived negatively (such as being too nurturing or too sensitive). men were also perceived positively as being more laid back but also, in interesting contrast, as being more strict than their female colleagues. other respondents indicated that men were more successful in the elementary setting simply because they were men. many of these perceptions may have their roots in stereotypes about men and women. they could also be based on differences in the socialization of men and women, and subsequent variations in their classroom functioning. perceived differences between female and male teachers are especially apparent when it comes to their ability to serve as role models and the jobs they are expected to fill in the elementary setting. although seven female participants disagreed that male elementary teachers were effective role models for their students, this was the only pair of items on the online survey where women received more neutral responses than men. one reason for this pattern may be that female participants did not feel comfortable stating their perception because of perceived or self-imposed pressure to support female colleagues. teachers who were surveyed expected to see more men than women in administrative positions, particularly in those that were high-ranking. they were aware that their male colleagues were often encouraged to seek administrative positions and stated that, at least in one district, there was no attempt to hire female administrators. men may often enter administration because of the low number of male teachers and the desire for camaraderie. gender-based differences among elementary school teachers / t. d. wood 333 3. a vast majority of participants feel that more male elementary teachers are needed in the school setting. nearly every respondent indicated that more males were needed in elementary education. this finding supports existing literature that suggests that the main reason for needing more male elementary teachers is so that they can serve as role models for students, especially for those students who lack male figures at home or outside school. survey responses were interesting, considering that teachers also perceived males more negatively than their female colleagues, both generally, and more specifically, with regard to their functioning as role models for their students. finally, teachers in this study felt that male teachers “balanced” the staff, exposed their students to a variety of teaching styles, and offered a beneficial presence in the building. 4. differences between female and male elementary teachers are unclear in many areas. finally, perceived differences between female and male elementary teachers with regard to the characteristics addressed in this study remain unclear. statistically significant differences between perceptions of female and male teachers were not found; however, there were qualitative differences, with males receiving more negative comments than their female colleagues. differences could not be determined in each of the three domains, even though most participants responded to the open-ended item that such differences did, indeed, exist. . . . acknowledgments my advisor and chair, dr. rina chittooran, for her support and guidance throughout the dissertation project. tracy darrin wood earned a ba in elementary education from missouri southern state university(1999), a ma in elementary education from university of missouri–saint louis (2002), and a phd in elementary education–curriculum and instruction from st. louis university. he has taught second, third, fourth, and fifth grades. tracy works as an adjunct instructor, teaching undergraduate and graduate courses at missouri baptist and lindenwood universities. his current research interests are in the areas of teacher gender, teacher differences, and teacher–student relationships. references alghazo, e. m. (2005). special education teacher perceptions towards effective instructional practices in the united arab emirates (uae). teacher education and special education, 28(3-4), 221-229. carrington, b., tymms, p., & merrell, c. (2008). role models, school improvement and the ‘gender gap’—do menbring out the best in boys and women the best in girls? british educational research journal, 34(3), 315-327. chudgar, a., & sankar, v. (2008). the relationship between teacher gender and student achievement: evidence from five indian states. a journal of comparative education, 38(5), 627-642. colorado state. (2007). writing guides. retrieved march 2, 2009, from http://writing.colostate.edu/guides/research/glossary/ cooney, m. h., & bittner, m. t. (2001). men in early childhood education: their emergent issues. early childhood education journal, 29(2), 77-82. international electronic journal of elementary education, 2012, 4(2), 317-345. 334 cresswell, j.w. (2003). educational research: planning, conducting, and evaluation quantitative and qualitative research (2nd ed.). upper saddle river, nj: pearson education. cushman, p. (2005). it’s just not a real bloke’s job: male teachers in the primary school. asia-pacific journal of teacher education, 33(3), 321-338. czaja, r., & blair, j. (2005). designing surveys: a guide to decisions and procedures (2nd ed.). thousand oaks, ca: pine forge press. data about men teachers (2007). menteach organization. retrived, october 23, 2011 from http://www.menteach.org/resources/data_about_men_teachers. dee, t. s. (2006). how a teacher’s gender affects boys and girls. education next, 6(4), 68-75. dillman, d. a. (2007). mail and internet surveys: the tailored design method. hoboken,nj: john wiley & sons. driessen, g. (2007). the feminization of primary education: effects of teachers’ sex on pupil achievement, attitudes, and behavior. review of education, 53(2), 183-203. gilligan, c. (1982). in a different voice: psychological theory and women’s development. cambridge, ma: harvard university press. glesne, c. (2006). becoming qualitative researchers: an introduction (3rd ed.). new york: longman. green, s.p., shriberg, d., & farber, s. (2008). what’s gender got to do with it? teacahers’ perceptions of situation severity and requests for assistance. journal for educational and psychological consultation, 18(4), 346-373. huang, s. l., & fraser, b. j. (2009). science teachers perceptions of the schoolenvironment: gender differences. journal of research in science teaching, 46(4), 404-420. internet access in u.s. public schools and classrooms (2007). national center for education statistics. retrieved, september 14, 2009 from http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2007020. johnson, b., & christenson, l. (2008). educational research. thousand oaks, ca: sage publications. jones, d. (2003). the right kind of man: the ambiguities of regendering the early years school environment—the case of england and wales. early child development, 173(6), 565-575. lincoln, y. s., & guba, e. g. (1985). naturalistic inquiry. newbury park, ca: sage. martin, a., & marsh, h. (2005). motivating boys and motivating girls: does teacher gender really make a difference? australian journal of education, 49(3), 320-334. martin, a.j., marsh, h.w., & cheng, j.h.s. (2008). a multilevel perspective on gender in classroom motivation and climate: potential benefits of male teachers for boys? journal of educational psychology, 100(1), 78-95. martino, w., & kehler, m. (2006). male teachers and the boy problem: an issue of recuperative masculinity politics. mcgill journal of education, 41(2), 113-131. mullola, s., ravaja, n., lipsanen, j., alatupa, s., hintsanen, m., jokela, m., & keltikangas-jarvinen, l. (2011). associations of student temperament and educational competence with academic achievement: the role of teacher age and teacher and student gender. teaching & teaching education, 27(5), 942-951. national education association. (2001, august). status of the american public school teacher. retrieved february 8, 2009, from http://www.nea.org/home/2233.htm. patton, m. q. (2003). qualitative evaluation checklist. western michigan university. retrieved march 2, 2009, from http://www.wmich.edu/evalctr/checklists/qec/. piper, p. s., & collamer, b. e. (2001). male librarians: men in a feminized profession. the journal of academic librarianship, 27(5), 406-411. gender-based differences among elementary school teachers / t. d. wood 335 rice, c. j. & goessling, d. p. (2005). recruiting and retaining male special education teachers. remedial and special education, 26(6), 347-356. sheehan, k. b. (2001, january). email response rates: a review. journal of computer-mediated communication. retrieved may 8, 2009, from http://jcmc.indiana.edu/vol6/issue2/sheehan/html smith, j. (2004). male primary teachers: disadvantaged or advantaged? paper presented to the australian association for research in education conference. melbourne, australia. sue, v. m., & ritter, l. a. (2007). conducting online surveys. los angeles, ca: sage publications. tatar, m., & emmanuel, g. (2001). teachers’ perceptions of their students’ gender roles. the journal of educational research, 94(4), 215-224. wiest, l. r. (2003). the current status of male teachers of young children. the educational forum, 68(1), 62-70. williams-johnson, m., cross, d., hong, j., aultman, l., osbon, j., & shutz, p. (2008). “there are no emotion in math”: how teachers approach emotions in the classroom. teachers college record, 110(8), 1574-1610. yazuz, m. (2009). an investigation of burn-out levels of teachers working in elementary and secondary educational institutions and their attitudes to classroom management. educational research and reviews, 4(12), 642-649. international electronic journal of elementary education, 2012, 4(2), 317-345. 336 appendix a teacher online survey gender-based differences among elementary school teachers / t. d. wood 337 international electronic journal of elementary education, 2012, 4(2), 317-345. 338 gender-based differences among elementary school teachers / t. d. wood 339 international electronic journal of elementary education, 2012, 4(2), 317-345. 340 gender-based differences among elementary school teachers / t. d. wood 341 international electronic journal of elementary education, 2012, 4(2), 317-345. 342 gender-based differences among elementary school teachers / t. d. wood 343 international electronic journal of elementary education, 2012, 4(2), 317-345. 344 gender-based differences among elementary school teachers / t. d. wood 345 international electronic journal of elementary education, september 2016, 9(1), 1-20. issn:1307-9298 copyright © iejee www.iejee.com living transdisciplinary curriculum: teachers’ experiences with the international baccalaureate’s primary years programme michael j. savage a * susan m. drake a a brock university, canada received: 15 august 2016 / revised: 27 september 2016 / accepted: 29 september 2016 abstract an integrated curriculum that is transdisciplinary in nature seems to be a good fit for 21st century learning. there are, however, few examples of transdisciplinary curriculum at the k to 12 level. one exception is the international baccalaureate’s primary years programme (pyp) which features transdisciplinary curriculum for students from ages 3 to 12 around the world. this phenomenological study explored the lived experience of 24 pyp educators to deepen understanding of what such a curriculum looks like in practice. three main themes were identified. the first, “it’s a framework” outlines participants’ understandings of transdisciplinary teaching and learning and the freedom a transdisciplinary framework can bring. the second theme, “get on board”, examines participants’ thoughts around what is required to successfully implement a transdisciplinary curriculum. the final theme, “their learning journey”, discusses participants’ beliefs around the success of a transdisciplinary curriculum. in general, participants appreciated the transdisciplinarity of the program. concerns revolved around implementation issues. suggestions on how to implement transdisciplinary teaching and learning in other contexts are provided. keywords: transdisciplinary, integrated curriculum, interdisciplinary curriculum, international baccalaureate (ib), elementary. introduction educational reformers in the 21st century note that integrated or transdisciplinary curriculum approaches address the needs of the 21st century learner to learn the skills and worldview necessary to negotiate a complex global world (hargreaves & fullan, 2012; hargreaves & shirley, 2009). a transdisciplinary curriculum is an iteration of an integrated curriculum (drake, 1993). a discussion of transdisciplinarity and its merits can be heard at the university level (see, klein, 2014), however, there is little application at the other end of the educational spectrum (richards & shea, 2006). a notable exception to this is the international baccalaureate (ib) primary years program (pyp) for students aged 3 * corresponding author: michael j. savage, welch hall, brock university, 1812 sir isaac brock way, st. catharines, ontario, canada, l2s 3a1 msavage@brocku.ca 1-905-688-5550 ext. 6183 http://www.iejee.com/ international electronic journal of elementaryeducationvol.9, issue1, 1-20, september2016 2 to 12 years. ib took the concept of transdisciplinary teaching and learning and based its entire pyp on the concept. it is very important to understand what we are doing in terms of teaching and learning and why we are doing it. it is equally important, however, that we understand how we are doing it (pop & maties, 2008). the purpose of this study is to explore the lived experiences of teachers, administrators and coordinators who teach in these transdisciplinary ib programs. since the ib pyp appears to be a good model of 21st century learning, a deeper understanding of implementing transdisciplinary curriculum as a lived experience may facilitate other educators with similar goals to move toward implementation in their own setting. defining transdisciplinary the term “transdisciplinary” first emerged during the 1970s. it was specifically referenced in an oecd conference held in france that was focused on problems experienced by instructors and researchers in postsecondary settings (cantar & brumar, 2011). it was defined as a, “comprehensive framework that tried to go beyond combining existing disciplinary approaches in an interdisciplinary fashion to create new frameworks, new overarching syntheses” (cantar & brumar, 2011, p. 637). wiesmann et al. (2008) argue that transdisciplinary teaching, learning and research has emerged as a response to the complex problems that exist in 21st century society. it has been found that disciplinary approaches to these complex problems are insufficient and that only by looking beyond the disciplines can the problems be understood and solutions determined. nicolescu (1998, 2008) posits that transdisciplinary teaching and learning is not just an intellectual activity but rather must involve the entire person: mind, body and emotions. he states, “a viable education can only be an integral education of the human being.” (p. 3). in the k to 12 context, transdisciplinary is often seen as at the far end of a continuum with increasing degrees of integration; it is an approach that transcends disciplinary boundaries (drake, 2012). the ib pyp context there are over 4500 ib schools around the world situated in 3 of ib’s geographic regions (the americas; europe, africa and the middle east; asia and australia) and they are public, private and international. the pyp program is noted for its rigour and relevance. studies show that students achieve high levels academically. for example, a study conducted in australia found that the science proficiency level of year 6 students enrolled in pyp schools was significantly higher than the national level (campbell, chittleborough, jobling, tytler, and doig, 2014). another study, conducted in australia and singapore, which focused on the early years pyp programme demonstrated that students’ literacy skills were well-developed, their school readiness was similar or better than children enrolled in a traditional early years program, and they were developing essential learning skills faster than a comparative sample (morrissey, rouse, doig, chao & moss, 2014). standardized test analysis conducted in new zealand indicated that academic achievement within pyp schools generally exceeded the academic achievement of schools with similar school populations (kushner, cochise, courtney, sinnema & brown, 2016). similarly, gough, sharpley, vander pal and griffiths (2014) found the scores of students enrolled in 13 pyp schools in victoria, australia on a standardized national assessment of reading and numeracy were significantly higher than the australian average in all cases, with just one exception (year 5 numeracy scores). importantly, students also do well beyond the academic. for example, administrators at 13 pyp schools believed their students exhibited greater motivation and positive learner living transdisciplinary curriculum / savage & drake 3 attributes than the australian average (gough et al., 2014). kushner et al. (2016) found students in the pyp schools demonstrated greater international-mindedness, inquiry skills and action than students in comparator schools. finally, demonstrating the relevance of the pyp, a study conducted in colombia found that 89.3% of students enjoyed being a student at their pyp school and 90% reported they were proud of their school (lester & lochmiller, 2015). these are consistent with a small, but growing, research literature that demonstrates transdisciplinary approaches to teaching and learning lead to academic outcomes that are as good as or better than traditional disciplinary approaches (for examples see guyette, sochaka, & costantino, 2015; la porte, 2016; sillisano et al., 2010; tan & bibby, 2010). there is also evidence from a recent meta-analysis of 213 transdisciplinary socioemotional learning (sel) programs (durlak et al., 2011) that transdisciplinary approaches to teaching and learning increase student motivation to learn, empathy, impulse control skills, self-management techniques and problem-solving skills – all important components of learning that are not always reflected in academic achievement scores. the pyp is grounded in a strong transdisciplinary framework based on the work of ernest boyer (1995) and others. the framework is holistic in nature with an emphasis on the whole child and authentic and significant inquiry (international baccalaureate organization [ibo], 2009). the importance of the disciplines is acknowledged, but disciplines alone are not enough. students need to learn the concepts and skills that transcend the disciplines and fall into the transdisciplinary realm. ib begins with the transdisciplinary themes (who we are, where we are in place and time, how we express ourselves, how the world works, how we organize ourselves and sharing the planet) which are organized into units of inquiry. each inquiry unit includes central ideas (enduring understandings), key concepts (e.g. form, function, causation, change, connection, perspective, responsibility and reflection), related concepts (e.g. property, structure, patterns, sequences, systems, etc.) and lines of inquiry that correspond to the central idea (ibo, 2012a) additionally, students acquire and apply transdisciplinary skills including thinking skills (e.g. higher order thinking skills, metacognition, and dialectical thought), social skills (e.g. respect, collaboration, conflict resolution, etc.), communication skills (reading, writing, listening, speaking, etc.), self-management skills (e.g. organization, safety, time management, healthy lifestyle, etc.) and research skills (e.g. questioning, observing, planning, collecting data, etc.) the ib pyp program, however, does not just focus on concepts and skills it also includes attitudes, values and behaviours. students need to demonstrate the attributes in the ib learner profile. these attributes are: inquirers, knowledgeable, thinkers, communicators, principled, open-minded, caring, risk takers, balanced and reflective. through this process students learn to be internationally minded and to be able to collaboratively solve the complex problems in the interdependent world they live in (ibo, 2009, 2012a, 2012b) the pyp consists of administrators who lead their schools and pyp coordinators who focus on assisting classroom and specialist teachers in the transdisciplinary process. within the pyp, teachers are identified as classroom or specialist teachers. classroom teachers teach the core subjects and are the main designers and implementers of the transdisciplinary units. specialist teachers are disciplinary specialists who typically focus on areas such as physical education, the arts and library. the pyp includes mandated collaborative planning time using a specified pyp planner that utilizes backwards design (wiggins & mctighe, 2006). the collaborative planning typically involves the classroom teachers and pyp coordinators. in some schools specialist teachers international electronic journal of elementaryeducationvol.9, issue1, 1-20, september2016 4 also collaboratively plan and in other schools they do not. the pyp planner encourages designers to choose their transdisciplinary theme, central idea and summative assessment task first. second they choose their key concepts, transdisciplinary skills, ib learner profile attributes and teaching strategies. summative assessment tasks are typically rich performance assessment tasks, not just traditional paper and pencil tests or essays. research questions the overarching research question for this study was: how do primary educators in ib schools translate transdisciplinary theory into practice as explored through the pyp teachers’, coordinators’ and administrators’ lived experience. additional questions included: 1) how did the participants in the study understand, define, plan for and approach transdisciplinarity? 2) what barriers did participants identify which hindered their ability to deliver a transdisciplinary program? 3) what supports were provided to help participants deliver a transdisciplinary program? method the study utilized an empirical phenomenological research design (creswell, 2012; moustakas, 1994). creswell (2012) states that the phenomenological approach is best suited for problems in which it is important to understand a collective groups’ common or shared experience of a phenomenon. in this case it was important to understand the common experiences of pyp teachers with transdisciplinary teaching in order to develop or refine practices and policies and to develop a deeper understanding about the features of the phenomenon under investigation. after securing ethics clearance, 24 ib pyp participants (teachers, coordinators or administrators) were interviewed from all 3 of ib’s geographic regions and represented public, private and international school contexts. a snowball recruiting method was utilized. every participant who agreed to participate in the study and completed a videoconference or telephone interview was asked to inform any of their friends and colleagues who currently taught in the pyp who they thought may be interested in participating in the study to directly contact the researchers for more information. participants were evenly distributed between ib’s three geographic regions with 8 participants (33.3 % of the total sample) coming from each. consistent with ib’s demographics 13 participants worked at international schools (54.2%), 7 worked at private schools (29.2%) and 4 worked at public schools (16.6%). the majority of the participants were female (83.3%) and the mean number of years participants had been teaching was 15.4 years with the mean number of years participants had worked at ib schools being 8.0 years. in terms of their roles, 12 participants (50%) were classroom teachers, 6 participants (25%) were specialist teachers, 4 participants (16.7%) were coordinators and 2 participants (8.3%) were administrators. data collection consisted of in-depth interviews with participants through videoconferencing software such as skype or over the telephone. as moustakas (1994) recommends we used a broad, general umbrella question that was further broken down into three more directed questions and several additional probes. the interviewer addressed the probes only if the participant did not offer any information in that area. living transdisciplinary curriculum / savage & drake 5 interviews with participants were conducted by one, or both, of the primary researchers and lasted approximately one hour in length. the interview was recorded on multiple audio recorders and then transcribed by a trained transcriber. participants were sent the transcript of their interview so they could provide clarification around what they stated. they were also encouraged to correct any errors they believed were in the transcript with regard to what they reported in the interviews. participants then sent their checked transcripts back to the researchers. data analysis data were analyzed using the empirical phenomenology approach (moustakas, 1994). in this approach the researcher attempts to suspend his or her own thoughts and beliefs to uncover the lived experiences of their participants. the goal is to discover any common themes that emerge across participants’ lived experiences and report these themes, to the greatest extent possible, in the participants’ own words. in order to accomplish this, the researchers read over each transcript several times. significant statements that were relevant to the phenomenon were identified. these identified statements are called the meaning units and could be as short as a sentence or as long as a page of the transcript. the meaning units were then thematically organized in a situational structured description for each participant. the themes in each situational structured description were called the categories. these categories were then analyzed for common themes across participants. the reported findings below emphasize the commonality that is present in the diverse experiences of the participants interviewed in terms of their lived experiences with transdisciplinary teaching and learning. findings and discussion three major themes emerged across the lived experiences of the 24 participants. they included: “it’s a framework!”, “get on board” and “their learning journey.” each theme is detailed below. “it’s a framework!” all the participants identified the pyp as a unifying framework that was designed to promote transdisciplinary teaching and learning. three main categories emerged that related to this theme and included: freedom, personal understanding, and concept confusion freedom. one participant explained, “what is great about the pyp … you have this framework that you can customize however it will fit your schools needs the best.” another participant noted, “i have loved working with it. of course it is not perfect but nothing is right now but i think this idea that it is a framework and finding these interpretations in it is interesting i think and everybody is making sense of it which is something that i enjoy.” a third stated, “pyp is a framework and then you have to find your own way into it, your own interpretations almost. of course you have to meet the standards but how you document children’s learning over time is up to you as long as you do it.” a fourth participant added, “we need to understand that the ib has schools all over the world and i personally appreciate the flexibility that it has.” finally, a fifth participant articulated, “it gives me the freedom as an educator to do lots of [things] with kids.” other participants noted specific ways that the pyp framework allowed their schools to integrate in written curriculum standards from various jurisdictions and detailed the curriculum mapping exercises they had engaged in to do that. one participant commented that solidifying “how to make the [pyp] and standards all work cohesively together” is something schools need to strive for. international electronic journal of elementaryeducationvol.9, issue1, 1-20, september2016 6 not all participants, however, thought the freedom the pyp framework provided was a good thing. one participant noted, “the scope and sequence i have to say is very, very vague, very wishy-washy.” another participant went further, “anyone can choose the learning goals that they wish for their report card which to me does not make sense … i think it needs to be set in stone, a lot more standardized … i would love to have the transdisciplinary skills set out in a developmental continuum.” personal understanding. the second category illustrates how participants interpreted the pyp framework and all the components embedded in it. a key component of the framework is teaching and learning. participants articulated their understanding of transdisciplinary teaching and learning in a multitude of ways including: “so when i think about transdisciplinary learning i think what you are trying to do is … create connections between, across and maybe beyond, the disciplines … it’s almost like it transcends the disciplines, it rises above it … [it] is more important than if you were just working in a disciplinary way.” another participant stated: “the way i have learned to frame it … is to stop compartmentalizing disciplines … if you stop putting disciplines up as isolated, standalone disciplines and core subjects, and instead of putting ‘writer’s workshop’ you put ‘large group’, instead of putting ‘reading’ you put “small group”, then you are able to take the unit of inquiry and mesh it with your daily activities.” a third participant described it: “ i really look at it from a multi-tiered approach … i think at the most basic level is where the subjects really are working together very seamlessly as we are exploring a real life situation, real life issue … something very relevant to the real world.” a participant described, “because i am a learner, i like concept learning. it’s a difficult thing. in fact in working with my [students] right now on regions and they want to do what i call ‘factoids’ all the time. i started with a question, ‘what makes your region unique?’, and i kept coming back to that. they want to look at things as ‘factoids’ instead of as a concept … all places have a unique culture but they want to look at tiny bits of information that will probably not serve them well in the future and so in teaching the concepts i try really hard to talk with those and use those in my vocabulary.” the skills portion of the pyp framework ib refers to as the transdisciplinary skills or the ‘approaches to learning’. one participant said, “if you look at the skills, you know, there is communication skills, social skills, there is research skills, thinking skills and selfmanagement skills and then under each header there is a further sort of definition of what these skills imply. so, for example, under thinking skills you find remembering, analyzing, synthesizing, creating, you know, that sort of further refinement of what these skills imply.” a second participant outlined, “they are skills that can be used throughout the subject areas or across subject areas throughout the day. and it’s not that content doesn’t matter but the content is used to teach the skills instead of the skills being used to teach the content.” a third participant noted, “we also have articulation to make sure that we are covering all of those skills and we are trying to get better at teaching them explicitly rather than just saying there are communication skills involved in the unit, we try to teach those skills so that the kids know how to be positive in their communication skills in that unit.” the values and attitudes component of the pyp framework is reflected in the ib learner profile. this component of the framework one participant referred to as “the heart of it all and it is the heart of the whole ib continuum.” the participant went on to articulate, “these are the attributes that we have and they’re personal and they’re academic and so we need to look at them in both of those ways and they interact with everything all the living transdisciplinary curriculum / savage & drake 7 time. at our school we tend to focus on a couple of them with each unit, ones that sort of speak to us and we link those quite strongly to the attitudes as well. now how we assess those we largely do a lot of reflection on them … a big part of the student portfolio relates to the learner profile.” similarly, another participant stated, “the learner profile is that learner outcome that we are aiming for. for me, everything else is a means to that end, to becoming an inquirer. you know, to becoming that internationally minded person that is represented from those, from that collection of attributes.” another participant noted, “they are probably more often sort of integrated into the teaching. so i know a lot of people use story books to sort of model what the attributes might look like and often those books will be integrated into a unit of inquiry so we will connect to whatever the unit is and so then they can look at how the characters in the book … are, you know, internationally minded or open-minded or caring, or whatever.” this participant continued, saying, “most visible are often the learner profile attributes and they are also posted all over the school.” concept confusion. several participants noted that the pyp framework, and especially the concepts embedded in it like the central idea, key concepts, related concepts, transdisciplinary skills and the ib learner profile, was quite complex and confusing. as a result, the final category that emerged in the “it’s a framework!” theme was concept confusion. statements from the participants demonstrated that they often did not understand, or were confused about, the concepts in the pyp framework. for example, one participant, while asked to describe his/her understanding of related concepts stated, “i would say the related concepts that i am looking at, that i am really emphasizing with kids, are the ideas of tenacity, resilience, determination, perseverance, you know the big ideas about anything to achievement, trying to strive for personal excellence.” these seem to not be concepts at all, but rather values that the participant wanted to instill in his or her students. another participant, responding to the same question, replied, “we always have them clearly displayed … i try to use them but to be honest in some ways they are decoration on the wall.” another participant was asked to give his/her understanding of the central idea, key concepts and related concepts and stated, “backwards by design is often used … as a planning tool and i think that’s why you start with that conceptual understanding and then …what skills can children build while they’re developing these understandings.” while using the current terms the participant veered from defining the conceptual understandings to focus on skills. another participant, when asked to provide an example of a skill he/she would teach, stated “we taught children to be risk takers.” this appears to not be a skill but rather an attribute from the ib learner profile. most of the participant’s confusion centred around the central idea, key concepts and related concepts. many participants were not able to provide understandings of these concepts that were congruent with the ib’s conception of the terms. one participant noted, “in my experience working with teachers … the related concepts seem to be easier to access for the students than for the teachers. the key concepts as well … especially for teachers who are new to the pyp, that kind of trips them up a little bit.” from participants’ responses it would appear that even teachers with several years of pyp experience still struggle with the central idea, key concepts and related concepts in the pyp framework. they do not seem to completely understand what a transdisciplinary concept consists of. “get on board” the second major theme was for the pyp to function as a transdisciplinary program people needed to “get on board” as one participant put it. the participants articulated there were multiple components that needed to be in place for true transdisciplinary international electronic journal of elementaryeducationvol.9, issue1, 1-20, september2016 8 teaching and learning to occur within the pyp and these components were the categories that made up this theme. they included: educator’s philosophy and attitude, administration, timetabling, collaborative planning, pyp planner issues, lack of experience with pyp, and professional development educator’s philosophy and attitude. personal philosophy and attitudes were central to the implementation of the pyp. one participant highlighted, “when i came in [the school] had been struggling for a few years because the head of the school was not on board with the pyp and he … hired a curriculum coordinator that wasn’t in tune with [the program] … and so a lot of the curriculum decisions she was making and the people she was hiring [were] working against the pyp and inquiry.” another participant noted, “there are some people on my team who are not so inquiry-based and they just want to get the job done … i guess it’s a difference in philosophy … it’s more about having a constructivist nature, like sort of an inquiry nature, and some can’t let go, they can’t let go of the reins, they don’t like the messiness [transdisciplinary teaching and learning] brings.” another participant noted there was “absolutely” a conflict between people who are more transdisciplinary and people who are discipline-based. the participant went on to state he/she sees “it all over … people talk about it all the time.” other participants provided examples of how a change in philosophy helped foster more transdisciplinary teaching and learning. one participant stated, “recently there have been a few really good hires, and there has been a new head of school and … pyp coordinator … a lot of people rethink their practice and start teaching [in a more transdisciplinary way] … teachers teaching a little bit less prescriptive.” another participant said, “there is now a drive to hire teachers who aren’t just good teachers but who also embrace the pyp. there are some transitions being made right now with the teaching staff and some teachers who have chosen to move on.” adding to this, a participant stated, “i am moving [to another school] next year … they need someone to come in and reassure them and work with them to be focusing on this inquiry-based learning and concept-based learning, which is really what the transdisciplinary framework is all about.”. one participant summed it up this way, “the only thing that hinders transdisciplinary [teaching and learning] are people’s attitudes.” administration. the second category that emerged was administration. one participant noted, “leadership is so important … i think the leader has to understand how that process [of transdisciplinary teaching and learning] works … but [there has] to be that instructional piece too, to help support the continuation.” another participant gave the example of an inspirational principal, “who has since passed away, who told us our mission. our mission was transdisciplinary [teaching and learning] and how we were going to work as an ib school. we did not have the answers at the beginning but we tried things … and they have carried on that mission for the last seven years. it took a lot of hard work, high expectations for teachers, high expectations for students and parents … we felt like we could not let her down.” a third participant stated, “our administrative team is really strong and the main goal is for us to be a true pyp school. so in almost any way they can they support [transdisciplinary teaching and learning]. it has not always been that way at this school but i would say for the last 5 years that has really been the case. everything is devoted to it, there’s money, there’s time, there’s support.” other participants noted that their administration in some ways hindered transdisciplinary teaching and learning.” one teacher complained, “labelling this as a pyp school is a farce. administration has created an environment, through the timetable and the lack of time for collaborative planning mostly, which focuses more and more on singlesubject instruction and specialist teachers. we are required to do blocks of reader’s and living transdisciplinary curriculum / savage & drake 9 writer’s workshop and standalone math and must write this into our classroom timetable so parents know at all times what we are doing.” timetabling. participants noted that how the timetable was constructed was often directly related to the underlying philosophy of the school and especially that of school administration. on occasion, however, participants stated that the timetable was the result of historical events or current events, such as building construction. one commented, “that if you are truly going to be transdisciplinary you can’t tell teachers to spend x amount of minutes on each subject. that totally contradicts the bigger idea.” similarly, another participant stated, “i personally believe that we are at school to learn. we shouldn’t have to be needing to do math from 9 to 10 o’clock.” another said, “some schools …. [are] very program driven so i was told to have a block of reading instruction for 50 minutes and a block of writing instruction for 50 minutes, etc., and it was a very segmented day. i believe … the more integrated the curriculum is, the more authentic the learning experiences can be for the children and the more meaningful for them. that it makes more sense for them [to not have a strictly set timetable] because their lives are not segmented in that way.” one participant who articulated that transdisciplinary teaching and learning was fully implemented, and implemented well, at his/her school stated, “we are given the time and also the freedom to design units as we like, with of course the oversight of the pyp coordinator. but unlike i know many of the public schools … we are not bound rigidly by timetables or time frames or state testing that we have to do. so just the freedom to merge subjects and deal with them in the pyp way, within the framework [promotes transdisciplinarity]”. a conflict emerged between the amount of time being scheduled to teaching classes by specialist teachers and the regular classroom teachers. one participant explained, “there is a bit of a conflict, so more and more of our daily timetable is being handed over to the single-subject [specialist] teachers … which is leaving homeroom teachers saying we do not have enough time in our daily schedule to teach all of the math, all of the language, all of our units of inquiry. we just don’t have enough time.” another participant noted, “something that really hinders [transdisciplinary teaching and learning] is the schedule because we have these specialist subjects … it would be nice to have it be a bit flowing …because good inquiry i believe really needs at least an hour.” other participants noted the timetable could also hinder transdisciplinary teaching and learning because it does not provide time for the classroom teachers and specialist teachers to collaboratively plan. another agreed, “the classroom teachers have timetabled planning time, it’s just the extra people like the librarian and other single-subject teachers who can’t always get to all of those meetings.” the result of this is that transdisciplinary teaching and learning is decreased. collaborative planning. the majority of participants stated that collaborative planning was essential to transdisciplinary teaching and learning. as one participant said, “you can’t do it without it … so you cannot, absolutely, cannot do any transdisciplinary curriculum without the team meetings.” another participant stated, “it’s key, it really is. my own development and understanding of the power of collaboration really came in when i started collaborating with other people … collaborating with different people from different disciplines and the kind of expertise they have … it builds their understanding of these big ideas but also to see where the links are.” another participant made the point, “at the schools that are very well into their ib you see [collaborative planning] everywhere. for example, at my current school the specialists and the [classroom] teachers work really well together to ensure that transdisciplinary teaching is occurring. other participants, however, identified problems with the collaborative planning process as it currently existed. the participants identified scheduling meetings where everyone international electronic journal of elementaryeducationvol.9, issue1, 1-20, september2016 10 could attend was difficult. as one participant stated, “it is something that is extremely hard to do, it is extremely hard to coordinate schedules … to be honest the biggest complaint that the staff have right now is that we are not coordinating with the specialists.” another participant noted, “we don’t [collaboratively plan] as a whole school because we have specialist teaching … when my students go to spanish that is sort of no contact time for me [and i have no collaborative planning time with the specialist teachers], so it’s not actually transdisciplinary if that makes sense.” another issue with specialist –classroom teacher collaboration was inequity. one participant explained, “what happens in the pyp is they create these units of inquiry as classroom teachers, so these units are already created, and then now [the specialist teachers are] supposed to fit into this. so suddenly we are having to fit into something that has already been created, we haven’t had equal ownership in the process.” similarly, another participant commented, “i think from what i understand it’s from a classroom teacher perspective it’s being done, they are looking at the transdisciplinary skills but in the single subjects it’s still not really being addressed.” a third participant noted, “because i am not [a classroom teacher] i am not at all of the planning meetings and … i am not there for the main parts of it.” a fourth participant stated, “okay so there is conflict between the specialists and the classroom teachers and this notion of transdisciplinarity as some of the specialists feel that their subjects, and themselves personally, are getting short shifted in the name of transdisciplinary teaching.” pyp planner issues. another category, which is related to collaborative planning, is pyp planner issues. in general, most participants stated they liked the backwards design process the planner promoted. one participant stated, “i like the process of it. the process makes sense to me. you have a central idea that fits under the theme, then you come up with …the lines of inquiry and the teacher questions … and then in box 1 is the summative assessment and it gives you … what’s the evidence that they’ve reached the understanding of that central idea by the end of the unit. then from there is box 3 where it shows all the formative assessments.” similarly, another participant stated, “the pyp planners do a really good job of listing everything in that specific order, so we look at what the summative assessment will be for a particular unit and then go backwards.” a third participant noted, “i feel it is necessary. i do feel that it is very time-consuming, especially when you are creating a new unit. i feel it makes sense in terms of the backwards design, it makes sense in terms of, you know, showing the different activities. i think some schools take it to the nth degree and really go overboard while some schools do the exact opposite and there is no use in it at all.” some participants, however, felt there were some issues with the design of the planner itself. one participant stated, “it’s a little too structured and the organization of it, like the skills and the learner profile and things like that, are a bit of an afterthought in the way the planner is organized.” another participant noted, “in the planner they kind of have it a little bit backwards because they don’t even have you decide what your learner profile and your skills are until the bottom of that box 4 and because we are not supposed to do anything [to change the planner] i have actually switched the orders inside the boxes.” a third participant stated, “it is backwards by design, which is great and it really gets you focusing on the things like the key concepts, but it doesn’t provide enough focus on those approaches to learning [skills] and the related concepts.” a fourth participant stated, “i feel like i need more space because there is not enough space there.” participants also noted that the planner was more of an archival document than a living document. most stated they did not actively use the planner while they were teaching. they used it as a guide for their planning and then they returned to it after the unit was complete to reflect on it. many participants talked about creating their own planners that living transdisciplinary curriculum / savage & drake 11 they used more frequently while they were teaching and some participants shared these with the researchers. one participant explained, “i don’t know how much they are using [the planner]. i mean they do sit down with it and they document all of their thinking around it but there is a lot of google docs at our school … and i think that kind of distracts them from all of the things that get placed on the planner… the expectation is that we use the planner but i think it is more a part of the culture of the team now [to use their own google doc planners].” another participant noted, “i have so many things going on in my classroom, like i change things every 10 minutes because of the age of these kids and i don’t have quite enough space on [the planner] so that’s why i colour code my own [planner], where i plan each day.” a third participant stated, “i know in our school our pyp coordinator has devised a series of other documents that we also use but we are not allowed to scrap the pyp planner completely.” a fourth participant said, “right now it is mainly a planning tool that we mainly return to, to do our reflection afterwards and then archive it and then move on to another unit.” lack of experience with the pyp. as one participant stated, “[we should] recruit more teachers that maybe have a bit more experience in the pyp.” discussions involved how there were often issues with teachers who were hired who did not have any experience with the pyp that hindered transdisciplinary teaching and learning. one participant stated, “i’ve seen this at all the ib schools i’ve worked at. when teachers are brought … into the pyp who have no experience with it, they quickly get overwhelmed and often then become negative forces in the school. they often … then start doing their own thing which is to teach standalone units.” another participant noted, “the [negative] attitude comes from lack of understanding, lack of experience, unwillingness to change, and fear of change. it’s not always easy to share what you are doing with someone, a lot of teachers close the door and do their own thing.” a third participant stated, “the pyp is complex. there are a lot of parts to it so for a new teacher to come in that was trained in a different way, they really struggle. there is a lot of stress and frustration. one way to fix that would be to only hire teachers with pyp experience or set up a good training program for new teachers interested in teaching in the pyp that they had to take before they would be hired.” professional development. the participants discussed that professional development could be an important way to foster transdisciplinary teaching and learning in the pyp but they had several concerns over how, and what, was being presented as professional development. one participant stated, “i think professional development is huge but there aren’t a lot of great models for how to teach in a transdisciplinary way.” another participant commented, “[ib] workshops are becoming a little bit dogmatic … they are just so broad because they have so many different kinds of schools that are now starting to use their program and they have to cater to so many people and their workshops are reflecting that. so the depth is gone because the kinds of people going to workshops are fairly different. and then when it comes to the in-school workshops they didn’t really cater to our specific needs whatsoever. we did a workshop on inquiry and we feel that our school is no further along as a result of that workshop, which is really disappointing.” when asked about professional development learning on transdisciplinary teaching and learning, a third participant answered, “transdisciplinary teaching and learning, i am not so sure i would actually say there is much [pd in that area].” other participants discussed the “train the trainer” model that ib appeared to be using for some of its professional development. one participant described, “they have a curriculum to be taught but the person who is delivering it always has their own spin on exactly what it is. and all of those ears in that room they hear what they think the person is saying and they say what they think they heard. the information is then passed down between so many ears and mouths that we are not getting the same message at all … [additionally] international electronic journal of elementaryeducationvol.9, issue1, 1-20, september2016 12 they send some people to different courses every year but there is no consistent way at our school for sharing knowledge acquired at these workshops … so it is possible that a grade two teacher goes and because i do not teach grade two i never hear anything at all about what this person learned.” another participant talked about how there were issues with bringing experts in to the schools as well saying, “we thought we were doing a good job [with transdisciplinary teaching and learning] and [the expert] met with the whole team, pe, music, art, library and we said hey this is what we are doing and she said, whoa, whoa, whoa, hang on a minute, you are not really doing the right thing here and then we didn’t have an opportunity to have a second meeting with her so we never found out how to make it right.” several participants also discussed some ideas of how to make professional development more effective. one participant suggested, “more modelling of what it looks like when you have all the pieces in place, you know, you’ve got the concepts, related concepts, your talk about the learner profile and your attitudes, and what does it look like when somebody is making all that happen in a meaningful way. you know what does it look like when kids are doing it … it doesn’t have to be in-person modelling but you could have videos, you can have people giving tours of classrooms by skype, of other places like that to see what it looks like when its working really well.” another participant suggested, “and there would be job-alikes across the city … where different schools … would offer to do a session … where you just kind of share what you are doing in your field. so it might have to do with a particular transdisciplinary theme, people might just get together to kind of share their ideas about that.” “their learning journey” despite some of the issues the participants discussed which made implementing transdisciplinary teaching and learning in the pyp a challenge, they all stated the pyp benefited students and aided them in their learning journey. in this theme two categories emerged from the participant’s responses: rich performance assessment tasks (rpats) and student-directed learning. rich performance assessment tasks (rpats). rpats were one of the most important aspects of the pyp; they contributed both to the program’s relevance for students and its accountability to local, ib and national standards. the participants described significant learning tasks their students had accomplished such as a grade 2 class going out into the rainforest for 5 days to learn about natural resources. many of the participants described these tasks as their best experiences in the pyp, and provided them as exemplars of transdisciplinary teaching and learning. participants described them in the following ways: “the kindergarteners were studying about celebrations … under the theme ‘how we express ourselves’ they were learning that people celebrate for different reasons and that individual celebrations are different around the world and that there is always a purpose. and so as they went to each of the specialists throughout their unit and they planned as a whole kindergarten by voting on which part they would like to create for their own celebrations. so week-by-week they sort of built on it, they picked what colours they would need for a celebration and the purpose of their celebration and what would be included in their celebration. they planned out the whole thing with the guidance of the teachers and then they had their celebration night … all created by the kids and guided by the teachers.” “last year, when i taught in the kindergarten classroom, doing a five senses unit we were able to … go out into the community, visiting pet stores, visiting the library and the fire station, inviting community members in and seeing the excitement on the children’s faces when they were able to have community members come in and share their knowledge … my best teaching probably related to that unit was working with my co-teacher and digging really living transdisciplinary curriculum / savage & drake 13 deeply into the five senses, authentic experiences like drawing blindfolded and smelling different smells and practising sign language. it was a great unit.” “so we were doing a unit on water conservation and the environment and it was a grade four class and they lacked interest, they lacked desire, they just wouldn’t engage in it at all. so with the permission of [the administrator] i set about turning one of the courtyards into an experimental area and we called it ‘the last bucket of water [in the country]’. the students basically had to decide how they were going to preserve it … the students were able to access research data that they perhaps would not have ever known was available to them and as a result of that we actually won a government award … there were several hiccups that i created along the way where the weather started to warm up and i may have helped evaporation along a little bit and they had to figure out what would happen now … there was an overnight pollution episode that occurred and they had to figure out how to clean the water without losing the water. it was transdisciplinary … we were measuring water, we were using fractions and things like that. we used literacy because we were reading about things that we could do to protect our water, we were writing letters to government departments asking for guidance and information. we were conducting science experiments on dirty water to see how we could clean it. you know we did some social studies and some geography because we were figuring out countries that didn’t have problems with water from a volume point of view so it was about the [problem] in every aspect of their learning.” “the children were exploring public spaces and how they serve the diverse needs of the community. so at the end of the unit the children were asked to create a public space with recyclable materials and then kind of explain what type of public space they have created and how people would use it. this is a great one. that seems to be quite an authentic task to assess children’s understanding of that big idea.” “we have a lot of trouble with stray animals in [our country] so they were interested in this idea so they started to explore everything and it was very difficult for them to narrow down what they wanted to explore. so first we got them down to a couple of different concepts, where they decided they wanted to look at how rescuing stray animals [in their country] works and what the history of that is, which we can call change, and how people have been helping stray animals and what our responsibility is as people. and so they started and came to the conclusion through education that people [in their country] don’t really have pets, they have like dogs for guarding but they don’t have pets so it’s very easy for them to just put a dog out on the street because they don’t see them the same way western people do. we have a lot of western kids at our school, or more westernized i suppose, so they find this quite upsetting. so they decided through action and education they could improve the lives of stray animals in [their country] … that gave them the focus to carry on and move forward with their work [their solution to the problem]”. “we also have a school store that third graders have to apply to run and we link it to ib, they have to tell what ib attitude would help them with this job or why they should be the one, or how they can show leadership by working at the school store. it’s like an application, like a real job application. we have been able to have 30 of our 100 students do that this year.” student-directed learning. many participants talked about student-directed learning being the epitome of transdisciplinary teaching and learning and a goal they were striving to do more of in the classroom. one coordinator stated, “there were some units that i really did feel were more transdisciplinary … we would put students in charge during the time and so they had some leadership roles within it. but then as teachers watched the inquiry really unfold, responded to questions and planned little mini-lessons and such to help [students] move forward in their thinking and then turned it back over to them and continued to watch and assess how things were going. so i really loved the amount of student direction within that unit and really for once i felt like that guide on the side, the presence of really moving them through their learning journey”. another participant replied, when asked what would improve the pyp at his/her school, “getting more of a student-centred approach to learning going here and really seeing that materialize.” international electronic journal of elementaryeducationvol.9, issue1, 1-20, september2016 14 responding to the same question another participant stated, “i think the attitude of the school will be to embrace children for being children and young learners and minds, not minds to be opened up and shoved information in, but people who can develop understanding on their own with opportunities to grow and be creative.” several participants discussed trying to bring in student-directed learning through genius hour, or passion projects, where students are allowed to direct their own learning. genius hour is a teacher-created phenomenon based on a practice at google where employees get a certain % of paid work time to work on inquiries of their own interest. in schools, students are given a set amount of time – for example, every friday afternoon to generate their own questions and conduct their own inquiries based on their personal interests and not the set curriculum. one participant stated, “what is happening a lot more is that … we have something, some people call it 20% time, genius hour, passion project, we have a lot of that starting to happen in the school as well because we do find we are struggling to find the time for really deep inquiry in our units because of all these time restrictions and that we want students to have real time like this … student-led inquiry is exciting for kids and that is where you are teaching those skills and the knowledge is coming more from the kids. so a lot in the upper [primary] grades they devote some time to genius hour every week.” another participant stated, “we also know how important it is for kids to guide their own learning and take steps to build their own questions and learn more and take next steps. so last year a quick story would be the specialists got together and talked about how we could let the kids plan their own unit and that turned into planning their own activity night. so now we do activity nights that complement our units.” other participants agreed, stating, “one of the biggest things that ib is trying to do is to let kids take ownership over their learning and being able to plan their learning and extend it on their own.” as evidence for this statement they made reference to the final exhibition, which is an excellent example of an rpat. one participant explained it by saying, “when students do an exhibition, when they start the process of exhibition, and it’s the closure to the pyp, it’s almost like they are kind of going through a unit planning process itself. they don’t literally do that but a lot of times they sort of do that in small groups. a group of students will take a topic, maybe a branch of what we are studying and they develop their own lines of inquiry, they develop their own action piece, they develop what their central idea is going to be, what they are going to study. so they really are, they really develop their own unit. it’s like a project that they develop and it really is a demonstration of their learning.” similarly, another participant stated, “it’s the closure to the pyp program and when we had our exhibition it was exciting to see the students, how they took on the learning themselves and they put together their whole projects, their experiences, and they really went through a whole unit planning process in their little groups and when they were sharing their knowledge and sharing how they connected with the community to do research about topics, kind of a branch off of what we were studying and it just led to other levels of learning. i think it was really exciting to see that we had the exhibition in a couple of different museums and the students and the community came out to watch them and to just watch the students talk about their learning. i think that was really exciting.” a third participant outlined, “i am fairly involved with the final exhibition … i do a lot of work with the whole class, so with 80 kids, and for this they have to explore the different key concepts that the ib uses and it’s hard for them to choose this because they do not have to do this at any other time in their lives [up to this point]. so we really explore what those concepts are and they break them down and they have to make their own central idea, which is kind of the big idea that they want people to take away from their project … student-led inquiry is the biggest piece of [transdisciplinary teaching and learning], where living transdisciplinary curriculum / savage & drake 15 they really understand what they are trying to explore.” a fourth participant questioned, “i don’t understand why ib does not just make the pyp a series of exhibitions in each grade. everyone, teachers, parents, the kids themselves, all value it the most, so why not base the entire program on student-directed learning?” conclusion as demonstrated by the themes above, all of our participants appreciated the potential of transdisciplinary teaching and learning. for the most part their comments about their experiences were positive. the more critical comments some participants made were not directed toward transdisciplinary teaching and learning itself, but rather toward poor implementation of the pyp. there are several conclusions which can be drawn from the participants’ experiences which may assist other educators who wish to implement transdisciplinary teaching and learning into their own practice. a majority of participants understood transdisciplinary teaching and learning, especially as outlined in the pyp documents (ibo, 2009; 2012a; 2012b), to be a framework with which they could work and which was flexible and transportable to different contexts and cultures as transdisciplinary teaching and learning is not culture-bound. such a framework seems particularly important in today’s society where global knowledge is required to solve the world’s complex problems, drive innovation, and expand the global economy. this level of flexibility is also helpful because a variety of subject specific scope and sequence documents may be integrated into the transdisciplinary framework. scope and sequence documents might come from national or state education mandates or have been locally developed by school administrators and curriculum specialists. the transdisciplinary framework itself, however, needs to clear, concise and easy for educators to understand and use. the know, do, be model (drake, 2007, 2012; drake, reid and kolohon, 2014), for example, is an elegant framework which promotes transdisciplinary teaching and learning. the transdisciplinary framework seems to be most helpful when it includes a curriculum planning tool to be used in every school; that tool focuses on backwards design principles and leads to rich performance assessment tasks (as opposed to traditional paper and pencil tasks, tests and exams) (dobozy & dalziel, 2016). a planner needs to be easy to understand and can be used on a daily basis, not just as an archival document (dobozy & dalziel, 2016). as highlighted by the participants, time to collaboratively plan is essential when implementing transdisciplinary teaching and learning. all the educators involved in curriculum planning and implementation need scheduled time to meet with each other and come up with the outline of the transdisciplinary units. this included generalist, classroom teachers, single-subject specialist teachers and any curriculum coordinators or specialists who are involved. it was important that this collaborative planning time was scheduled throughout the school year and not just at the beginning of the year or term. participants described transdisciplinary teaching and learning as being very fluid so frequent collaborative planning sessions are necessary so everything remains running smoothly. this is consistent with several studies that have found collaborative planning is not only beneficial for transdisciplinary curriculum development but also increases teacher efficacy and teachers’ sense of empowerment (linnell, zidenberg-cherr, briggs, scherr, brian, hillhouse, & smith, 2016; miller, 2013; tan & nashon, 2015) in addition to collaborative planning time, large blocks of time during the school day need to be devoted to transdisciplinary teaching and learning (as opposed to an hour for reading and an hour for math) (miller, 2013; nompula, 2012). when blocks are only loosely structured, more student-directed learning may occur. during these times students international electronic journal of elementaryeducationvol.9, issue1, 1-20, september2016 16 will likely be actively engaged in their own, or teacher-led, inquiry projects. participants described how strict timetabling makes it very difficult to implement transdisciplinary teaching and learning because it is very difficult to determine how long each inquiry activity will last. another element that seems required for successful implementation is that everyone ‘buy in’ to the program. supportive administrators who devoted time, resources, scheduled collaborative planning time and flexible classroom schedules led to effective implementation. successful teachers believed that the program promote student teaching and learning and they tended to be comfortable with the messiness that inquiry-based, transdisciplinary teaching and learning brings. several studies have recognized the need for school administrators to develop strategies that increase teacher motivation and confidence in the changes administrators are hoping to effect, no matter what the new initiatives involve (epstein & willhite, 2015; marsh, 2015). other researchers have found that teacher efficacy is a key component of successful curriculum development and implementation (la porte, 2016; nurlu, 2015) the transdisciplinary program did not just happen. a solid professional development was in place to give formal training on how to implement transdisciplinary teaching and learning (burton, 2012). this would be recommended to insure educators are on the same track (scripp & paradis, 2014, thomas, hassaram, rieth, raghaven, kinzer & mulloy, 2012). but, if this approach is as effective for the 21st century as it seems to be, then it should be implemented more widely. faculties of education need to include transdisciplinary teaching and learning methods into their curriculum. teacher candidates need to be given the opportunity to engage in this type of education during their practice teaching blocks and should be provided with opportunities to see transdisciplinary teaching and learning being modelled (corlu, capraro, & corlu, 2015; dowden, 2014; thomas et al., 2012). we hope that by considering the lessons learned from this study, more educators will feel comfortable implementing transdisciplinary programs. such programs that are both accountable to local and national mandates and still relevant to students while preparing them for the complexity of the 21st century world will greatly benefit the next generation. • • • acknowledgements this research was supported by a grant from the international baccalaureate organization. references boyer, e. (1995). the educated person. the 1995 ascd yearbook. alexandria, va: ascd. bridwell-mitchell, e. n., & mezias, s. j. (2012). the quest for cognitive legitimacy: organizational identity crafting and internal stakeholder support. journal of change management, 12(2), 189207. burton, l. l. (2012). professional development in an international baccalaureate primary years programme (doctoral dissertation). retrieved from proquest llc. (ed538528). living transdisciplinary curriculum / savage & drake 17 campbell, c., chittleborough, g., jobling, w., tytler, r., & doig, b. (2014). science literacy in the international baccalaureate primary years programme (pyp): nap-sl outcomes. melbourne, australia: school of education, deakin university. canter, c., & brumar, c. i. (2011). transdisciplinary niches fostering lifelong learning. procedia – social and behavioral sciences, 28, 636-639. corlu, m. s., capraro, r. m., & corlu, m. a. (2015). investigating the readiness of pre-service teachers for integrated teaching. international online journal of educational sciences, 7(1), 17-28. creswell, j. w. (2012). qualitative inquiry and research design: choosing among five approaches (3rd ed.). thousand oaks, ca: sage. dobozy, e., & dalziel, j. (2016). transdisciplinary pedagogical templates and their potential for adaptive reuse. journal of interactive media in education, 1, 1-11. dowden, t. (2014). challenging, integrated, negotiated and exploratory curriculum in the middle years of schooling: designing and implementing high quality curriculum integration. australian journal of middle schooling, 14(1), 16-27. drake, s. m. (1993). planning integrated curriculum: the call to adventure. alexandria, va: ascd. drake, s. m. (2012). creating standards-based integrated curriculum: the common core state standards edition. thousand oaks, ca: corwin press. durlak, j., weissberg, r. p., dymnicki, a. b., taylor, r. d., & schellinger, k. b. (2011). the impact of enhancing students' social and emotional learning: a meta-analysis of school-based universal interventions. child development, 82(1), 405-432. epstein, a., & willhite, g. l. (2015). teacher efficacy in an early childhood professional development school. international electronic journal of elementary education, 7(2), 189-198. fullan, m. (2013). stratosphere: integrating technology, pedagogy, and change knowledge. toronto, on: pearson. gough, a., sharpley, b., vander pal, s., & griffiths, m. (2014). the international primary years programme in victorian government primary schools, australia. bethesda, md: international baccalaureate organization. guyette, k. w., sochaka, n. w., & costantino, t. e. (2015). collaborative creativity in steam: narratives of art education students’ experience in transdisciplinary space. international journal of education, 16, 2-38. hargreaves, a., & fullan, m. (2012). professional capital: transforming teaching in every school. new york, ny: teacher’s college press. hargreaves, a., & shirley, d. (eds.; 2009). the fourth way: the inspiring future for educational change. thousand oaks, ca: corwin press. international baccalaureate organization. (2009). making the pyp happen: a curriculum framework for international primary education. cardiff, wales: author. international baccalaureate organization (2012a). developing a transdisciplinary programme of inquiry. cardiff, wales: author. international baccalaureate organization (2012b). what is an ib education? cardiff, wales: author. klein, j. t. (2014). discourses of transdisciplinarity: looking back to the future. futures, 63, 68-74. doi: 10.1016/j.futures.2014.08.008 kushner, s., cochise, a., courtney, m., sinnema, c., & brown, g. (2016). international baccalaureate primary years programme in aotearoa new zealand: a case-study in whole-school innovation. bethesda, md: international baccalaureate organization. international electronic journal of elementaryeducationvol.9, issue1, 1-20, september2016 18 la porte, a. m. (2016). efficacy of the arts in a transdisciplinary learning experience for culturally diverse fourth graders. international electronic journal of elementary education, 8(3), 467-480. lester, j. n., & lochmiller, c. r. (2015). a mixed-methods case study of international baccalaureate primary years programmes in four colombian schools. bethesda, md: international baccalaureate organization. linnell, j. d., zidenberg-cherr, s., briggs, m., scherr, r. e., brian, k. m., hillhouse, c., & smith, m. h. (2016). using a systematic approach and theoretical framework to design a curriculum for the shaping healthy choices program. journal of nutrition education and behavior, 48(1), 60-69. marsh, s. (2015). a model forleadership that improves learning: new insights for schools and scholars. leadership & policy in schools, 14(1), 67-103. miller, b. a. (2013). joining forces: a collaborative study of curricular integration. international journal of education & the arts, 14(special issue), 1-24. morrissey, a., rouse, e., doig, b., chao, e., & moss, j. (2014). early years education in the primary years programme: implementation strategies and programme outcomes. bethesda, md: international baccalaureate organization. moustakas, c. (1994). phenomenological research methods. new york, ny: sage publications. nicolescu, b. (1998). the transdisciplinary evolution of the university condition for sustainable development. paris, france: centre international de researches et etudes transdisciplinaries. nicolescu, b. (2008). transdisciplinarity – theory and practice (ed.). cresskill, nj: hampton press. nompula, y. (2012). an investigation of strategies for integrated learning experiences and instruction in the teaching of creative art subjects. south african journal of education, 32(3), 293-306. nurlu, o. (2015). investigation of teachers’ mathematics teaching self-efficacy. international electronic journal of elementary education, 8(1), 489-507. pop, i. g., & maties, v. (2008). a transdisciplinary approach of the mechatronical education in the context of the knowledge-based society. problems of education in the 21st century, 8, 90-96. richards, j. c., & shea, k. t. (2006). moving from separate subject to interdisciplinary teaching: the complexity of change in a pre-service teacher k-1 early field experience. the qualitative report, 11(1), 1-19. scripp, l., & paradis, l. (2014). embracing the burden of proof: new strategies for determining predictive links between arts integration teacher professional development, student arts learning, and student academic outcomes. journal for learning through the arts, 10(1), 2-18. sillisano, j. r. et.al. (2010). evaluation of international baccalaureate programmes in texas schools. college station, tx: state of texas education centre. tan, l. & bibby, y. (2010). ib pyp and myp student performance on the international schools’ assessment (isa). melbourne: australian council for educational research. tan, y. s. m., & nashon, s. m. (2015). promoting teachers’ collaborative exploration of a new science curriculum: the case of a singapore learning study. professional development in education, 41(4), 671-689. thomas, c. n., hassaram, b., rieth, h. j., raghaven, n. s., kinzer, c. k., & mulloy, a. m. (2012). the integrated curriculum project: teacher change and student outcomes within a universityschool professional development collaboration. psychology in the schools, 49(5), 444-464. wiesmann, u., biber-klem, s., grossenbacher-mansuy, w., hadorn, g. h., hoffman-riem, h., joye, d., … & zemp, e. (2008). enhancing transdisciplinary research: a synthesis in fifteen propositions living transdisciplinary curriculum / savage & drake 19 in g. h. hadorn, h. hoffman-riem, s. biber-klemm, w. grossenbacher-mansuy, d. joye, c. pohl, u. wiesmann, & e. zemp (eds.), handbook of transdisciplinary research (pp. 433-441). new york, ny: springer. wiggins, g. & mctighe, j. (2006). understanding by design. alexandria, va: ascd. international electronic journal of elementaryeducationvol.9, issue1, 1-20, september2016 20 www.iejee.com this page is intentionally left blank http://www.iejee.com/ 187 © 2022 published by kura education & publishing. this is an open access article under the cc byncnd license. (https://creativecommons. org/licenses/by/4.0/) copyright © www.iejee.com issn: 1307-9298 international electronic journal of elementary education january 2023, volume 15, issue 3, 187-197 the crossroads: interdisciplinary teams and alternative treatments justin b. leafa,*, joseph h. cihonb, julia l. fergusonc, christine milned, misty l. oppenheim-leafe abstract introduction behavior analysts collaborating within interdisciplinary teams are likely to find themselves at difficult crossroads. some of these crossroads include implementing alternative treatments, defining and determining risk and harm, and evaluating research and interventions. the purpose of this paper is to highlight some of these crossroads and provide guidelines on successfully navigating them. we contend that it is possible to navigate these crossroads while minimizing harm or risk for the client, adhering to the principles of science and behavior analysis, and remaining respectful of all members of the interdisciplinary team. that is, we can maintain the scientific tenets of philosophic doubt, empiricism, and experimentation, while remaining humble, and ensuring our clients access the most effective interventions available. behavior analysts value effective collaboration with professionals within and across different disciplines and with our consumers. as kelly and tincani (2013) stated, “there is no standard operational definition for collaboration...” (p.121). despite this, behavior analysts have continued to discuss collaboration with interdisciplinary teams within the peer-reviewed literature (e.g., bowman et al., 2021; kelly & tincani, 2013; lafrance et al., 2019). across the behavior analytic literature on collaboration with interdisciplinary teams, there appears to be consensus that to effectively collaborate, behavior analysts should engage in a multitude of behaviors, including: (a) joint problem solving, (b) recognizing the strengths of other members of the interdisciplinary team, (c) active listening, (d) engaging in good communication amongst team members, (e) having a universal and agreed upon code of ethics, (f) not engaging in disciplinary centrism, and (g) not being judgmental (e.g., bowman et al., 2021; brodhead, 2015; cox, 2012; galloway & sheridan, 1994; hall, 2005; kelly & tincani, 2013; lafrance et al., 2019; lawson, 2004). for collaboration to occur, “it is imperative that all members of a team recognize their own knowledge limitations and value the expertise afforded by professionals who have keywords: alternative treatments, collaboration, evidence based practice, autism received : 2 january 2023 revised : 15 february 2023 accepted : 20 march 2023 doi : 10.26822/iejee.2023.291 a,* corresponding author: justin b. leaf, autism partnership foundation, endicott college, california, usa. e-mail: jbleaf@apfmail.org orcid: https://orcid.org/0000-0001-8315-7203 b joseph h. cihon, autism partnership foundation, endicott college, usa. e-mail: jcihon@apfmail.org orcid: https://orcid.org/0000-0001-9272-7749 c julia l. ferguson, autism partnership foundation, endicott college, usa. e-mail: jferguson@apfmail.org orcid: https://orcid.org/0000-0001-6863-3152 d christine milne, autism partnership foundation, usa. e-mail: cmilne@apfmail.org e misty l. oppenheim-leaf, contemporary behavior consultants, usa. e-mail: mistyoleafcbc@gmail.com 188 january 2023, volume 15, issue 3, 187-197 been trained in other areas” (lafrance et al., 2019, p. 710). additionally, all members of the team need to value maximizing benefits while minimizing any potential harm. an interdisciplinary team should: (a) develop shared goals, (b) develop interventions jointly, (c) develop data tracking methods, and (d) set a plan for communication (newhouse-oisten et al., 2017). cox (2012) discussed the importance of a united set of ethical principles that provides common ground amongst professionals in an interdisciplinary team. these common ethical principles and obligations for interdisciplinary autism interventionists included: (a) beneficence, (b) nonmaleficence, (e) respect for persons, and (d) professional commitments. these principles were provided to help ensure competence and define acceptable behaviors, so all members of the interdisciplinary team are held to a high standard from the outset. brodhead (2015) created a decisionmaking model to help behavior analysts determine if a nonbehavioral treatment would put the client’s safety at risk, how to navigate with colleagues if the client’s safety is at risk, and how the behavior analyst can navigate with colleagues if the client’s safety is not at risk. this decision-making model can be useful to help behavior analysts navigate if they should address concerns about alternative treatments with outside colleagues. bowman and colleagues (2021) expanded the tools available for behavior analysts working on interdisciplinary teams by outlining standards for interprofessional collaboration when providing intervention for autistics/individuals diagnosed with autism spectrum disorder1 (asd). these standards included: (a) collaborative communication, (b) roles in collaboration, (c) what the roles are for individuals within the organization, (d) ensuring quality client care, (e) ways to resolve conflict, (f) creating joint partnerships across professionals, and (g) best ways to ensure appropriate treatment approaches across disciplines. finally, kirby and colleagues (2022) described a framework to help guide behavior analysts to develop skills to advance and maintain professional relationships when working in an interdisciplinary team, specifically with respect to cultural reciprocity. the framework included skills such as self-reflecting, listening, validating, and compromising. the ethical codes for which certified behavior analysts are bound also place value on and promote collaboration. for example, the board-certified behavior analyst (bcba) code of ethics for bcbas specifically notes that behavior analysts are to collaborate with others in the best interest of clients and stakeholders (behavior analyst certification board, 2020). further, the behavior analyst certification board (bacb) code of ethics for bcbas states that any conflicts should be addressed “by compromising when possible and always prioritizing the best interest of the client” (behavior analyst certification board, 2020, p. 11). the international behavior analysis organization’s (ibao) ethics code (international behavior analysis organization, 2021) directly states “certificants participate in collaborative relationships with professionals in other disciplines and treatment teams, prioritizing the client’s best interest” (p. 8). the recently developed progressive behavior analyst autism council (pbaac) ethics code (progressive behavior analyst autism council, 2022) also highlights the importance of collaboration and states, “the cpba-ap works collaboratively with other professionals to serve their clients effectively within the context of providing only evidence-based procedures/interventions” (p. 3). the numerous benefits of effective collaboration have been long discussed in the literature. for example, lawson (2004) stated that effective collaboration could better enhance problem solving. hall (2005) concluded that effective collaboration could result in better outcomes for clients and that professionals will report higher job satisfaction. galloway and sheridan (1994) found that effective collaboration is preferred by clients, results in better treatment integrity, and leads to better maintenance of gains. brodhead (2015) provided a decision-making model to assist behavior analysts navigate non-behavioral treatment recommendations by members of a treatment team to help maintain professional collaboration and help develop a better understanding of other approaches to interventions. in addition to collaboration, behavior analysts have also placed value on the use of interventions that are effective, efficient, and conceptually systematic (baer et al., 1968, 1987). as such, behavior analysts only implement, recommend, and endorse procedures that would be considered an evidencebased practice (ebp) and are empirically supported. although promoting and only using ebps and ensuring effective collaboration are core values of behavior analysis, sometimes these two values find a behavior analyst at a crossroads. for example, what does a behavior analyst do when providing services for a client in a clinic setting and an outside professional recommends a procedure for that client that may be harmful? what if a behavior analyst is working in a school setting, as part of an interdisciplinary team, and an outside professional recommends an intervention that will not cause physical harm to the learner but has been proven ineffective in the empirical research? what if a behavior analyst, who is also a researcher, is attending a conference where presenters are touting misinformation about behavioral intervention, while making inaccurate claims about their intervention? should the behavior analyst remain silent or complacent with the alternative interventions, make compromising statements (e.g., “that is fine as long 189 the crossroads: interdisciplinary teams and alternative treatments / leaf, cihon, ferguson, milne & oppenheim-leaf as we use progress monitoring data to evaluate the decision”), or critique those interventions based upon the universal concepts of science and research but risk being ostracized from the interdisciplinary team or accused of lacking humility? there are no easy answers to these questions, and the context as well as many other variables would likely influence possible answers and solutions. when the concepts of evidence based/alternative treatment and collaboration come to a crossroads, it is difficult for any behavior analyst to navigate the collaborative circumstances effectively. while there have been guidelines and tools created and discussed within the literature to help behavior analysts navigate these issues (e.g., brodhead, 2015; kirby et al., 2022), the guidance and tools likely reflect the authors’ personal values (brodhead, 2015) and more discussion and guidance is warranted. thus, the purpose of this paper is to provide some guidelines for navigating collaboration within an interdisciplinary team. some crossroads as previously noted, there are several potentially difficult crossroads to navigate when collaborating as a behavior analyst on an interdisciplinary team. what follows are examples of three major crossroads related to ebps and alternative treatments that are likely to occur when collaborating as a behavior analyst on an interdisciplinary team. implementing alternative treatments the first crossroad with respect to navigating collaboration within an interdisciplinary team is that as a field relates to many certified behavior analysts implementing alternative treatments. marshall and colleagues (2023) conducted a survey of doctoral level bcbas (i.e., bcba-ds), bcbas, board certified associate behavior analysts (bcabas), and registered behavior technicians (rbts) on the implementation of alternative treatments. the responses from 921 individuals were evaluated regarding the implementation of 30 different interventions that were categorized as: (a) effective (e.g., interventions based on the principles of applied behavior analysis), (b) interventions that have emerging evidence (e.g., music therapy), (c) interventions that have not been proven (e.g., social thinking™), and (d) interventions that can be considered ineffective and possibly harmful (e.g., rapid prompting method). these categories were created using national standards that have been developed to evaluate autism interventions (national autism center, 2009, 2015). the results of the survey showed that bleach therapy was the only one of the 26 interventions that was categorized as either having emerging evidence, not established, or harmful, that was not reported as being implemented by the respondents (i.e., those certified by the bacb). as such, interventions such as social thinking™, hyperbaric oxygen chamber, and relationship development intervention were reported as being implemented. furthermore, 2.6% of bcba-d respondents indicated that they implement rapid prompting method. these results are concerning in that certified behavior analysts, who are ethically bound to the use of ebps, are implementing alternative and dangerous interventions. defining risk and harm another crossroad that occurs when navigating collaboration within an interdisciplinary team is the lack of a comprehensive definition of harm or risk. brodhead (2015) defined risk as “…as any treatment that will likely cause shortor long-term psychological or physical harm to the client” (p. 72). this definition is a great step in defining risk and harm, as it guides behavior analysts away from interventions that may cause psychological or physical harm. as such, behavior analysts should be less likely to implement interventions such as chelation or bleach therapy and more likely to avoid and advocate against their use within the interdisciplinary team. brodhead’s (2015) definition of risk is a great start; however, we suggest that it should be expanded further to mitigate possible risk and harm for our clients. specifically, we contend that there is harm and risk involved in spending valuable instructional time on ineffective alternative treatments. for example, social thinking™ is an intervention that is not empirically supported (leaf et al., 2018; leaf et al., 2016) and is considered as an unestablished intervention within the national standards project evaluating autism interventions (national autism center, 2015), but would not be considered a risky intervention or harmful based on brodhead’s definition. as a result, if a professional on an interdisciplinary team in a school setting recommends the implementation of social thinking™ for 30 min per day and this recommendation is implemented, it would result in the team implementing 5,400 min (i.e., 90 hr) of social thinking™ across an entire 180-day school year. that is 90 hr of instructional time that could have been spent on an intervention with documented effectiveness. while 90 hr of instructional time may not seem significant, researchers have shown the tremendous outcomes that autistics/individuals diagnosed with asd can make with even less time. for example, leaf et al. (2017) demonstrated that participants of a behaviorally based social skills group that lasted a total of 64 hr resulted in the participants acquiring over 100 social and adaptive skills. the results also indicated significant improvements on standardized social and adaptive assessments. if there is agreement that there is harm and risk involved in spending valuable instructional time on ineffective alternative treatments, 190 january 2023, volume 15, issue 3, 187-197 behavior analysts must weigh that risk and harm with advocating against the use of alternative treatments and possibly eroding professional relationships, being removed from the interdisciplinary team, or not being labeled as a humble behaviorist (kirby et al., 2022) evaluating research and interventions kirby et al. (2020) contended that “disciplinary centrism within our field [behavior analysis] can also result in claims that programs designed by nonbehavior analysts are unscientific and not supported by evidence when such practices do not readily fit within our behavior analytic model...” (p. 136). kirby and colleagues further argued that “hubris is at the core of the idea that if an intervention wasn’t designed by a behavior analyst, it is not scientific nor supported by evidence” (p. 136). it is true that an intervention should not be considered not scientific or supported by evidence just because an intervention was not designed by a behavior analyst. rather, an intervention is not scientific or supported by evidence if it does not meet well established standards in research and practice. as such, even interventions developed by behavior analysts can be classified as not scientific or supported by evidence if they have not met the criteria to be considered as such. kirby et al. (2020) also implied that the hallmarks of research and science are not universal and that behavior analysts cannot, or should not, evaluate non-behavioral research using well established standards in research and science. it should be noted that the value behavior analysts place on the use of single case designs was developed, in part, by evaluating and identifying the limitations of other common research methodologies (sidman, 1960). the research methodology used by and advocated for by behavior analysts has resulted in numerous advances in the development of meaningful behavior and amelioration of dangerous behavior (e.g., functional analysis; iwata et al., 1982). it, therefore, seems reasonable and fruitful to evaluate research, behavior analytic or otherwise, using the tenants of behavior analytic research methodology (e.g., operationally defined and observable dependent variables, clearly described independent variables, interobserver reliability). relatedly, it is important to note that threats to internal and external validity go across various disciplinary research. thus, threats of maturation, history, testing, instrumentation, regression, selection, or mortality within research exist across disciplines (e.g., psychology, occupational therapy, behavior analysis) and researchers (e.g., behavior analysts, speech language pathologists, occupational therapists). additionally, while some research designs are more common (e.g., single case designs within behavior analysis) and less common (e.g., pre-test post-test control group designs within behavior analysis) within different disciplines, no field can lay claim to one or more designs. rather, the research design should be selected based on what best answers the experimental question and that design should be implemented as designed to ensure as much experimental control as possible. just as the possibility of threats to external and internal validity are not unique to any discipline’s research, the hallmarks of science, pseudoscience, and antiscience (green, 1996) are not concepts that belong to the field of behavior analysis. rather, they are universal concepts and principles that can be applied to any intervention or population demographic. listening and/or following the topic of listening is not new within the behavior analytic literature, but much of that literature is discussing listening from a verbal behavior perspective (e.g., hayes, 1996; schlinger, 2008). recently, the topic of listening within the literature and other outlets (e.g., social media) has shifted to listening to consumers of applied behavior analysis (aba) interventions and autistic advocates. it should be noted, however, that listening to consumers has been a hallmark of aba for many years (i.e., wolf, 1978). discussing listening from a verbal behavior perspective is beyond the scope of this paper, but a crossroad many behavior analysts will find selves at while navigating an interdisciplinary team is what does it mean to listen. it appears that listening, in this context, does not necessarily mean agreeing or avoiding disagreement especially in situations in which risk or harm are likely (brodhead, 2015). if this is the case, then a behavior analyst could listen to the recommendations of other members of the team, while still disagreeing and advocating for an alternative recommendation. however, some may contend that listening requires following as opposed to discourse, difficult discussions, and possible compromise. in these situations, not following may lead to being ostracized or accused of not listening to other members of the interdisciplinary team. when this is the case, the pathway forward for the team is often a rocky one. navigating the crossroads: practitioners there is no doubt that challenges will arise for behavior analysts navigating the crossroads of collaborating with members of an interdisciplinary team and the implementation of alternative treatments. there have been some previous discussions within the literature to help provide guidance to behavior analysts in these situations (e.g., brodhead, 2015; kirby et al., 2022). what follows are some additional guidelines to navigate these crossroads for practicing behavior analysts. we, like others (e.g., brodhead, 2015), acknowledge that the guidelines we offer are influenced by our professional and personal histories. furthermore, we offer these as guidelines and encourage the reader 191 the crossroads: interdisciplinary teams and alternative treatments / leaf, cihon, ferguson, milne & oppenheim-leaf to view them as such (i.e., not apply them as rules to use across interdisciplinary teams as each team is likely to be unique and require the behavior analyst to use clinical judgement to address challenges as they arise). thus, a behavior analyst/technician must evaluate their specific situation (e.g., environmental variables) when considering these guidelines. determine your professional values: the implementation of alternative treatments one initial step any practicing behavior analyst should take is determining their personal values with respect to the implementation of alternative treatments. said differently, which procedures are you comfortable or uncomfortable implementing? some have conceptualized different interventions using a red, yellow, green system (e.g., association for science in autism treatment, n.d.; autism new jersey, n.d.; weiss et al., 2022). weiss et al. (2022) described each of these tiers as, green light treatments signal efficacy. yellow light treatments are those that should be implemented with caution and need additional research regarding their potential impact. red light treatments are those that have been proven to be ineffective and/or harmful and that should not be implemented (p. 143). more specifically, green light interventions are those that would be considered ebps with documented evidence of their effectiveness. yellow light interventions are those that may have emerging evidence, but are not ebps, or those that may pose little risk or harm for the client and can be closely monitored and altered quickly if necessary. those who are uncomfortable with this three-tiered system may choose to adhere to a stricter red/green system, where there are only interventions that meet or do not meet the requirements to be implemented. regardless of the system the practicing behavior analyst chooses, they must determine how they view ebps and non-ebps. will you view ebps as a list of procedures and packages that meet qualifications regarding the available evidence (e.g., national autism center, 2009, 2015), in terms of manualized treatment packages that consist of randomized control trials to support their effectiveness (i.e., smith, 2013), or as a decision making model drawing upon the best available scientific research, client values and context, and a behavior analyst’s clinical expertise (e.g., slocum et al., 2014)? this view will help determine the system to use to help determine if a procedure should or should not be implemented as well as the components of that system (e.g., which interventions should be green, how to determine if an intervention is yellow). select a professional environment that aligns with your values it is imperative that practicing behavior analysts select a workplace whose organizational values align with their own. this could help prevent some of the challenges associated with the aforementioned crossroads. for example, if a practicing behavior analyst decides that implementing procedures with no empirical data to support their use does not align with their values, then it would be helpful to avoid working for organizations where those procedures are readily implemented. conversely, and hopefully unlikely, if a practicing behavior analyst decides that implementing procedures with limited to no empirical data to support their use does align with their values, then it would be helpful to avoid working for organizations where those procedures would not be implemented under any conditions. as such, it is important to proactively evaluate possible employers/organizations prior to employment to see if their values align. if this preliminary evaluation results in identifying an organization with aligned values, it will be important to follow-up and ask questions (e.g., “does your organization support the implementation of alternative treatments?”) during the interview process to confirm the results of the preliminary evaluation and address any areas that may be unclear. concurrently, the interview is a good opportunity to describe your values and any possible areas of conflict and how those conflicts will be resolved. be as proactive as possible similar to progressive approaches to treating challenging behavior (e.g., ala’i-rosales et al., 2019), practicing behavior analysts should proactively discuss the use of alternative treatments when collaborating on an interdisciplinary team. the approach to being proactive is likely to vary depending on the setting in which the practicing behavior analyst is working. for example, if a behavior analyst finds themselves working in a home or clinic setting, the process should start during client screening and no later than intake. at this time the behavior analyst should clearly articulate for the consumer: (a) how the parent/client values are incorporated at all stages of intervention, (b) the procedures that will be implemented, (c) the philosophy/rationale behind those procedures, (d) what constitutes an ebp and why it is important, (e) what procedures will not be implemented, (f) what happens if these procedures are implemented or suggested, and (g) how collaboration occurs within the agency. throughout this initial discussion meaningful and genuine rationales should be provided and any questions are answered. this discussion should continue to be revisited periodically throughout the course of the intervention. 192 january 2023, volume 15, issue 3, 187-197 many practicing behavior analysts work within school settings where the likelihood of collaborating on an interdisciplinary team is practically inevitable. although, the approach within this setting will differ from services that take place within a clinic or home setting, there are several steps the behavior analyst may take. first, when possible, seek out members of the interdisciplinary team prior to collaborating on any specific case. this provides an opportunity to develop rapport with other members of the team and could involve getting to know the team members, identifying their general ideas and philosophies, and how to best communicate with their team (e.g., how to provide feedback, how to bring up disagreements). second, communicate your values and approaches to intervention with the team members. this could be done in a manner like the approach within home and clinic settings described previously. finally, establish the nature of the collaborative relationship, outlining how to work together best, listen to each other’s recommendations, and how to best navigate professional discourse. similar to working in a home or clinic setting, these discussions should be revisited periodically to maintain rapport and address any changes since the initial, or previous, discussion. when all else fails, develop a reactive plan each of the previous guidelines were designed to proactively address recommendations or the implementation of alternative treatments. it is likely that even if these guidelines are used, members of an interdisciplinary team will recommend or implement alternative treatments. it is possible to be reactive in these situations while minimizing harm or risk for the client, adhering to the principles of science and behavior analysis, and remaining respectful of all members of the interdisciplinary team. first, it will be important to engage in active listening. psychologists and behavior analysts have recommended active listening for many years, which involves engaging in appropriate nonverbal behavior (e.g., facing the person speaking, making eye contact, appropriate facial expressions) and listening to the other person without interrupting (borck & fawcett, 1982). throughout the exchange, it will be important to engage in behavior that demonstrates listening and reflecting (e.g., head nods, reflective statements, verbal encouragements). when it is appropriate to engage in meaningful and productive dialogue (e.g., the other person has stopped talking), it will be important ask clarifying questions, seek out more information, and outline one’s values and perspective. this should be a discussion, but does not mean there may not be discourse. at some point in the discussion there will be agreement to implement or not implement the alternative treatment. if there is a recommendation for the implementation of an alternative treatment, then it will be necessary to conduct a risk benefit analysis. in this situation, a risk benefit analysis would involve identifying and comparing the relative risk of implementing the alternative treatment to its possible benefits. assessing possible benefits should involve, but is not limited to, identifying research supporting the effectiveness of the alternative treatment and client and consumer satisfaction and preferences. assessing possible risks should involve, but is not limited to, an analysis of the possibility of shortor long-term psychological or physical harm to the client (brodhead, 2015) and negative effects on the members of the interdisciplinary team. we strongly encourage practicing behavior analysts to include the possibility of time spent on an ineffective, but not harmful, alternative treatment as a possible risk. if it is determined that the risks outweigh the possible benefits the treatment should not be implemented, and it will be important to share this information with the interdisciplinary team, client, and caregivers in a clinically sensitive and responsive manner. if it is determined that the risks do not outweigh the possible benefits and the decision is made to implement the alternative treatment, there are at least two options available to the practicing behavior analyst. one option that has been previously recommended is to systematically evaluate the effects of the alternative treatment (e.g., normand, 2008). this recommendation has proven fruitful and has resulted in published evaluations of alternative treatments and methods (e.g., chok et al., 2010; lerman et al., 2008). however, great caution must be taken when systematically evaluating the effects of the alternative treatment in this way. in essence, the team would be conducting a mini experimental analysis and all the same procedural safeguards used within research to control for threats to internal and external validity should be considered. as such, this systematic analysis needs to go beyond simply taking baseline and intervention data and involve best practices in single subject research such as, but not limited to, establishing stable baseline responding, repeated baseline conditions, implementing the intervention without any other changes, repeated and/or staggered intervention conditions, and measures of treatment fidelity. a second option that has been previously recommended is to alter the alternative treatment to align with behavioral principles (kirby et al., 2022). this can be done by adding components of behavior analytic intervention to the alternative treatment (e.g., including preference assessments) or identifying if any components of the alternative treatment are conceptually aligned with aba. similar to the previous option, this option must be exercised with great caution for several reasons. first, the addition 193 the crossroads: interdisciplinary teams and alternative treatments / leaf, cihon, ferguson, milne & oppenheim-leaf of behavior analytic components may increase the perceived benefits and effectiveness of the alternative treatment. however, like any treatment package, it will remain unclear what is responsible for any behavior change (desired or undesired). as a result, time could be spent implementing ineffective or less effective components. second, if the alternative treatment has competing components, it could decrease the effectiveness of any added behavioral components. this could lead the team and consumer to conclude that otherwise effective behavioral interventions are not effective. third, as goldiamond (2002) importantly noted, “just because some sets of procedures can be analyzed in operant terms does not make them behavior modification procedures” (p. 149). nonetheless, conceptualizing an alternative treatment as behavior analytic may lead the team and consumer that the alternative treatment is, in fact, an aba-based intervention. navigating the crossroads: researchers behavior analytic researchers have a long history of critically and experimentally evaluating alternative treatments that are implemented with autistics/ individuals diagnosed with asd (e.g., chok et al., 2010; howard et al., 2005; leaf et al. 2018; lerman et al., 2008; normand, 2008). these critiques have included such alternative interventions as facilitated communication (montee et al., 1995), rapid prompting method (schlosser et al., 2019), social thinking™ (leaf et al., 2018), bonding (attachment) therapies (chaffin et al., 2006) and sensory integration (lang et al., 2012). recently, some have suggested that such evaluations are examples of disciplinary centrism, hubris, and “may stunt our science and the progression of our field” (kirby et al., 2022, p. 136). when statements are made in less formal settings (e.g., social media) to avoid these evaluations, it is concerning; however, such statements occurring in peer reviewed behavior analytic journals is alarming. regardless of the outlet, we could not disagree more. these evaluations have provided much needed critical and experimental evaluations of alternative treatments which can help save consumer resources and prevent harm for the clients we serve. we contend these evaluations are based on the scientific tenets of philosophic doubt, empiricism, and experimentation, not disciplinary centrism. suggesting otherwise minimizes the importance of the scientific method and ebps and may result in an increased adoption of harmful alternative treatments. the previously cited examples of critical and experimental evaluations of alternative treatments did not conclude the evaluated treatments were negative or ineffective simply because they were not conducted or developed by behavior analysts. no intervention should be discounted simply because it stems from a different field, practice, or researchers (kirby et al., 2022). there are numerous examples of the development of quality interventions and research evaluations stemming from the fields of speech language pathology (e.g., speech-language & audiology canada, 2018), occupational therapy (e.g., bodison & parham, 2018), psychology (e.g., sanders, 1999), and pediatrics (e.g., american academy of pediatrics, 1998) to name a few. research and interventions should be evaluated upon the universal principles and standards of science (previously described) and interventions should be selected or not selected based on these evolutions regardless of the origins of the interventions. for example, the quality of the research methodology (e.g., clear operational definitions, appropriate research methodology) used in social stories™ research has been evaluated numerous times (e.g., kokina & kern, 2010; leaf et al., 2015; rust & smith, 2006; sansosti et al., 2004). thus, warnings against the use of social stories™ (e.g., leaf et al., 2020) have been issued as a result of a lack of experimental evidence of effectiveness not the credentials of the individuals conducting the research. similarly, social thinking™ was evaluated to determine if the research met the definitions of an ebp or empirically supported treatment (e.g., leaf et al., 2018). social thinking™ research was found to not meet these definitions but did have many of the hallmarks of pseudoscience (leaf et al., 2016), and this would be the case regardless of the credentials of the individuals conducting the research. we contend that behavior analytic researchers have an obligation to the science and practice of behavior analysis to continue to critically and experimentally evaluate alternative treatments. this is not to diminish the value of other interventions, but, rather, to provide necessary information to inform best practices and maximize the effectiveness of the interventions for our clients. this information can also be invaluable in informing effective and compassionate collaboration within interdisciplinary teams. a hallmark of our profession is ensuring that all procedures implemented by our professionals are conceptually systematic, effective, and empirically supported; this is an aspect of alternative treatments that must continue to be discussed in the literature. conclusion there are hundreds, if not thousands, of interventions designed to help improve the symptomology associated with autism (jacobson et al., 2010), with the ultimate goal of improving the quality of life for autistics/individuals diagnosed with asd. many of the interventions available have no empirical evidence to support their effectiveness (e.g., rapid prompting method, son-rise), while others have studies in peer reviewed journals (e.g., social thinking™, relationship 194 january 2023, volume 15, issue 3, 187-197 development intervention) but include methodological flaws limiting the interpretation of the results (e.g., see leaf et al., 2018; milne et al., 2020). behavior analysts collaborating within interdisciplinary teams are likely to find themselves at difficult crossroads that include implementing these alternative treatments, defining and determining risk and harm, and evaluating research and interventions. these crossroads and the decisions that result have always been a relevant aspect of being a behavior analyst, especially when working within an interdisciplinary team. nonetheless, the challenges associated with collaborating within interdisciplinary teams are likely increasing with behavior analysts continuing to expand the field by working with different populations and within different contexts. ultimately, when navigating any crossroad that a behavior analyst might find themselves in, it is important to remember the moto “primum non nocere” (i.e., first, do no harm; normand, 2008). the guidelines and discussion provided here was done to demonstrate the possibility of navigating these crossroads while minimizing harm or risk for the client, adhering to the principles of science and behavior analysis, and remaining respectful of all members of the interdisciplinary team. that is, we can maintain the scientific tenets of philosophic doubt, empiricism, and experimentation while remaining humble and ensuring our clients access the most effective interventions available. footnotes 1the terms diagnosed with autism/asd, on the autism spectrum, individual with autism/asd, and autistic are used throughout this paper. the authors recognize that there are varied preferences and conventions related to personand identify-first language among the academic and autistic communities. the terminology selected for use in this paper is to be inclusive of varying preferences as well as grammar and stylistic needs and does not reflect a terminological intent. conflict of interest: this paper was submitted as part of a special issue that the first and second authors were asked to develop, solicit papers, and serve as editors. the paper was sent to a different editor and was sent for blind review. all authors currently or have provided behavioral intervention for autistics/ individuals diagnosed with autism. all authors have commercially available products related to applied behavior analysis and individuals diagnosed with autism. the first and fifth author own a company that provides behavioral intervention for individuals diagnosed with autism. references ala'i-rosales, s., cihon, j. h., currier, t. d. r., ferguson, j. l., leaf, j. b., leaf, r., mceachin, j., & weinkauf, s. m. (2019). the big four: functional assessment research informs preventative behavior analysis. behavior analysis in practice, 12(1), 222 234. https://doi.org/10.1007/s40617-018-00291-9 american academy of pediatrics. (1998). auditory integration training and facilitated communication for autism. pediatrics, 102(2), 431-433. association for science in autism treatment. (n.d.). learn more about specific treatments. https:// asatonline.org/for-parents/learn-more-aboutspecific-treatments/ autism new jersey. (n.d.). treatment. https:// www.autismnj.org /understanding-autism/ treatment/ baer, d. m., wolf, m. m., & risley, t. (1987). some stillcurrent dimensions of applied behavior analysis. journal of applied behavior analysis, 20(4), 313327. https://doi.org/10.1901/jaba.1987.20-313 baer, d. m., wolf, m. m., & risley, t. r. (1968). some current dimensions of applied behavior analysis. journal of applied behavior analysis, 1(1), 91-97. https://doi.org/10.1901/jaba.1968.1-91 behavior analyst certification board. (2020). ethics code for behavior analysts. https://bacb. com/wp-content/ethics-code-for-behavioranalysts/ bodison, s. c., & parham, l. d. (2018). specific sensory techniques and sensory environmental modifications for children and youth with sensory integration difficulties: a systematic review. american journal of occupational therapy, 72(1), 7201190040p72011900417201190040p7201190011. https://doi.org/10.5014/ ajot.2018.029413 borck, l. e. & fawcett, s. b. (1982). learning counseling and problem solving skills. new york: haworth press. bowman, k. s., suarez, v. d., & weiss, m. j. (2021). standards for interprofessional collaboration in the treatment of individuals with autism. behavior analysis in practice, 14(4), 1191-1208. https://doi.org/10.1007/s40617-021-00560-0 195 the crossroads: interdisciplinary teams and alternative treatments / leaf, cihon, ferguson, milne & oppenheim-leaf brodhead, m. t. (2015). maintaining professional relationships in an interdisciplinary setting: strategies for navigating nonbehavioral treatment recommendations for individuals with autism. behavior analysis in practice, 8(1), 70-78. https://doi.org/10.1007/s40617-015-0042-7 chaffin, m., hanson, r., saunders, b. e., nichols, t., barnett, d., zeanah, c., berliner, l., egeland, b., newman, e., lyon, t., letourneau, e., & millerperrin, c. (2006). report of the apsac task force on attachment therapy, reactive attachment disorder, and attachment problems. child maltreatment, 11(1), 76-89. https://doi. org/10.1177/1077559505283699 chok, j. t., reed, d. d., kennedy, a., & bird, f. l. (2010). a single-case experimental analysis of the effects of ambient prism lenses for an adolescent with developmental disabilities. behavior analysis in practice, 3(2), 42-51. https://doi.org/10.1007/ bf03391764 cox, d. j. (2012). from interdisciplinary to integrated care of the child with autism: the essential role for a code of ethics. journal of autism and developmental disorders, 42(12), 2729-2738. https://doi.org/10.1007/s10803-012-1530-z galloway, j., & sheridan, s. m. (1994). implementing scientific practices through case studies: examples using home-school interventions and consultation. journal of school psychology, 32(4), 385–413. https://doi.org/10.1016/00224405(94)90035-3 goldiamond, i. (2002). toward a constructional approach to social problems: ethical and constitutional issues raised by applied behavior analysis. behavior and social issues, 11(2), 108 197. https://doi.org/10.5210/bsi.v11i2.92 green, g. (1996). evaluating claims about treatments for autism. in c. maurice, g. green, & s. c. luce (eds), behavioral intervention for young children with autism: a manual for parents and professionals (pp. 15 28). pro-ed. hall, p. (2005). interprofessional teamwork: professional cultures as barriers. journal of interprofessional care, 19 suppl 1, 188-196. https://doi. org/10.1080/13561820500081745 hayes, l. j. (1996). listening with understanding and speaking with meaning. journal of the experimental analysis of behavior, 65(1), 282-283. https://doi.org/10.1901/jeab.1996.65-282 howard, j. s., sparkman, c. r., cohen, h. g., green, g., & stanislaw, h. (2005). a comparison of intensive behavior analytic and eclectic treatments for young children with autism. research in developmental disabilities, 26(4), 359-383. https://doi.org/10.1016/j.ridd.2004.09.005 international behavior analysis organization (2021). ibao ethical guidelines. https://theibao.com/ docs/ibao-ethical-guidelines-v100.pdf iwata, b. a., dorsey, m. f., slifer, k. j., bauman, k. e., & richman, g. s. (1982). toward a functional analysis of self-injury. analysis and intervention in developmental disabilities, 2(1), 3-20. https:// doi.org/10.1016/0270-4684(82)90003-9 jacobson, j. w., mulick, j. a., & foxx, r. m. (2010). controversial therapies for developmental disabilities: fad, fashion, and science in professional practice. routledge. kelly, a., & tincani, m. (2013). collaborative training and practice among applied behavior analysts who support individuals with autism spectrum disorder. education and training in autism and developmental disabilities, 48(1), 120-131. kirby, m. s., spencer, t. d., & spiker, s. t. (2022). humble behaviorism redux. behavior and social issues, 31(1), 133-158. https://doi.org/10.1007/s42822-02200092-4 kokina, a., & kern, l. (2010). social story interventions for students with autism spectrum disorders: a meta-analysis. journal of autism and developmental disorders, 40(7), 812-826. https:// doi.org/10.1007/s10803-009-0931-0 lafrance, d. l., weiss, m. j., kazemi, e., gerenser, j., & dobres, j. (2019). multidisciplinary teaming: enhancing collaboration through increased understanding. behavior analysis in practice, 12(3), 709-726. https://doi.org/10.1007/s40617-01900331-y lang, r., o’reilly, m., healy, o., rispoli, m., lydon, h., streusand, w., davis, t., kang, s., sigafoos, j., lancioni, g., didden, r., & giesbers, s. (2012). sensory integration therapy for autism spectrum disorders: a systematic review. research in autism spectrum disorders, 6(3), 1004-1018. https://doi.org/10.1016/j.rasd.2012.01.006 lawson, h. (2004). the logic of collaboration in education and the human services. jounral of interprofessional care, 18(3), 225-237. https://doi. org/10.1080/13561820410001731278 196 january 2023, volume 15, issue 3, 187-197 leaf, j. b., cihon, j. h., ferguson, j. l., milne, c. m., leaf, r., & mceachin, j. (2020). recommendations for behavior analysts regarding the implementation of social stories for individuals diagnosed with autism spectrum disorder. behavioral interventions, 35(4), 664-679. https:// doi.org/10.1002/bin.1736 leaf, j. b., cihon, j. h., ferguson, j. l., taubman, m., leaf, r., & mceachin, j. (2018). social thinking®, pseudoscientific, not empirically supported, and non-evidence based: a reply to crooke and winner. behavior analysis in practice, 11(4), 456-466. https://doi.org/10.1007/s40617-018-02410 leaf, j. b., kassardjian, a., oppenheim-leaf, m. l., cihon, j. h., taubman, m., leaf, r., & mceachin, j. (2016). social thinking®: science, pseudoscience, or antiscience? behavior analysis in practice, 9(2), 152-157. https://doi.org/10.1007/s40617-016-0108-1 leaf, j. b., leaf, j. a., milne, c., taubman, m., oppenheimleaf, m., torres, n., townley-cochran, d., leaf, r., mceachin, j., & yoder, p. (2017). an evaluation of a behaviorally based social skills group for individuals diagnosed with autism spectrum disorder. journal of autism and developmental disorders, 47(2), 243-259. https://doi.org/10.1007/ s10803-016-2949-4 leaf, j. b., oppenheim-leaf, m. l., leaf, r. b., taubman, m., mceachin, j., parker, t., waks, a. b., & mountjoy, t. (2015). what is the proof? a methodological review of studies that have utilized social stories. education and training in autism and developmental disabilities, 50(2), 127 141. lerman, d. c., sansbury, t., hovanetz, a., wolever, e., garcia, a., o'brien, e., & adedipe, h. (2008). using behavior analysis to examine the outcomes of unproven therapies: an evaluation of hyperbaric oxygen therapy for children with autism. behavior analysis in practice, 1(2), 50-58. https://doi.org/10.1007/bf03391728 marshall, k. b., bowman, k. s., tereshko, l., suarez, v. d., schreck, k. a., zane, t., & leaf, j. b. (2023). behavior analysts' use of treatments for individuals with autism: trends within the field. behavior analysis in practice. advance online publication. https://doi.org/10.1007/s40617-02300776-2 . milne, c. m., leaf, j. b., cihon, j. h., ferguson, j. l., mceachin, j., & leaf, r. (2020). what is the proof now? an updated methodological review of research on social stories. education and training in autism and developmental disabilities, 55(3), 264 276. montee, b. b., miltenberger, r. g., wittrock, d., watkins, n., rheinberger, a., & stackhaus, j. (1995). an experimental analysis of facilitated communication. journal of applied behavior analysis, 28(2), 189-200. https://doi.org/10.1901/ jaba.1995.28-189 national autism center. (2009). findings and conclusions: national standards project, phase 1. author. national autism center. (2015). findings and conclusions: national standards project, phase 2. author. normand, m. p. (2008). science, skepticism, and applied behavior analysis. behavior analysis in practice, 1(2), 42-49. https://doi.org/10.1007/bf03391727 progressive behavior analyst autism council (2022). code of ethics for certified progressive behavior analyst – autism professionals™. https://progressivebehavioranalyst.org/wpcontent/uploads/2022/06/2022_code-ofethics-booklet_updated-1-1.pdf rust, j., & smith, a. (2006). how should the effectiveness of social stories to modify the behaviour of children on the autistic spectrum be tested? lessons from the literature. autism, 10(2), 125-138. https://doi.org/10.1177/1362361306062019 sanders, m. r. (1999). triple p-positive parenting program: towards an empirically validated multilevel parenting and family support strategy for the prevention of behavior and emotional problems in children. clinical child and family psychology review, 2(2), 71-90. https://doi. org/10.1023/a:1021843613840 sansosti, f. j., powell-smith, k. a., & kincaid, d. (2014). a research synthesis of social story interventions for children with autism spectrum disorders. focus on autism and other developmental disabilities, 19(4), 194-204. https://doi.org/10.1177/ 10883576040190040101 schlinger, h. d. (2008). listening is behaving verbally. the behavior analyst, 31(2), 145-161. https://doi. org/10.1007/bf03392168 sidman, m. (1960). tactics of scientific research. basic books. speech-language & audiology canada. (2018). use of facilitated communication and rapid prompting method. https://www.sac-oac. ca/sites/default/files/resources/sac_official_ statement_on_facilitated_communication_ and_rapid_prompting_method_jan2018_en.pdf 197 the crossroads: interdisciplinary teams and alternative treatments / leaf, cihon, ferguson, milne & oppenheim-leaf schlosser, r. w., hemsley, b., shane, h., todd, j., lang, r., lilienfeld, s. o., trembath, d., mostert, m., fong, s., & odom, s. (2019). rapid prompting method and autism spectrum disorder: systematic review exposes lack of evidence. review journal of autism and developmental disorders, 6(4), 403412. https://doi.org/10.1007/s40489-019-00175-w slocum, t. a., detrich, r., wilczynski, s. m., spencer, t. d., lewis, t., & wolfe, k. (2014). the evidencebased practice of applied behavior analysis. the behavior analyst, 37(1), 41-56. https://doi. org/10.1007/s40614-014-0005-2 smith, t. (2013). what is evidence-based behavior analysis? the behavior analyst today, 36(1), 7-33. https://doi.org/10.1007/bf03392290 weiss, m.j., tereshko, l., bowman, k., marshall, k., rose, k. (2022). effective collaboration: maximizing outcomes in autism intervention in an interdisciplinary model. in j. b. leaf, j. h., cihon, j. l. ferguson, & m. j. weiss (eds.), handbook of applied behavior analysis interventions for autism. autism and child psychopathology series (pp. 125-149). springer, cham. https://doi. org/10.1007/978-3-030-96478-8_8 wolf, m. m. (1978). social validity: the case for subjective measurement or how applied behavior analysis is finding its heart. journal of applied behavior analysis, 11(2), 203-214. https://doi.org/10.1901/ jaba.1978.11-203 microsoft word 8_iejee_4_1_meyer_ray international electronic journal of elementary education, 2011, 4(1), 127-152. issn:1307-9298 copyright © iejee www.iejee.com structure strategy interventions: increasing reading comprehension of expository text bonnie j. f. meyer ∗∗∗∗ the pennsylvania state university, united states melissa n. ray the pennsylvania state university, united states abstract in this review of the literature we examine empirical studies designed to teach the structure strategy to increase reading comprehension of expository texts. first, we review the research that has served as a foundation for many of the studies examining the effects of text structure instruction. text structures generally can be grouped into six categories: comparison, problem-and-solution, causation, sequence, collection, and description. next, we provide a historical look at research of structure strategy interventions. strategy interventions employ modeling, practice, and feedback to teach students how to use text structure strategically and eventually automatically. finally, we review recent text structure interventions for elementary school students. we present similarities and differences among these studies and applications for instruction. our review of intervention research suggests that direct instruction, modeling, scaffolding, elaborated feedback, and adaptation of instruction to student performance are keys in teaching students to strategically use knowledge about text structure. keywords: text structure, structure strategy intervention introduction reading for understanding is vital for readers of all ages. the 2009 national assessment of educational progress (u. s. department of education national institute for education statistics, 2010) reported that 33% of 4th-grade students examined could not read at the basic level required to understand what they read. comprehension of expository text is critical for academic success in school (national educational goals panel, 1999). despite its importance, in comparison to narrative texts, students receive less exposure to expository ∗ bonnie j. f. meyer, department of educational and counseling psychology and special education, 204 cedar building, penn state, university park, pa 16802, united states. phone: 814-863-7501; fax: 814-863-1002. e-mail: bjm8@psu.edu international electronic journal of elementary education 128 texts in early elementary school (e.g., duke, 2000). this lack of exposure may place readers at a disadvantage because beginning in 4th-grade students increasingly are expected to learn from expository texts in language arts, science, and social studies (guthrie & davis, 2003). reading comprehension involves actively constructing new understandings by building relationships among the parts of text and between the text and one's pre-existing knowledge. good readers build coherent mental representations of what they read by understanding different text structures, generating inferences, monitoring their understanding, and using multiple strategies to construct meaning. use of text structure to understand how the important ideas of a text are inter-related increases readers’ meaning making. readers who use text structure can mentally examine how ideas in text are interrelated through the use of such relationships as sequence, comparison, causation, or problem and solution. these readers also may use external aids that show the top-level structure of a text to reduce memory demands. these aids include templates, text structure patterns, graphics, matrices, outlines, knowledge maps, or tree structures (e.g., meyer, young, & bartlett, 1989). over the last 40 years, meyer (e.g., 1971, 1975) and her colleagues (e.g., meyer & rice, 1982; meyer et al., 1989; meyer & poon, 2001; meyer et al., 2010) have studied text structures and readers’ abilities to use them (see table 1). readers who use a “structure strategy” seek to identify and use the author’s organization to organize their own understanding (meyer, brandt, and bluth, 1980). the structure strategy facilitates comprehension by helping the reader to organize concepts based on the explicit or implied relationships that are communicated by the text. the strategy promotes comprehension compatible with van den broek’s coherence-based processes in his simulation model of comprehension (van den broek, rapp, & kendeou, 2005). the causal connections of his model focus on the important cause-and-effect relationships that make up the logical structure of narrative text just like text structures build on each other to establish the logical structure for nonfiction text. text structures not only describe the text itself, but also characterize readers’ cognitive coherence representations (e.g., meyer & freedle, 1979, 1984; sanders & noordman, 2000). good readers use their knowledge of text structures to build coherent memory representations (meyer et al., 1980). signaling words (see table 1) can cue text structures and assist readers toward building coherent text representations. the key role of signaling words is in selection and encoding, particularly if readers have learned the structure strategy (e.g., meyer & poon, 2001). the power of teaching students the structure strategy is that it enables them to a) follow the logical structure of text to understand how an author organized and emphasized ideas; b) use processes parallel to these structures to increase their own learning and thinking (e.g., comparing, finding causal relationships, looking for solutions to block causes of problems); and c) use these text structures to organize their own writing, such as written summaries, recalls, and essays. in this paper we review empirical studies of interventions designed to teach the structure strategy in order to improve reading comprehension. first, we briefly discuss the basic research that served as a foundation for instruction about text structures. next, we provide a historical look at structure strategy interventions. in this historical overview, we examine both the history and advancements in studies of structure strategy instruction. finally, we review recent text structure interventions for elementary school students. for each intervention study included in this review, we examine the strategy instruction in terms of the number and types of text structures and/or signaling words taught, the methods of instruction, the rigor of the intervention design, and relevant findings. structure strategy interventions / meyer & ray 129 table 1. text structures with signaling comparison (compare/contrast) – relates ideas by differences and/or similarities; complexity can be increased by the number and detail of issues compared. the main ideas are organized to provide a comparison, contrast, or alternative view (e.g., political speech). instead, but, however, alternatively, whereas, on the other hand, while, compare, in comparison, in contrast, in opposition, not everyone, all but, have in common, similarities, share, resemble, the same as, just as, more than, longer than, less than, act like, look like, despite, although, difference, differentiate, different… problem-and-solution – relates responding ideas; complexity can be increased by the identification of causes of the problems and ways to reduce them. the main ideas are organized in two parts: the problem (or question) part and the solution (or answer) part, which responds to the problem part (e.g., popular science articles, medical information). problem: problem, trouble, difficulty, hazard, need to prevent, threat, danger, puzzle, can hurt, not good, bad… solution: to satisfy the problem, ways to reduce the problem, so solve these problems, protection from the problem, solution, in response, recommend, suggest, reply… cause-and-effect (causation) – relates ideas casually; complexity can be increased by embedded cause and effect paths and causal chains and reduced by similarity to familiar narratives. the main ideas are organized into cause and effect parts (e.g., directions, explanations, economic or science texts) cause, led to, bring about, originate, produce, make possible, owing to, by means of, accomplish by, since, due to, because, in order to, reasons, why, if/then, on account of, in explanation, effect, affects, so, as a result, consequence, thus, therefore, accordingly, for the purpose of… sequence (time ordered collection of events, ideas) – relates ideas via time. the main ideas are the steps or history presented (e.g., recipe steps, history books, biographies) later, afterwards, after, then, subsequently, as time passed, following, continuing on, to end, finally, year(s) ago, at the start of, first, second, third, 1, 2, 3…, next, primarily, early, before, to begin with, more recently, again, finally, the former, the latter, not long after, soon, now, today, after a short while, meanwhile, steps, stages, time line, history, sequence, development, look for a series of dates in histories… collection (listing, enumeration) – relates ideas simply by grouping them together; sometime the grouping is made explicit with enumeration. the main idea is the grouped list (e.g., “to do list,” botany). collection can be used with any of the other structures; sequence is a subtype of collection. for example, groups of solutions or causes are often presented. and, in addition, also, include, moreover, besides, first, second, third, etc., subsequent, furthermore, at the same time, another , and so forth… description (generalization, settings) – relates ideas by elaboration of attributes, specifics, or setting information. the main ideas is that aspects of a topic are presented (e.g., newspaper article) attributes of, characteristics are, for example, for instance, in describing, marks of, namely, properties of, qualities are, specifically, such as, that is… international electronic journal of elementary education 130 basic text structure research historical antecedents of many structure strategy interventions were basic laboratory studies about text structure and its effects on learning and memory (meyer & mcconkie, 1973; meyer, 1975; meyer & freedle, 1984; meyer & rice, 1982). meyer (1971) identified the hierarchical, logical structure of two articles from scientific american magazine and examined the relationship between the structure and what college students remembered from the text. effects of this logical structure were seen in a) the kinds of idea units that were remembered (more high level information than low level information; this effect came to be known as the "levels effect"); b) the stability of the idea units in consecutive recalls (stability related to the logical structure [.55], rather than rated importance [-.25]); and c) the tendency for clustering on this basis (if a particular idea unit was recalled, then the idea directly above it the logical structure was recalled 70% of the time; overall recall was 23% of ideas from a passage) (meyer, 1971). logical structure accounted for much of the variance that might ordinarily be attributed to other variables, such as serial position effects and rate importance. these findings suggested that the logical structure of a passage is related to certain aspects of the cognitive structure that the participants constructed. next, meyer (1974, 1975) combined the logical structure approach of the initial study with work by joseph grimes (1975) in linguistics. this provided methods for studying naturally occurring text and ways to control aspects of text structure and signaling for future experiments (see simplification of the approach in meyer, 1985). other text analysis methods at the time included work by crothers (1972) and frederiksen (1972, 1975). an important manipulation by meyer (1975) was the embedding of a causation (cause-and-effect) paragraph in the same serial position in two texts. in one text the paragraph was located at the top third of the hierarchical, logical text structure. in the other text, the paragraph was located at the bottom third of the text structure (see figure 1 for example of hierarchical, logical structure of a text). this manipulation was repeated with a text with the same structure but different content. also, versions of the set of four texts were prepared with and without signaling words that explicitly signaled the text structures so that patterns in recall of ideas could be compared as they varied in content, structure, or level in the logical structure. meyer found that the type and structure of relationships among concepts in text dramatically influenced comprehension when they occur at the top third of the structure. however, when the same pattern of relationships occurred low in the structure, they affected comprehension minimally (meyer, 1975). this was an important finding because it encouraged future text structure research to focus on the main ideas organized within a text rather than the details. due to this focus on top-level of text structures, meyer (1975) hypothesized that more organized text structures (causation, comparison, and problem-and-solution) would have greater mnemonic hooks for learning and memory than the structure of description (i.e., a collection of descriptions about a topic). meyer and freedle’s (1979, 1984) data supported this hypothesis, showing that for college learners, comparison and causation structures had greater benefits for recall in comparison to description. later work showed that the memory benefit for the more organized structure of comparison over a collection of descriptions held for adults who could use the structure strategy and who had high average and above vocabulary skills (meyer et al., 1989). however, for young, middle-aged, and older adults with average vocabularies and no training in the structure strategy, the collection of descriptions yielded better recall than the comparison structure (vincent, 1985). this effect of text structure has also been found with college level english as second language (esl) readers (carrell, 1984). structure strategy interventions / meyer & ray 131 by 1976, kintsch’s hierarchical, text base analysis was becoming popular. the structure developed by kintsch and colleagues (kintsch, kozminsky, streby, mckoon, & keenan, 1975) was based on argument repetition (i.e., repetition of words from the text), while meyer’s hierarchy was based on the semantic relations among ideas represented by text structures (see figure 1 and table 2). dunn (1978) stated, “unlike kintsch’s system, meyer’s system not only produces a hierarchical arrangement, but also states explicit inter-propositional relationships (including inter-paragraph relations among the items in text) to a much greater degree than do either kintsch and his associates (e.g., turner & green, 1977) or frederiksen…” (p. 10) in 1976 meyer directed her research efforts to the strengths of her approach; that is, representations of the top-level structures in text. several research efforts were launched to investigate the differences in use of the structure strategy among students with varying reading and vocabulary proficiencies and ways to increase use of the structure strategy by changing the text (e.g., signaling), the task (e.g., repeated readings of text with different content but the same text structure), or the reader (e.g., various levels of instruction). meyer, brandt, and bluth (1978, 1980) studied whether good and poor 9th-grade readers organized their recall with the same text structure as that used by the author. good and poor readers were identified by scores on a standardized reading comprehension test and corroborated by teacher appraisals. two basic strategies were expected from the 9th graders: the structure strategy or the default/list strategy. while the structure strategy involves systematic processing based on text structure, the default strategy is not systematic. the reader using the default strategy lacks focus and simply tries to remember some ideas from the text. the recall produced by such a reader is a list-like collection of descriptions about the topic with little to no attempt to interrelate the ideas. meyer et al. (1980) also examined the effects of signaling words for problem-and-solution and comparison text structures (see underlined words in table 2 for the problem-and-solution text). table 2. the supertanker text. (note: capitalized = author's message or main ideas; lowercase = major details; italics = minor details; underlined = signaling. in actual studies regular font was used for the text.) a problem of vital concern is prevention of oil spills from supertankers. a typical supertanker carries a half-million tons of oil and is the size of five football fields. a wrecked supertanker spills oil into the ocean; this oil kills animals, birds, and microscopic plant life. for example, when a tanker crashed off the coast of england, more than 200,000 dead seabirds washed ashore. oil spills also kill microscopic plant life which provide food for sea life and produces 70 percent of the world's oxygen supply. most wrecks result from the lack of power and steering equipment to handle emergencies, such as storms. supertankers have only one boiler to provide power and one propeller to steer the ship. the solution to the problem is not to immediately halt the use of tankers on the ocean since about 80 percent of the world's oil supply is carried by supertankers. instead, the solution lies in the training of officers of supertankers, better building of tankers, and installing ground control stations to guide tankers near shore. first, officers of the supertankers must get top training in how to run and maneuver their ships. second, tankers should be built with several propellers for extra control and backup boilers for emergency power. third, ground control stations should be installed at places where supertankers come close to shore. these stations would act like airplane control towers, guiding tankers along busy shipping lanes and through dangerous channels. source: meyer, b. j. f., brandt, d. m., & bluth, g. j. (1980). use of the top-level structure in text: key for reading comprehension of ninth-grade students. reading research quarterly, 16, 72–103. copyright @ 1980 by the international reading association (www.reading.org). international electronic journal of elementary education 132 for a subgroup of readers identified as comprehension “underachievers,” signaling was expected to be particularly helpful. students in this subgroup had better vocabulary (stanine score at least 4) than reading comprehension skills (one stanine below a student’s vocabulary score). while “underachievers” had vocabulary test scores closer to the good readers, reading comprehension scores of the underachievers were closer to the poor readers. they were identified as readers who could use the structure strategy, but would not without the explicit prodding provided by the signaling words. meyer et al. (1980) found that reading ability was associated with the number of ideas recalled, as well as the organization of recalls. readers classified as good readers recalled more ideas, and more frequently used the authors’ structure to organize their recalls. the researchers also found that text signaling influenced underachievers text processing. when reading text with signaling, as shown in table 2 and figure 1, these readers switched to use of the structure strategy instead of the default/list strategy. signaling did not affect the use of the structure strategy by good readers or poor readers. regardless of random assignment to the signaling or no signaling conditions, good readers recalled the top-level structure of the passage and used it to organize their recall, while most poor readers did not (meyer et al., 1980). this finding pointed to the importance of teaching signaling words as part of instruction in the structure strategy for students with poor reading comprehension skills. in this study, only 48% of the entire sample of 9th-grade students organized their recall with the same structure as the text on at least one of the problem-and-solution and comparison texts (meyer et al., 1980). from this text structure research, meyer and collaborators (meyer & freedle, 1979, 1984; meyer, 1984; meyer & rice, 1982) developed a processing model for getting text information into organized schemata for storage in memory based on text structures. first, readers determine whether they are interested in communicating with the writer of the text and following the writer’s thesis and rationale. if not, readers should use a different reading strategy rather than the structure strategy. next, readers select the structure strategy. if readers cannot use this strategy, the default/list strategy by default would be used. figure 2 depicts the processing steps in using the structure strategy. the end point in the model is a reader using the identified working schema, corresponding to a text structure, as an organizing framework to differentially select, encode, and organize ideas from the text into a long-term memory representation. structure strategy interventions figure 1. top-level structure of the supertanker text. from meyer, b. j. f., brandt, d. (1980). use of the top-level structure in text: key for reading comprehension of ninth reading research quarterly, 16, 72 structure strategy interventions / meyer & ray 133 level structure of the supertanker text. from meyer, b. j. f., brandt, d. level structure in text: key for reading comprehension of ninthreading research quarterly, 16, 72-103. copyright @ 1980 by the international reading association (www.reading.org). level structure of the supertanker text. from meyer, b. j. f., brandt, d. m., & bluth, g. j. -grade students. 103. copyright @ 1980 by the international reading association international electronic journal of elementary education 134 figure 2. model for getting text information into organized schemata for storage in memory. from meyer, b. j. f. (1984). text dimensions and cognitive processing. in h. mandl, n. stein & t. trabasso (eds.), learning and comprehension of text. hillsdale, nj: lawrence erlbaum associates. permission: learning and comprehension of text by mandl, heinz. copyright 1984 reproduced with permission of taylor & francis group llc books in the format journal via copyright clearance center. structure strategy interventions / meyer & ray 135 empirical studies of text structure and its effect on text processing and comprehension had several important implications for future intervention research. first, they provided evidence that text structure indeed exerted an influence on readers’ mental representations of texts. second, they suggested that readers vary in their ability to use the structure strategy, and this variability may be related to their overall reading ability and vocabulary. finally, the findings from meyer et al. (1978, 1980) suggested that many readers may be lacking in their knowledge of text structures. overall, this basic research provided evidence that many readers would likely benefit from explicit instruction in the use of text structure and encouraged studies which investigated methods of teaching readers to use them. early text structure interventions (1978-1990) in this section we discuss the early history of text structure interventions. these began in the 1970s with several doctoral dissertations or projects under meyer’s mentoring, which examined instruction designed to increase middle school to junior college students’ use of the structure strategy (e.g. brandt, 1978; jessen, 1981; meyer, bartlett, woods, 1978). bartlett’s dissertation (1978) was the first study to provide extended multiple sessions of explicit structure strategy instruction; he taught 9th-grade students meyer’s expository discourse types of problem/solution, comparison, causation, and collection of descriptions. all four structures were taught each day for five days of 1-hr instruction. bartlett’s approach asked students to find the main idea and then determine the text structure that organized the main idea. emphasis was not placed on signaling words or patterns/templates to write main ideas or recalls. explicit instruction of 6-steps to follow before, during, and after reading to use the top-level structure strategy were modeled and practiced. texts used as examples for instruction and practice increased in complexity across the five sessions. structure strategy instruction increased students’ ability to identify and use the text's toplevel structure and nearly doubled the amount of information remembered over students in the control condition who received the same texts, but with instruction about punctuation (bartlett, 1978). instruction effects appeared durable over an extended period for readers scoring above the 19th percentile on a standardized vocabulary test (bartlett, 1978). teachers reported performance advantages across the curriculum for students who had received training with the structure strategy rather than the punctuation instruction. armbruster also worked with meyer and her approach to text structure in the late 1970s and 80s. subsequently, armbruster, anderson, and ostertag (1987) taught the problem-andsolution structure to 5th-grade students in 11 days of direct instruction with social studies materials. one of two classrooms of students was randomly assigned to structure strategy training with the problem-and-solution structure and the other classroom to traditional instruction. students in both groups read 13 100 to 500-word texts taken from 4thand 5thgrade textbooks. the structure strategy group received problem-and-solution frames accompanied by blank lines for writing passage summaries. the frames were three boxes with an arrow from the problem box (something bad) pointing towards the action box (what people do attempting to solve the problem) and the results box (outcome of the action). a second arrow went from the action box to the results box. instructors modeled how to identify the text structure and how to use the problem-and-solution structure to write a summary, following principles of direct instruction (e.g., rosenshine & stevens, 1986). guidelines were provided for writing a summary of a problem-and-solution passage. these guidelines prompted students to write the problem, the solution, and the result of this solution, but did not emphasize problem and solution signaling words. students in the structure strategy class wrote 50% more main ideas on an essay exam and included more main ideas in written summaries of a text than the traditional class, but the classes did not international electronic journal of elementary education 136 differ on a short-answer fact test (armbruster et al., 1987). although the study’s experimental design was limited in rigor, it provided instruction with a 5th-grade classroom setting with authentic texts. in addition to these interventions, several early interventions investigated structure strategy interventions with adult learners. these studies covered a diverse group of learners including english language learners, college students, and adults learning within a community setting. carrell (1985) generally followed bartlett’s (1978) training materials with the same four text structures outlined by meyer (e.g., meyer & freedle, 1984). unlike bartlett, carrell instructed high-intermediate esl students from various language backgrounds enrolled in an intensive english program. one section of 14 students received structure strategy instruction, while the other section of 11 students read the same reading materials, but worked with linguistic operations, such as sentence combining, cohesion, and vocabulary. similar to bartlett, carrell found that the structure strategy group showed substantially and significantly higher performance on measures of reading comprehension than the control group after training as well as three weeks later. carrell’s study was the first to show that direct instruction with the structure strategy instruction increased reading comprehension of esl students. in their intervention, cook and mayer (1988) built on the work of meyer (meyer, 1975; meyer et al., 1980) by adding a number of important components to an instructional program which taught college students to use text structure to comprehend science texts. these components included a) a sorting task to measure text structure awareness, b) the application of text structure to passages taken from students’ chemistry textbooks, and c) specification of descriptive text structures that occur in science textbooks. cook and mayer conducted two studies about text structure. they added a novel component of a sorting task in which students sorted 20 texts taken from high school science textbooks according to five expository text structures. in the first study, half of a group of 32 undergraduate university students received a 5-page instructional pamphlet about five text structures used in science textbooks: generalization, enumeration, sequence, classification, and compare/contrast. cook and mayer presented a couple of signaling words per structure for the classification (i.e., “there are two types”), comparison (i.e., “in contrast to”), and sequence (“and then”) structures. the text structures classified by cook and mayer, particularly, generalization and classification, differ somewhat from meyer’s classification, but could be subsumed as subtypes of description. the students in the two groups worked at their own pace sorting texts into similar groups based on structure. the training group sorted the 4 texts per 5 text types correctly 79% of the time, while the no training group sorted them correctly 61% of the time. the second study conducted by cook and mayer (1988) involved training junior college students and provided instruction about three of the five text structures. students filled out three worksheets for the three trained text structures (generalization, enumeration, and sequence) using nine passages taken from their chemistry textbook. for example, the sequence worksheet had three steps: step 1 – identify the passage topic; step 2 – name each step in the sequence and outline the details of each; step 3 – say what varies from one step in the sequence to the next. another section of the chemistry class received no training and served as the control group. trained students increased their recall of the most important information and ability to answer application questions, but did not increase memory for facts. the results demonstrated the value of the structure strategy using an outlining format with description, listing, and sequence structures in the context of a science class. structure strategy interventions / meyer & ray 137 the research about knowledge-map (k-maps) by dansereau and colleagues (e.g., dansereau et al., 1979; holley, dansereau, mcdonald, garland, & collins, 1979) has similarities to the structure strategy. we will only briefly mention this large research literature (see o’donnell, dansereau, & hall, 2002). to make a k-map from an existing text the creator of the k-map needs to identify and use text structures. four of the links in k-maps are “part of,” “type of/example of,” “characteristic of,” and “evidence for;” these are subtypes of the description text structure. the other two links are “leads to” and “analogous to;” they correspond to causation and comparison, respectively. holley et al. (1979) provided college students with 5.5 hours of training over four sessions about these links applied to sentences, texts, and their own textbooks. students with the k-map training performed better than a control group of students on multiple measures of reading comprehension associated with understanding main ideas. geva’s (1983) flowcharting of expository text is a combination of representations of text by meyer et al. (1980) and holley et al. (1979). geva trained community college students to represent text in node-relation flowcharts. relations were represented by different types of lines in a flowchart matching a key with relations of elaboration, cause-effect, process, example, and detail as well as topic and conclusion. findings indicated that training with this approach led less skilled readers to more carefully read expository text and as result increase their reading comprehension. mayer (1999) considered k-maps as an effective type of structure strategy along with cook and mayer’s (1988) intervention and meyer et al.’s (1980) hierarchical text structures displayed in figure 1. in addition to academic settings, early research examined the efficacy of structure strategies in non-academic settings. meyer, young, and bartlett (1989) utilized some of the texts, feedback materials, and formats from bartlett’s training program in an instructional program designed for young (18 to 32 years old) and older (65 years and older) adults and primarily used everyday reading materials. instruction consisted of 7.5 hours of instruction (5 sessions) spread over two weeks instructors and peers (old and young adults) modeled the use of templates unique to each text structure in the composition of written main ideas and recall protocol. participants were instructed to “choose it (overall/top-level text structure), use it, or lose it.” the design of this study was the strongest of extant structure strategy intervention studies at the time. participants were randomly assigned to three groups: structure strategy instruction, a contact control that practiced reading and recalling the same texts that were used in the structure strategy instruction, but with no instruction about text structure, and a no contact, wait-list control group. alternate equivalent forms of measures were counterbalanced over a pretest, immediate post-test, and two-week delayed post-test. the instruction changed from bartlett’s earlier approach by incorporating use of signaling words to help identify text structures in simple advertisements to more complex materials (e.g., magazine articles). another major change from bartlett’s instruction (1978) was the addition of an initial step in which students were prompted to first search for the top-level structure of the text that could interrelate all the ideas in the text. this top-level structure would then lead to the main idea of the text. the main idea was identified as the ideas interrelated by the top-level structure. this step was added to assist the reader in constructing a coherent representation or situation model. some interesting findings regarding elements of effective instruction emerged from the meyer et al. (1989) study. there were minimal effects of training when students were simply told the definition of different text structures along with some of their signaling words and an example of each text type. more intensive instruction, including modeling how to use text structure strategically for understanding and remembering, appeared to be required to see strong effects of structure strategy training. often composition or reading textbooks simply international electronic journal of elementary education 138 mention different text structures; this is not sufficient for increasing reading comprehension. this brief treatment of text structures in such textbooks may relate to barriers in getting teachers to use the structure strategy in elementary and middle school classrooms. teachers may have encountered a cursory examination of text structure in college classes for composition or reading instruction, but did not see its value for increasing reading comprehension of their students. thus, they may discount structure strategy interventions as recommended by research or colleagues who have used the strategy. this discounting may be attributed to a lack of depth in understanding the structure strategy as well as a lack of materials needed for modeling and direct instruction about strategically using text structure to increase reading comprehension. some of the recent structure strategy interventions use intelligent tutors or scripted lessons for teachers, which may overcome some of these obstacles. overall, the early work on structure strategy instruction showed its potential for increasing reading comprehension. positive effects for using the structure strategy were noted from work with elementary school children to retired adults. most of the research involved 6th graders, 9th graders, high school students, college students (including junior college and esl students) and adults, rather than early elementary school children. most instruction programs involved modeling, practice, direct instruction, scaffolding, and multiple instructional sessions of increasing complexity of text materials. recent developments in structure strategy interventions text structure instruction across cultures and languages. more recent intervention research has explored the instruction of text structure with linguistically diverse populations. several studies have explored instruction of expository structure in languages other than english including french (raymond, 1993), spanish (leon & carretero, 1995), and dutch (broer, aarnouste, kieviet, & leeuwe, 2002). one of the first of these was raymond (1993) who used the instruction and texts from meyer et al. (1989) translated into french. as with the studies of bartlett (1978) and carrell (1985), raymond randomly assigned two intact classrooms to either structure strategy instruction or a control group that read the same materials as the strategy group, but without instruction about text structures. in raymond’s study the control students worked on answering questions about the texts. the structure strategy training was in french and presented to native english speakers with high-intermediate level skills in french. participants read articles for the pretest and post-test in french, but recalled them in english. the structure strategy group outperformed the control group on number of ideas recalled on the post-test. similarly, leon and carretero (1995) conducted an intervention to teach the structure strategy to high school students (ages 14 to 15 years) in spain, adapting the instruction of meyer et al. (1989) and a dependent measure from meyer (1984). they examined the effect of reading comprehension skills (good and poor readers), the text structure intervention in spanish, signaling, and time of post-testing. there were a number of interesting interactions among these variables. overall, direct instruction about the structure strategy improved reading comprehension over the control groups who read the same social studies materials, but without instruction in the structure strategy. additionally, the structure strategy instruction transferred to a text structure not studied in the intervention. in their study of dutch 6th graders, broer, aarnoutse, kieviet, and leeuwe (2002) provided structure strategy instruction for two text structures in a treatment they called the ‘making schematics’ strategy. the schematics were graphics for the causation structure and the classification structure (similar to cook and mayer’s [1988] enumeration and a subset of meyer and freedle’s [1984] collection of descriptions). for example, the schematic for structure strategy interventions / meyer & ray 139 causation was a table headed with a subject row for the topic of a text to be placed. the next row in the table was for signaling words (i.e., “cause,” “causes,”, “result,” and “led to”). the following row was divided into three parts: a column for cause(s), a box for a causal arrow, and a column for result(s). the next row was blank for students to write causes and effects found in a text. the final row was a place to write the main idea organized with causation. many aspects of the schematic are similar to classroom applications of the structure strategy (e.g., meyer, ireland, & ray, 2011). broer et al. (2002) used a non-equivalent control group design with multiple schools, 18 classes, and 354 6th-grade students from middle class homes in the netherlands. pairs of classes in a school were randomly assigned to the structure strategy or traditional dutch reading and answering questions approach. there were 16 instructional lessons taught by the classroom teachers who were trained in the strategy. students in the schematic/structure strategy condition increased their recognition of text structure, ability to make schematics, ability to formulate and deduce main ideas, and transfer of deducing main ideas from text with different top-level structures than those explicitly taught in the instruction. effect sizes between the structure strategy and control condition ranged from 0.15 (on a general reading comprehension test) to 0.26 (main idea text) to 0.78 (making schematics four weeks after instruction). the broer et al. (2002) study was an impressive applied study of structure strategy instruction delivered by classroom teachers at the upper level of elementary schools. all three of these studies showed the usefulness of the structure strategy across cultures. recently, schwartz and colleagues (mendoza & schwartz, 2011; yeh, schwartz, & baule, in press) conducted two interventions using the structure strategy via the training materials and testing materials of meyer et al. (1989) with bilingual college students. one new aspect of this research is the examination of the effects of the structure strategy instruction on eyemovement patterns. eye-tracing measures examined online processing during learning from texts. they indicate not only what a participant looks at but also how long the person’s gaze remains at a particular point and how the eye moves over the text. yeh, schwartz, and baule (in press) reported that after the text structure instruction recall of text information increased and eye-movement patterns changed. after instruction, english for speakers of other languages (esol) students made more fixations on the signaling words (e.g., “different” for the comparison text structure) and key areas of the text. the study demonstrated the effectiveness of the structure strategy for esol students. additionally, mendoza and schwartz (2011) taught bilingual college students from the utep and universidad autónoma de ciudad juárez the structure strategy. results showed that these spanish speakers also could learn the structure strategy in english. interestingly, most students were able to transfer their new knowledge about text structures and their english signaling words to parallel structures and signaling in their native (spanish) language. texts used as the dependent measures were written in english and spanish about topics relevant to prenatal screening. researchers who have examined the use of the structure strategy in diverse language contexts have included both elementary and college students. their research adds to the earlier investigations by showing that instruction of the structure strategy aids reading comprehension for bilingual students and students in various cultures. structure strategy, signaling, and transfer to everyday learning. in an extension of the earlier work by meyer et al. (1989), meyer and poon (2001, 2004) examined the interaction between structure strategy instruction and signaling in text. their instructional program, which targeted younger and older adults, provided more instructional emphasis on writing with international electronic journal of elementary education 140 the use of templates for writing main ideas or recalls with each text structure. these writing patterns have become important components of our recent work with children. for example, for the comparison structure the pattern for writing a main idea is located below. “the main idea was ______ and ______ (2 or more ideas) were compared on _____, ______, _____ (a number of issues).” the pattern for writing a recall from a text with a comparison top-level structure would include a) an introductory sentence with a comparison signaling word contrasting two ideas or political candidates; b) a paragraph or more about the first idea describing the issues for this idea or candidate; c) a transitioning signaling word, such as “in contrast,” as a new paragraph is started to describe the second idea or candidate on the same issues. meyer and poon also added a sixth instructional session due to needs of older adults (meyer, talbot, stubblefield, & poon, 1998); this session applied the structure strategy to note-taking, long, unedited magazine articles, medical decision-making, articles from the internet, and watching an informative video on nutrition. younger and older adults participated in nine hours of either structure strategy or interest strategy training (reading the same texts as the structure strategy group, but with a motivation strategy), or no training (waiting-list control group). participants also were randomly assigned to texts with or without signaling for pretests and post-test. both training groups reported positive changes in reading, but only the structure strategy group showed increased total recall from a variety of texts (d = 0.64), an informative video (d = 1.47), and information from the medical decision-making task (d = 0.93). the structure strategy intervention affected the organization of recall and was critical for producing readers who could use the structure strategy consistently across a variety of expository texts. the instruction also helped learners to use signals in text more effectively in order to employ the structure strategy across five passages consistently. when compared to the use of signaling without structure instruction, structure strategy training had a larger effect on reading comprehension. however, the relationship between strategy instruction and signaling was shown to be additive; structure strategy instruction plus signaling in texts produced more consistent use of the strategy across five texts. results also indicated that signaling effects the encoding processes rather than retrieval processes. this was the first study with the structure strategy to show transfer from structure strategy training with texts to multimedia learning and remembering medical information during a simulated decisionmaking scenario. moreover, this study, along with meyer et al. (1989) and meyer, talbot, poon, and johnson (2001) demonstrated that successful structure strategy instruction could be conducted in a variety of both formal and informal educational contexts. structure strategy interventions in elementary schools web-based structure strategy instruction. in their studies of adult learning meyer and colleagues frequently received feedback from adult participants who voiced their belief that the structure strategy instruction would be of particular use to their grandchildren coping with learning from texts in the school context. the first investigation to follow up on this suggestion (meyer et al., 2002) involved 5th-grade students tutored by older adults in the structure strategy via the internet. this invention became the first of many structure strategy interventions targeted for elementary school readers, as part of a renewed interest in investigating structure strategy instruction with younger readers. meyer et al. (2002) developed a web-based delivery of the structure strategy intervention for 5th-grade readers based on the program developed by meyer et al. (1989). in this intervention five text structures were introduced sequentially rather than all at once because previous work (meyer, poon, theodorou, talbot, & brezinski, 2000) had found that adults with slightly reduced working memory resources learned the structure strategy better with structure strategy interventions / meyer & ray 141 lessons introducing one or two text structures per training session versus all five at once. in meyer et al. (2002) the comparison and problem and solution structures were taught first followed by causation, sequence, and description. these earlier structure were then reviewed in later lessons and integrated implicitly and explicitly with other text structures (cause and effect, sequence, and description) throughout a set of 25 lessons. meyer et al. (2002) tested whether 5th-grade students can learn the structure strategy via the internet with feedback and support from their own personal human tutor. two approaches to teaching the structure strategy via the internet were examined. one approach involved an internet instructor (depicted with a static picture) and retired adults trained in the structure strategy as tutors (displayed with a picture of a tutor or animal representation). emails from internet tutors provided delayed feedback on students’ last lesson, encouragement, daily assignments, and additional instruction about the strategy with other examples, if necessary. the other approach only involved the internet instructor, who provided delayed feedback on student work, but no tutors. the students in the two structure strategy groups were compared to students in a control group who participated in extra sessions of the school's regular reading program. students were randomly assigned to three groups: structure strategy with tutors, structure strategy without tutors, and a control group. immediately after the intervention, the groups receiving structure strategy instruction tended to recall more information than the group with extra days of regular classroom reading. the average reader receiving structure strategy training had a total recall score equal to a reader in the control group who scored at the 77th percentile on the immediate posttest (effect size = 0.74). the superiority in total recall for the structure strategy group with tutoring over the control group in reading was clearly evident 2 1/2 months after the end of training. the average reader receiving structure strategy training with the aid of tutors had a total recall score equal to a reader in the control group who scored at the 81st percentile on the delayed posttest (effect size = 0.92). the structure strategy group with help from tutors tended to make more progress in mastering the strategy than the group without tutors. this was particularly the case for students whose messages from tutors focused on providing feedback about the structure strategy and the subject matter of the lessons rather than off-task socializing. fifth graders in the structure strategy group with tutors made significantly greater gains in self-efficacy than students in the other two groups. most students in both structure strategy groups made progress in learning the structure strategy, although few consistently demonstrated mastery of the strategy after training. those students who mastered the strategy were particularly diligent in their work in the lessons. one english as a second language learner (ell) was conscientious in completing her lessons and following her particularly skilled tutor's instructions; she made outstanding gains in reading performance. prior to the intervention she scored at the 28th percentile on a standardized reading comprehension test and did not use the structure strategy. after the intervention, however, she showed mastery of the strategy and scored at the 68th percentile on the standardized reading test. equivalent texts about different content (supertankers or killer bees) with the same text structure (depicted in figure 1) were counterbalanced over the pretest and post-test. her pretest and immediate posttest recall of problem-and-solution texts are listed below (meyer, 2003): pretest: “this passage is about oil spills. the oil spills on the ocean and poisens them. when the oil spills it kills animals too and, poisens them. i can only remember something about 3 football fields.” international electronic journal of elementary education 142 posttest: “the problem is prevention of killer bees. bees make honey 150 pound per year. they reproduce quickly in warmer climates. they don’t live under 59 decreas. some of them escaped from africa and came to s.a. brazil. if their nest is desterbed they will sting. one man was riding his horse in brazil and the bees came up and started stinging him and his horse. he fell from his horse and survived, but his horse died because of all the stings from the bees. bees can not see red, that’s why bee keepers wear red when working with bees. a lot of bee strikes can kill a person. mostly they live up to north carolina. dust can calm the bees. on way scientists teach the people of brazil is don’t desterb their nests and run from killer bees. scientists can’t stop all the killer bees” (meyer, 2003). this student recalled nine ideas on the pretest about supertanker text shown in table 2. in contrast to the nine ideas recalled on the pretest, she recalled 88 ideas on the post-test about the problem of killer bees (meyer, 2003). most of the students (70%) in the control group did not organize their ideas with a problem part and a solution part. most completely missed any of the suggested solutions, but this student has both a problem part, signaled with "problem," and a solution part, signaled with "one way." her last three sentences are an attempt to recall the solution part of the passage. there was considerably more about the solution than produced by fifth-grade students in the control group. in this study we found that a problem with cause(s) and a solution designed to eliminate/reduce the cause(s) was particularly difficult for 5th-grade students. one important finding from this investigation that was modified in subsequent webbased structure strategy instructional programs was that 5th-grade students’ had difficulty working with a table of signaling words in a pop-up help screen while the text was presented on another screen. a solution to this problem was an instructional aid in the form of laminated keys for each text structure. each key contained a list of signaling words, an example on the topic of whales, the template/pattern for writing a main idea with the text structure, and a template for writing a recall with the structure. when a text structure was initially introduced to the student, it was added to a key ring available for consultation during the lessons (see meyer, wijekumar, & lin [2011] for a picture of the problem-andsolution key). one of the challenges that remained in extending this program of research from a paperbased to an electronic medium was the quality of the delivery of the system. therefore, in more recent investigations, modifications in programming (e.g., use of a pedagogical animated agent to teach the structure strategy) have been explored. in order to increase the accessibility and quality of the delivery of the structure strategy instruction, we developed a web-based system called intelligent tutoring of the structure strategy (itss) to teach the structure strategy to 5thand 7th-grade students (meyer & wijekumar, 2007). in several studies the researchers have examined the effectiveness of itss with 5thand 7th-grade students. unlike the previous web-based program, itss used a web-based delivery system in which several instructional design features could be adapted including: feedback (immediate vs. delayed, minimal vs. elaborated), topic choice (whether or not students could choose text topics for practice lessons), and individualization (the extent of individualization of texts to match students’ prior lesson performance). meyer et al. (2010) tested whether 5thand 7th-grade students could learn the structure strategy via itss in six months of training for 90 minutes a week spread over two or three days. we examined different feedback and motivation conditions in delivering itss with a 2x2 pretest post-test design comparing: a) type of tutor immediate feedback (minimal feedback of "good," & "try again," versus substantial and specific feedback) from the i.t. (intelligent tutor, the animated agent); and b) motivational condition (programmed structure strategy interventions / meyer & ray 143 sequence of practice examples vs. student choice of practice examples). in the elaborated feedback version, the animated agent provided elaborated feedback with scaffolding to improve performance on subsequent trials, while the other feedback version involved the animated agent providing only simple feedback about the correctness of a student’s response. for example, in response to the same student’s performance, the tutor in the elaborated feedback version said, “your structure, main idea, and details are correct. great job! but your signaling words were incorrect. using your signaling chart (key) as your guide, rewrite the signaling words.” on a third deficient trial, students in the elaborated feedback group were given a model response to correct their writing of main ideas. this type of elaborated feedback was not provided for students in the simple feedback condition. students were stratified on reading comprehension and then randomly assigned to the conditions (including conditions for counterbalanced testing materials over time of testing). choice of practice texts affected performance as students completed instruction on the first structure learned – comparison, but had no effect on post-test performances. number of ideas recalled on this formative evaluation was greater for the choice group, but competency in using the structure strategy was not. type of tutor feedback affected the ability to identify issues compared when students wrote main ideas in a formative evaluation of the comparison structure, but not the experimenter-designed post-test materials. below is the pretest recall for a below-average reader in the 5th grade. pretest (for an article comparing monkeys): “the monkeys are the smalls monkeys weghy less 4 onces a few in. tall.” formative post-test (after 10 itss lessons about the comparison structure): “there are 2 different kinds of bats. a black flying fox bat and a leaf-nosed bat. the black flying fox bat is one of the bigest, they grow up to 6 feet wide and weigh more than 3 pounds. they are jet black. leaf-noised is smaller than the black flying bat. the leaf-noise bat is only 1 foot wide. the leaf-noise bats come in different (colors) and mostely feeds on masquitoes and moths.” data from a standardized reading comprehension test (gray silent reading test [gsrt]; wiederholt & blalock, 2000) clearly showed that below-grade-level readers made substantial gains from pretest to post-test with all versions of itss, and effect sizes ranged from 0.42 to 1.16. all ability levels of readers made greater gains on the gsrt if they received elaborated feedback. students who received itss with elaborated feedback showed more improvement (d = 0.55) than students who received itss with simple feedback (d = 0.15). according to the gsrt manual, the average performance of the simple feedback group corresponds to a percentile rank of 79, while the average performance of the elaborated feedback group corresponds to a percentile rank of 91 (wiederholt & blalock, 2000). substantial effect sizes also were found from pretest to post-test on various measures of reading comprehension, such as recall and strategy competence (d = .39 to .79, meyer et al., 2010). students also demonstrated maintenance of performance over summer break on most measures. for example, there was complete maintenance of the ability to use comparative signaling words 4 months after itss instruction. in another design feature study with itss (meyer, wijekumar, & lin, 2011), a more individually tailored version was developed to provide remediation or enrichment lessons to better match the needs of 5th-grade readers. stratified random assignment was employed to compare the effects of two itss versions. fifth-grade students in the more individualized condition made greater improvements from pretest to posttest on the standardized reading comprehension test (d = 0.55) than students in the standard condition (d = 0.30). international electronic journal of elementary education 144 additionally, students receiving more individualized instruction demonstrated higher mastery achievement goals when working in itss lessons than students receiving the standard instruction (d = 0.53). also, 5th-grade students receiving more individualized instruction showed greater improvement using signaling, better work in lessons, and more positive posttest attitudes toward computers than students receiving standard instruction. students in both conditions improved their recall of ideas from texts, use of the structure strategy, and understanding of comparison signaling words. more recently, 4thand 5th-grade classrooms have participated in efficacy trials with itss conducted in rural and suburban school districts. for this work wijekumar, meyer, and lei (2011) are using a multi-site cluster randomized trial (crt) design to increase statistical power in testing treatment effects. each school district serves as a site in the multi-site crt design. within each site or school district, classrooms were randomly assigned to itss or a control condition involving the usual language arts curriculum. the study is a replication of past work with 5th-grade students and an extension to 4th-grade students. a modification was made in the instruction to reduce the burden of typing for the younger students. once students identified two or more topics compared and the issues on which they were compared, they completed a matrix task in which they clicked on text ideas relevant to cells in the matrix, rather than typing the information this procedure to reduce typing was used for both 4thand 5th-grade students in the first five out of nine practice lessons for the comparison structure. in addition, during implementation some 4th-grade teachers voiced concern about typing demands. as a result, 4th graders, but not 5th graders, were switched to only constructing main idea statements and not recalling all they could remember from the texts in the itss lessons. preliminary results show a significant interaction between grade level and treatment on the post-test with 5th-grade students making more progress. however, 4th graders show significant effects on most measures of reading comprehension but not for details on the comparison text. for the more difficult problem-and-solution text, 4th graders performed significantly better on all measures. this probably results from the fact that without training about the more complex problem with cause and solution, most upper elementary students completely miss the solution part. however, the comparison text contrasted two penguins on a number of the same attributes, many of which were memorable, such as orange ear patches. fourth graders in control classrooms apparently remembered such isolated details as well 4th graders in the structure strategy classrooms. in a pretest/post-test design, struggling readers in 4th and 5th grades received special group instruction with itss (meyer & wijekumar, 2011). the reading teacher who administered itss explained that his students, who had the greatest problems in reading, showed a 20 to 70 point gain on the state language arts assessment between 2010 and 2011, which he attributed to the immediate feedback in their structure strategy instruction from i.t. in itss. interestingly, the teacher believed that his 5th-grade students gained more than his 4th-grade students, who seemed to struggle more with itss. the data showed statistically significant increases in reading comprehension on a standardized reading comprehension text from pretest to post-test, and no statistically significant time (pretest vs. post-test) by grade interaction (4th vs. 5th). one grade level did not make significantly greater progress than the other. in fact, effect sizes between pretest and post-test reading test scores were .78 and .79, respectively for 4thand 5th-grade students. additionally, performance on a signaling test, which tested the ability to use comparative signaling words, also showed statistically significant effects between pretest and post-test and no time by grade interaction. the effect sizes on the signaling test were 1.42 for 4th graders and 1.05 for 5th graders. the fact that 4th-grade students started at a lower level than the 5th-grade students may have led to more difficulty, frustration, or complaining, but they still benefited structure strategy interventions / meyer & ray 145 as much as the 5th-grade students. nearly all (80%) of the 4th graders in this study used the regular 5th-grade itss lessons (constructing recalls) rather the modified 4th-grade lessons (without constructing recalls) used in the itss efficacy trial. itss lessons were originally designed for 5th-grade students via matching of standards and curriculum at the 5th-grade level. perhaps itss is best suited to the 5th-grade level in terms of content and readiness for learning about text structure. however, itss covers a wide variety of content in science, social studies, history, and some sports topics in order to promote transfer to reading many types of nonfiction texts so interest in itss topics should not be confined to 5th grade. additionally, itss may be appropriate for 4th graders because struggling 4th-grade readers had performance gains as large as struggling 5th-grade readers’ after using itss in a pretest/post-test design (meyer & wijekumar, 2011). the greater effects for 5th-grade students in the efficacy trial may have resulted from the better match to the needs of 5th graders or to an ill-advised decision to delete the requirement of creating a recall protocol from the itss lessons for 4th grade. the efficacy data cannot differentiate between these two explanations due to different versions of itss for the two grade levels. currently, another efficacy trial is underway with 7thand 8th-grade students as part of a grant awarded by the u.s. department of education institute of education sciences. the study with 7th-grade students is a replication study of earlier work (meyer et al., 2010), but also examines effects in both rural and suburban schools. work with 8th-grade students extends our lessons to older middle school students. in preparation for this extension we wrote lessons to meet pennsylvania standards in social studies, science, and writing for 8thgrade students. series of two or three related lessons on a topic using multiple texts with complex structures were added to the compilation of over 100 itss lessons as we extended web-based structure strategy to 8th grade. classroom-based interventions in the primary grades. in addition to web-based instructional programs there has also been renewed interest in teaching text structure in the early grades during the elementary school and preschool years (e.g., culatta, hall-kenyon, black, 2010; hall, sabey, & mcclellan, 2005; hall-kenyon, & black, 2010; williams et al., 2005; williams et al. 2007; williams, stafford, lauer, hall, & pollini, 2009). for example, culatta, hall-keyon, and black (2010) reported a pretest post-test pilot study project with 71 children ages 4 to 5 years old in four preschool classrooms. the study was a collaborative project of speech-language pathologist and early childhood educators using playful, but systematic instruction in theme-based units over 16 weeks with instruction four days each week. teachers adapted the unit that was initially co-planned by university researchers (culatta and hall-kenyon). exit data from participating teachers indicated that the teachers had learned to value expository texts and explicit instruction about them. the intervention focused on two expository text structures: comparison and problem-and-solution. meyer (e.g., meyer & freedle, 1984) had originally thought that the problem-and-solution structure would be particularly easy for readers due to its similarity to narratives, but learned that was not the case (e.g., meyer et al., 1989; 2010; meyer, 2003) when problem-andsolution texts involved identifying and reducing the cause of problems. for example, when adults were asked to identify the most difficult of five text structures to learn and use with the structure strategy, problem-and-solution was most frequently listed. with readers across the life span, use of the structure strategy with scientific text organized with a problem (& cause) and solution (blocking/reducing of the cause) is usually more difficult than text organized with a comparison structure (meyer et al., 1989). also, meyer (2003) reported that over 70% of 5th graders showed no understanding about using the problem-and-solution structure after reading a newspaper article of this type. overall, research with 4th graders to international electronic journal of elementary education 146 retired adults suggests that the problem-and-solution texts appear particularly challenging when used in scientific exposition with a problem and its cause(s) and a solution that was aimed at eliminating or reducing the cause(s). without the underlying causal relationships, this responding structure (question–answer; problem–solution) can be easy to learn, and this appears to be the tact taken by culatta et al. (2010). a teacher presented a problem in a narrative about a son’s escaping hamster to preschool children. then, the children constructed hamster cages. next, the problem of hamsters escaping from cages and cages to solve the problem were discussed in an expository framework. except for bartlett’s (bartlett, 1985; 1989; 2010; bartlett & meyer, 1981) continued interest in the structure strategy with preschool and elementary school children, until recently there was a dearth of studies with children younger than 5th grade. williams, hall, and lauer (2004) reviewed the literature on awareness of text structure and interventions with text structure, particularly in elementary school grade levels. they noted the lack of intervention studies to teach the structure strategy with students in early grades until recently. they mentioned that a few studies were conducted in the 1980s, but interest in the topic waned without leading to any movements of magnitude in applying research findings to the classroom. however, williams and her students’ research represents major strides in bringing the structure strategy to 2nd grade (e.g., williams et al., 2005; williams et al., 2007; williams, et al., 2009) as well as preschool children (e.g., culatta et al., 2010). williams et al. (2005, 2007) examined direct instruction with the comparison structure (compare-contrast) with second grade students. her work combines the influences of meyer (e.g., meyer et al., 1980; meyer & freedle, 1984; meyer et al., 2002), armbruster et al. (1987), and her own prior work with narratives, strategies, concept learning, and reading with learning disabled students (e.g., gersten, fuchs, williams, & baker, 2001). in williams et al. (2005, 2007) classrooms (128 students in all) were randomly assigned to structure strategy instruction with the comparison text structure, traditional instruction which contained the same content but no structure strategy instruction, or a no instruction control. signaling words were taught (called clue words); eight clue words from williams et al. (2005) were “both,” “however,” “and,” “alike,” “compare,” “but,” “than,” and “contrast.” note that “and” appears to be a confusing comparison signaling word in that it usually indicates the collection of two things together. content aligned with the new york state curriculum standards in both language arts and the content areas. williams’ teacher-led instruction with at-risk primary school children used clue words (signaling words), general text structure focused questions (what is this paragraph about? how are they the same? how are they different?), and graphic organizers (e.g., matrix). in addition, the intervention included text analysis (the close analysis of short pieces of well-structured text) and paragraphs that embody the characteristics of a particular text structure. the goal of using well-structured text was to increase familiarity with structure in order to help children strengthen their mental representation of a specific structure. these target paragraphs became more complex across the lessons in the intervention. at first, all the sentences reflect the structure, and later, other sentences (called distractors) are added that include details about the topic but do not reflect the structure. in the second lesson, lions and eagles were contrasted on one issue, skin covering, in a one-paragraph text that also included a similarity (williams et al., 2005). this contrasts to itss where the introductory comparison lessons involved twoparagraph texts that compare two ideas on three contrasted issues. sequentially moving from one contrasted issue per lesson in early lessons to more contrasted issues in later lessons may be particularly important for at-risk learners and younger readers. in williams et al.’s first intervention study (2005) students learned how to classify animals on the basis of issues, such as how they bear their young or get oxygen. students in the structure strategy interventions / meyer & ray 147 structure strategy group as well as students in the control group learned this content about classifying animals. adding structure strategy instruction did not reduce learning of vocabulary or other content area concepts. both groups that received content instruction performed better on the content than the no content control group. children in the structure strategy group also demonstrated use of the comparison structure with both similar content (animals) as well as new content, indicating that readers were able to transfer strategy knowledge to new contexts (williams et al., 2005). williams, stafford, lauer, hall, and pollini (2009) replicated these findings in an extended and improved version focused on science topics (e.g., animals) and the comparison text structure. the clue words were reduced from eight to six, and “and” and “than” were deleted from the original list. additionally, later lessons added a pro-con version of the comparison structure and added some pro-con clue words. as students progress through instructional modules, they become more challenging. students answer comprehension questions with oral and written responses in early lessons. later they write summaries with assistance of a form, and by the end of the module they write free summaries without assistance. again, primary-grade children demonstrated greater learning of the comparison (compare/contrast) structure and improved their ability to understand novel text (williams et al. 2009). additionally, they learned domain knowledge similar to students who received content instruction without time devoted to text structure (williams et al. 2009). in addition to work with the comparison text structure, researchers have investigated structure strategy instruction with causative text in the primary grades. in a series of lessons developed and evaluated by williams et al. (2007), the cause and effect structure was taught in the domain of social studies. the lessons focused on the colonists and early american pioneers. the cause-effect general questions were “what happened?” and “why did that happen?” the clue words were “because,” “so,” “therefore”, and “since.” similar effects were reported as those reported about the comparison lessons and science topics in terms of boosts to reading comprehension and domain learning similar to students who did not learn about text structure. transfer was observed, but it appeared to be not as strong as that for the comparison lessons (williams et al. 2007). in summary, recent interventions with elementary and pre-school students have indicated that the structure strategy can be successfully taught to younger learners, and is associated with improvements in comprehension similar to previous interventions targeted to adults. although questions remain as to the best method of web-based instruction with readers in earlier grades, from these interventions several important implications for instruction can be gleaned. first, it is important to provide appropriate scaffolding and instructive feedback to students, increasing the complexity of text and instruction as students improve in their use of the structure strategy. in addition, it is important to select texts which match the reading level of the learner, particularly in initial instruction of a particular text structure. intervention research also suggests that not only can the structure strategy be taught within a classroom setting; doing so will provide comprehension benefits while not detracting away from content learning. finally, in adapting structure strategy instruction to younger learners, careful thought of readers’ needs and capabilities is needed in order to avoid creating instructional tasks that are too difficult or confusing for young readers to complete. conclusions much progress has been made in the rigor and extent of research examining the effects of the structure strategy with different types of readers in different contexts. there is substantial and consistent evidence over 30 years that instruction with the structure strategy international electronic journal of elementary education 148 increases recall from expository text and the organization and quality of readers’ recalls. additionally, there is evidence that structure strategy instruction can increase understanding and use of signaling words, production of good main ideas and summaries, standardized reading comprehension tests scores, and answers to questions. additionally, structure strategy instruction changes the type of ideas readers underline as important, readers’ thinkaloud protocols, and their eye movement patterns while reading. the momentum to bring the structure strategy to elementary school children is currently in full force in part due to funding from the u. s. department of education institute of education sciences for the work of meyer and williams and their colleagues in recent years. it is hoped that this momentum continues with positive effects on readers from preschool to retirement age. in order for this to happen in k-12 schools, teachers and administrators need to understand the strategy and its importance for reading and learning from text. although the structure strategy can be provided through a web-based platform, many students will not acquire the benefits of structure strategy instruction without a school environment valuing and supporting the intervention. there are still many questions yet to answer about the structure strategy. these include how much instruction to provide with each text structure at different age levels and with what types of texts. additional research in adapting instruction to meet the needs of children throughout the elementary school years is needed in order to help readers meet the demands of progressively difficult texts in a variety of different domains. care should be taken so that children learning about text structures in preschool or primary grades realize that they have started on a path of increasing their knowledge and use of the structure strategy with increasingly more varied and complex reading materials and tasks. this tact may avoid giving younger readers the notion that they already know about the structure strategy and need not engage in further instruction. as can be noted in the intervention by samuels et al. (1988), college students benefited from instruction about the text structure of a scientific journal article. this benefit for reading comprehension held when a journal was presented in the ideal organized manner and when it was not. as readers age, their instructional needs change, and it is important to create structure strategy instruction which is sensitive to the needs of readers in particular phases of their education. potential areas for study include the timing and amount of structure strategy review throughout years of schooling and extensions to meet the developing needs of the maturing reader. this further research will allow educators to meet the needs of the maturing reader who will interact with more varied and complex informational materials in nonfiction texts and on the web. • • • acknowledgements the research reported here was supported in part by the institute of education sciences, u.s. department of education, through grants r305g030072 and r305a080133 to the pennsylvania state university. the opinions expressed are those of the authors and do not represent views of the institute or the u.s. department of education. bonnie j. f. meyer, professor of educational psychology at the pennsylvania state university, studies reading comprehension, decision-making, text structure, and the structure strategy. she has over 50 publications about the structure strategy and its effects on understanding and applications for reading or writing. her current research, funded by the institute for education sciences of the u.s. department of education, is focused on structure strategy instruction for students in grades 4-8. she has served on editorial boards for discourse processes, the journal of educational psychology, the structure strategy interventions / meyer & ray 149 reading research quarterly, cognition and instruction, the educational psychologist, and the journal of literacy research and is a fellow in the american psychological association, american educational research association, and american psychological society. melissa n. ray is a doctoral candidate in educational psychology at the pennsylvania state university. her primary research interests are reading comprehension, literacy development, and literacy instruction. references armbruster, b. b., anderson, t. h., & ostertag, j. (1987). does text structure/summarization instruction facilitate learning from expository text? reading research quarterly, 22, 331-346. bartlett, b. j. (1978). top-level structure as an organizational strategy for recall of classroom text. (unpublished doctoral dissertation). arizona state university, tempe. bartlett, b. j. (1985). organisational structure: the key to improved comprehension and memory. in l. unsworth, (ed.): reading: an australian perspective. melbourne: nelson 1985, pp. 83-102. bartlett, b. j. (1989). children writing, reading and thinking – and top level structuring. unpublished manuscript. bartlett, b. j. (2010). learning about written language, literacy and meaning: a metalinguistic gift . in raich, m., schober, p., & zelger, j. (2010). gabek iv. sprachliche strukturen, theorie und anwendungen. innsbruck: studienverlag. bartlett, b. j., & meyer, b. j. f. (1981). a special plan for reading. mt. gravatt press, brisbane, australia. berkowitz, s. j. (1986). effects of instruction in text organization on sixth grade-students’ memory of expository reading. reading research quarterly, 21(2), 161-178. brandt, d. m. (1978). prior knowledge of the author's schema and the comprehension of prose. (unpublished doctoral dissertation). arizona state university, tempe. broer, n. a., aarnoutse, c. a. j., kieviet, f. k., & leeuwe, j. f. j. (2002). the effect of instructing the structural aspect of texts. educational studies, 28, 213238. carrell, p. l. (1984). the effects of rhetorical organization on esl readers. tesol quarterly, 17(4), 441469. carrell, p. l. (1985). facilitating esl reading by teaching text structure. tesol quarterly, 19(4), 727-752. cook, l. k., & mayer, r. e. (1988). teaching readers about the structure of scientific text. journal of educational psychology, 80, 448-456. doi:10.1037/0022-0663.80.4.448 crothers, e. j. (1972). memory structure and the recall of discourse. in r. o. freedle & j. b. carroll (eds.), language comprehension and the acquisition of knowledge (pp. 247–283). new york: wiley. culatta, b., hall-kenyon, k., & black, s. (2010). teaching expository comprehension skills in early childhood classrooms. topics in language disorders, 30, 323-338. dansereau, d. f., collins, k. w., mcdonald, b. a. holley, c. d., garland, j.c., diekhoff, g., et al. (1979). development and evaluation of an effective learning strategy program. journal of educational psychology, 71, 64-73. duke, n. k. (2000). 3.6 minutes per day: the scarcity of informational texts in the first grade. reading research quarterly, 35, 202–224. dunn, b. r. (1978, march). the use of selected text grammars for describing recall from prose. paper presented at the annual meeting of the southeastern psychological association. frederiksen, c. h. (1972). effects of task-induced cognitive operations on comprehension and memory processes. in r. o. freedle & j. b. carroll (eds.), language comprehension and the acquisition of knowledge (pp. 211–246). new york: wiley. frederiksen, c. h. (1975). representing logical and semantic structures of knowledge acquired from discourse. cognitive psychology, 7, 371-458 gersten, r., fuchs, l. s., williams, j. p., baker, s. (2001). teaching reading comprehension strategies to students with learning disabilities: a review of research. review of educational research, 71(2), 279-320. geva, esther. 1983. facilitating reading through flowcharting. reading research quarterly 18:384-405. grimes, j. e. (1975). the thread of discourse. the hague, the netherlands: mouton. guthrie, j. t., & davis, m. h. (2003). motivating struggling readers in middle school international electronic journal of elementary education 150 through an engagement model of classroom practice. reading & writing quarterly, 19, 59-85. hall, k. m., sabey, b. l., & mcclellan, m. (2005). expository text comprehension: helping primary-grade teachers use expository texts to full advantage. reading psychology, 26, 211-234. hall-kenyon, k., & black, s. (2010). learning from expository texts: classroom-based strategies for promoting comprehension and content knowledge in the elementary grades. topics in language disorders, 30, 339-339-349. holley, c. d, dansereau, d. f., mcdonald, b. a., garland, j. c., & collins, k. w. (1979). evaluation of hierarchical mapping techniques as an aid to prose processing. contemporary educational psychology, 4, 227-237. jessen, j. (1981). effects of reading comprehension on training delinquent adolescents how to identify the top-level structure in text. (unpublished master’s thesis), arizona state university, tempe. kintsch, w., kozminsky, e., streby, w. j., mckoon, g., & keenan , j.m. (1975). comprehension and recall of text as a function of content variables. journal of verbal learning and verbal behavior, 14, 196-214. león, j. a., & carretero, m. (1995). intervention in comprehension and memory strategies: knowledge and use of text structure. learning and instruction, 5, 203-220. mendoza, l., & schwarz, a. i. (2011). improving comprehension strategies for reading health-related texts in a hispanic second language speakers of english population. unpublished manuscript. meyer, b. j. f. (1971). idea units recalled from prose in relation to their position in the logical structure, importance, stability, and order in the passage (unpublished master’s thesis). cornell university, ithaca, new york. meyer, b. j. f. (1974). the organization of prose and its effects on memory (dissertation). cornell university, ithaca, new york. meyer, b. j. f. (1975). the organization of prose and its effects on memory. amsterdam: north-holland. meyer, b. j. f. (1984). text dimensions and cognitive processing. in h. mandl, n. stein & t. trabasso (eds.), learning and comprehension of text. hillsdale, nj: lawrence erlbaum associates. meyer, b. j. f. (1985). prose analysis: purposes, procedures, and problems. in b. k. britton, & j. black (eds.), analyzing and understanding expository text (pp. 11-64, 269 304). hillsdale, nj: erlbaum. meyer, b. j. f. (2003). text coherence and readability. topics in language disorders, 23, 204-224. meyer, b. j. f., bartlett, b. j., & woods, v. (1978). facilitative effects of passage structure on recall. arizona state university, unpublished paper. meyer, b.j.f., brant, d.h., & bluth, g.j. (1978, april). use of author's textual schema: key for ninth graders' comprehension. paper presented at the annual meeting of the american educational research association, toronto. meyer, b. j. f., brandt, d. m., & bluth, g. j. (1980). use of the top-level structure in text: key for reading comprehension of ninth-grade students. reading research quarterly, 16, 72-103. meyer, b. j. f., & freedle, r.o. (1979). effects of discourse type on recall (prose learning series research report no. 6). tempe, ariz.: arizona state university. meyer, b. j. f., & freedle, r. o. (1984). effects of discourse type on recall. american educational research journal, 21, 121-143. meyer, b. j. f., ireland, j. j., & ray, m. n. (2011). connecting reading and writing: applying the structure strategy to persuasive texts. manuscript submitted for publication. meyer, b. j. f., & mcconkie, g. w. (1973). what is recalled after hearing a passage? journal of educational psychology, 65, 109-117. meyer, b. j. f., middlemiss, w., theodorou, e., brezinski, k. l., mcdougall, j., & bartlett, b. j. (2002). effects of structure strategy instruction delivered to fifth-grade children using the internet with and without the aid of older adult tutors. journal of educational psychology, 94, 486-519. meyer, b. j. f., & poon, l. w. (2001). effects of structure strategy training and signaling on recall of text. journal of educational psychology, 93, 141-159. meyer, b. j. f. & poon, l.w. (2004) effects of structure strategy training and signaling on recall of text. in robert b. ruddell and norman j. unrau (eds.) theoretical models and processes of reading, fifth edition, pp. 810-851. newark, delaware: international reading association. meyer, b. j. f., poon, l. w., theodorou, e., talbot, a., & & brezinski, k. l. (2000, april). effects of adapting reading instruction to the individual differences of older adults. paper presented at cognitive aging conference, atlanta, ga. structure strategy interventions / meyer & ray 151 meyer, b. j. f., & rice, g. e. (1982). the interaction of reader strategies and the organization of text. text, interdisciplinary journal for the study of discourse, 2, 155-192. meyer, b. j. f., talbot, a. p., poon, l. w., & johnson, m. m. (2001). effects of structure strategy instruction on text recall of older african american adults. in j. l. harris, a. kamhi, & k. pollock (eds.), literacy in african americans communities (pp. 233-263). mahwah, nj: erlbaum. meyer, b. j. f., talbot, a. p., stubblefield, r. a., & poon, l. w. (1998). interest and strategies of young and old readers differentially interact with characteristics of texts. educational gerontology, 24, 747-771. meyer, b. j. f., & wijekumar, k. (2007). a web-based tutoring system for the structure strategy: theoretical background, design, and findings. in d. s. mcnamara (ed.), reading comprehension strategies: theories, interventions, and technologies (pp. 347-375). mahwah, new jersey: lawrence erlbaum associates. meyer, b. j. f., wijekumar, k., middlemiss, w., higley, k., lei, p., meier, c., & spielvogel, j. (2010). webbased tutoring of the structure strategy with or without elaborated feedback or choice for fifthand seventh-grade readers. reading research quarterly, 45, 62-92. meyer, b.j.f., wijekumar, k.k., lin, y. (2011). individualizing a web-based structure strategy intervention for fifth graders’ comprehension of nonfiction. journal of educational psychology, 103, 140-168. meyer, b. j. f., & wijekumar, k. k. (2011). struggling 4th and 5th grade readers using itss. manuscript in preparation. meyer, b. j. f., young, c. j., & bartlett, b. j. (1989). memory improved: enhanced reading comprehension and memory across the life span through strategic text structure. hillsdale, nj: lawrence erlbaum. national educational goals panel (1999). reading achievement state by state, 1999. washington, dc: u.s. government printing office. o’donnell, a. m., dansereau, d. f., & hall, r. h. (2002). knowledge maps as scaffolds for cognitive processing. educational psychology review, 14, 71-96. raymond, p. m. (1993). the effects of structure strategy training on recall of expository prose for university students reading french as a second language. the modern language journal, 77, 445-458. rosenshine, b., & stevens, r. (1986). teaching functions. in m. c. wittrock (ed.), handbook of research on teaching (3rd ed., pp. 376-391). new york: macmillan. sanders, t. j. m., & noordman, l. g. m. (2000). the role of coherence relations and their linguistic markers in text processing. discourse processes, 29, 37-60. samuels, s.j., tennyson, r., sax, l., mulcahy, p., schermer, n., & hajovy, h. (1988). adults' use of text structure in the recall of a scientific journal article. journal of educational research, 81, 171174. u.s. department of education, institute of education sciences, national institute for education statistics. (2010, march). the nation’s report card: reading 2009. retrieved from http://nces.ed.gov/nationsreportcard/pubs/main2009/2010458.asp van den broek, p., rapp, d. n., & kendeou, p. (2005). integrating memory-based and constructionist processes in accounts of reading comprehension. discourse processes, 39, 299-316. vincent, j.p. (1985). effects of discourse types on memory of prose by young, middle-age, and old adults with average vocabularies (unpublished doctoral dissertation), arizona state university, tempe, arizona. wiederholt, j. l, & blalock, g. (2000). gray silent reading tests (gsrt). austin, tx: pro-ed. wijekumar, k., meyer, b.j.f., lei, p. (2011). multisite randomized controlled trial examining intelligent tutoring of the structure strategy. manuscript in preparation. williams, j.p., hall, k.m. & lauer, k.d. (2004). teaching expository text structure to young at-risk learners: building the basics of comprehension instruction. exceptionality, 2004, 12, 129-144. williams, j. p., hall, k. m., lauer, k. d., stafford, k. b., desisto, l. a., & decani, j. s. (2005). expository text comprehension in the primary grade classroom. journal of educational psychology, 97, 538550. international electronic journal of elementary education 152 williams, j. p., nubla-kung, a. m., pollini, s., stafford, k. b., garcia, a., & snyder, a. e. (2007). teaching cause-effect text structure through social studies content to at-risk second graders. journal of learning disabilities. special issue: teaching history--in all its splendid messiness--to students with ld: contemporary research, 40, 111-120. williams, j. p., stafford, k. b., lauer, k. d., hall, k. m., & pollini, s. (2009). embedding reading comprehension training in content-area instruction. journal of educational psychology, 101, 120. yeh, l., schwartz, a. i., & baule, a. l. (in press). the impact of text-structure strategy instruction on the text recall and eye-movement patterns of second language english readers. psycholinguistic research on bilingualism and reading. copyright © www.iejee.com issn: 1307-9298 © 2022 published by kura education & publishing. this is an open access article under the cc by ncnd license. (https://creativecommons.org/ licenses/by/4.0/) 145 international electronic journal of elementary education december 2022, volume 15, issue 2, 145-159 use of an ebook for oral health literacy© curriculum to elicit functional health knowledge, decision making, and goal setting among school-aged children valerie a. ubbesa,*, sophia whiteselb abstract introduction an ebook for oral health literacy© curriculum was used as a brief intervention to help school-aged children use their functional health literacy skills of reading, writing, and speaking to learn about oral health hygiene and the importance of choosing healthy food and beverages for their teeth. the curriculum focused on building functional knowledge about oral health hygiene and nutrition behaviors through a health literacy intervention that highlighted the need for reasoned actions when setting food goals for healthy teeth and making decisions to keep teeth healthy and strong. the aims of the study included: 1) determining what children learned about their teeth from two chapters of the ebook for oral health literacy© curriculum and what they remembered from reading the oral health and nutrition stories; 2) determining whether children liked to read and what they liked about the words and the pictures of the stories, and; 3) determining whether children brushed their teeth in the morning and at night; had any cavities; had teeth or mouth pain; and visited the dentist every six months for a dental checkup. the visual textual narrative helped children to build a vocabulary about oral health when reading and writing about health. children wrote about words and phrases that they recalled after reading each chapter. their elaborations exemplified the importance of using a constructivist theoretical framework to elicit children’s functional health knowledge about oral health and nutrition. chi-square results showed that there was a moderate significant correlation between children who reported brushing their teeth in the morning and having been to the dentist in the past year (p = .021). childhood dental caries are a preventable global issue that affects more than 600 million children annually (tinanoff et al., 2019; ramos-gomez et al., 2020). preventive approaches for tooth infections (e.g., dental caries) have included health education, regular dental checkups, better nutrition (sharma et al., 2022), and school sealant programs keywords: oral health literacy, nutrition, health education, curriculum, functional health knowledge, functional health literacy received : 20 november 2022 revised : 26 december 2022 accepted : 28 december 2022 doi : 10.26822/iejee.2023.286 a,* corresponding author: valerie a. ubbes, department of kinesiology, nutrition, and health miami university, oxford, ohio, usa. e-mail: ubbesva@miamioh.edu orcid: https://orcid.org/0000-0003-0459-5198 b sophia whitesel, department of kinesiology, nutrition, and health, miami university, oxford, ohio, usa. e-mail: whitessl@miamioh.edu orcid: https://orcid.org/0000-0002-8957-1330 146 december 2022, volume 15, issue 2, 145-159 (griffin et al., 2017; griffin, et al., 2016; griffin et al., 2014). however, dental caries and gum inflammation do not only affect the mouth. a recent oral health report from the national institute of dental and craniofacial research (nidcr, 2021) indicated that more than 50 systemic diseases begin in the mouth and contribute to lifelong chronic disease. oral health promotion and disease prevention interventions are necessary across the lifespan because the prevalence of disease increases with age (nidcr, 2021). oral health education lack of resources such as access to a toothbrush, toothpaste, floss, or dental checkups are critical issues that have not been adequately addressed for children in the u.s. (brown et al., 2006). to date, public health plans to reduce dental caries in children include advocating for fluoridated water in community water supplies, imposing a sugar tax on high-caloric foods, increasing health literacy initiatives, implementing more dental services in school-based health clinics (arenson et al., 2019), and changing the age of the first dental visit to one-year old to help children start school healthier (american academy of pediatrics, 2003). because education predicts health literacy by 18% with health literacy serving as a mediator between socioeconomic status and health (lastrucci et al., 2019), oral health education programs are essential in schools to give children of all socioeconomic backgrounds affordances to maximize their health and wellbeing. results from the 2016 school health policies and practices study (cdc, 2017) indicated that 57.7% of school districts had adopted a policy stating that their elementary schools will teach oral health. as a result, health education curricula need to take a skillbased approach through decision making and goal setting in order to educate children on the oral health behaviors they can practice every day to become healthy (ubbes et al., 2020a; ubbes & witter, 2021). for school-aged children, proper dental hygiene and nutrition education can support cognitive development, school readiness, self-esteem, and a longer quality of life (ramos-gomez et al., 2020). midstream influencers of children are elementary teachers and parents who can help to identify the underlying reasons for poor school attendance from dental pain (jackson, et al., 2011). for example, missing school contributes to poor academic attendance by children (eklund et al., 2022), which can affect academic performance in the form of overall test scores. and unfortunately, tooth pain can reduce children’s ability to focus on classroom learning and homework completion when they are in school (ruff et al., 2019). untreated dental caries can cause pain and infection and ultimately lead to problems in eating, speaking, and learning (nihcd, 2021). health literacy health literacy has been defined as functional, interactive, and critical (nutbeam, 2000) with more recent recommendations to “define health literacy in the context of quality school health education” and to “support school health education as a strategic avenue to achieve health literacy” (videto & dake, 2019). the institute of medicine’s (iom) report entitled health literacy: a prescription to end confusion stated that the “most effective means to improve health literacy is to ensure that education about health is a part of the curriculum at all levels of education” (nielsen-bohlman, panzer, & kindig, 2004, p. 149). when considering health literacy for early childhood, the health literacy of a child’s parents (or primary caregivers) is considered to be the most important because low parent and caregiver health literacy are linked to poorer health-related knowledge, behaviors, and health outcomes of their children (morrison et al., 2019). velardo and drummond (2017) have advocated for child centric approaches in health literacy research and “the importance of working with children” (p. 9). this includes the need for school-based health literacy (p. 10) to include easily accessible health information that can be understood by younger age groups. there has also been a call to action on “understanding children’s needs by including children as active participants in research….with the goal of understanding how children construe their social worlds” (p. 9) and how “children make sense of health messages” (p. 7). theoretical framework a constructivist theoretical framework in health education aims to teach concepts that focus on patterns and relationships in health-related information while encouraging students to take an active collaborative role in their learning by making meaning and constructing their own understanding and inferences about health (ubbes et al., 2009). children learn about health through the influence of their families and peers in a social context (vygotsky, 1978). making meaning of concepts related to health promotion and disease prevention requires the practice of functional health literacy skills (e.g., reading, writing, speaking) in order to build functional health knowledge at a foundational level in health education. tappe et al. (2009) define functional knowledge as specific concepts and information that students need to know in order to engage in healthy behaviors. more recent descriptions of functional health knowledge includes “valid and reliable information and concepts that support health beliefs, skills, and behaviors” (national consensus for school health education, 2022, p. 20). 147 use of an ebook for oral health literacy© curriculum / ubbes & whitesel oral health literacy curricula are available to encourage elementary children to brush their teeth with fluoridated toothpaste twice a day, floss their teeth to reduce gum inflammation, eat healthy foods and beverages, and get a dental checkup every six months while building their functional health literacy skills (ubbes et al., 2020a; ubbes & witter, 2021). if children do not brush their teeth and practice poor oral hygiene habits, plaque will build up on the surface of the tooth and cause the tooth to decay as evident by changes in oral bacterial flora (butera et al., 2022). purpose of the study the ebook for oral health literacy© curriculum was implemented with elementary children to teach them how to set goals and make decisions about their oral health hygiene and nutrition. the main goals of the study were: 1) determining what children learned about their teeth from two chapters of the ebook for oral health literacy© curriculum and what they remembered from reading the oral health and nutrition stories; 2) determining whether they liked to read and what they liked about the words and the pictures of the stories; and 3) determining whether children brushed their teeth in the morning and at night; had any cavities; had mouth pain; and visited the dentist every six months for a dental checkup. method participants participants (n = 39) included a convenience sample of second grade (n = 21) and third grade (n = 18) students from a city school district in the midwestern united states. children were from intact classrooms in two different elementary schools from the same school district. gender breakdowns for the participants (n = 39) included 19 girls, 18 boys, and 2 did not specify their gender. second grade students (n = 21) read chapter 6 of the curriculum called “decisions to keep my teeth healthy and strong”. third grade students (n = 18) read chapter 6 and also read chapter 5 of the curriculum called “setting food goals for healthy teeth”. procedures this pilot study assessed second and third grade children (n = 39) on their oral health vocabulary recall after reading and listening to the narration of two chapters of the e-book for oral health literacy© curriculum, specifically from theme 2 entitled “oral health and nutrition of children”. classroom instruction at the elementary schools was led by a faculty member from the local university, who was a registered dietitian and instructor of a senior-level nutrition course. chapters 5 and 6 were read by the elementary students who advanced the slides at their own pace using their computer mouse. each chapter took one minute to read and then the children were asked to recall and write what they had learned from the e-book chapters through an open-ended conversational process. third graders (n = 18) read chapters 5 and 6 of the curriculum whereas second graders (n = 21) read only chapter 6 of the curriculum. students responded to the survey questions that are explained below. survey questions there were two types of surveys for the elementary children. children responded to four open-ended questions: what is something that you learned about your teeth today? what is something you remember about the pictures in the book? what is something you remember about the words in the book? what did you like about the book that you read? children also responded to six multiple choice questions: (1) did you brush your teeth this morning? (yes, i did!; no, i did not.); (2) have you been to the dentist in the last year? (yes; no); (3) how many cavities (or fillings) do you have in your teeth? (open ended response); (4) have you ever missed school because your teeth or mouth hurt? (yes; no); (5) do you like to read? (yes, all of the time!; sometimes; no, not really); and (6) gender. curriculum materials the ebook for oral health literacy© is a theory-based intervention that encourages students to read about decision-making and goal-setting messages for improving their oral health and nutrition behaviors in tandem. the curriculum and supportive materials are found on the digital literacy partnership website at miami university (ubbes & miami university libraries, 2022). the ebook for oral health literacy© has been previously evaluated for its readability, suitability, understandability, actionability, and gist-based message design (ubbes, et al., 2020) including eye tracking feedback from three audiences (ubbes, et al., 2018c). the primary prevention curriculum with five themes was designed and targeted directly to children in order to shape their health-related beliefs, skills and, behaviors regarding oral health and nutrition practices in their everyday life. overall, 17 chapters include visual-textual-gestural storylines that demonstrate healthy actions for brushing, rinsing, flossing, and going to the dentist for a 6-month checkup, including eating nutrient-dense foods and beverages. for the purposes of this study, only chapters 5 and 6 of the curriculum were selected as a brief intervention in the elementary classrooms. chapters 5 and 6 represented one curriculum theme with two synergistic health behaviors: oral health and nutrition. figure 1 shows the cover page of chapter 5, entitled “setting food goals for healthy teeth”. chapter 5 encourages children to improve the health of their teeth by setting fresh food goals every day. 148 december 2022, volume 15, issue 2, 145-159 the following polysyllabic vocabulary words are integrated into 10 different sentences on 10 slides: yogurt, fibrous, fiber, remove, sugar, eating, visit, dentist, healthy, toothpaste, buying, evening, watching, brushes, improve, healthy, calcium, and grocery. main generalizations and phrases of the chapter include: set a food goal to improve the health of your teeth by eating healthy meals. eat yogurt because calcium helps your teeth to grow strong. eat fibrous fresh foods to remove plaque and sugar from your teeth. brush your teeth after snacks and meals. figure 1. cover page of chapter 5 “setting food goals for healthy teeth” figure 2 shows the cover page of chapter 6, entitled “decisions to keep my teeth healthy and strong”. chapter 6 focuses on food decisions for healthy and strong teeth. the following polysyllabic vocabulary words are integrated into 10 sentences on 10 slides: healthy, eating, drinking, morning, apples, candy, water, soda, sugar, decisions, vegetables, broccoli, calcium, nutritious, beverages, and family. main generalizations and phrases include: decide to keep your teeth healthy and strong. eat your vegetables like broccoli because broccoli contains calcium for teeth. drink your milk at lunch and at home. eat apples instead of candy. drink more water instead of sugary soda because water can rinse and clean your teeth and gums. figure 2. cover page of chapter 6: "decisions to keep my teeth health and strong" data analysis data analysis was conducted using spss version 25 (ibm corporation). significance was set at p ≥ 0.05. for the chapter 5 multiple choice questions, shapirowilk normality tests were performed to evaluate the type of correlation needed. spearman rank order correlations were run because the data were skewed and thus, a non-parametric statistic was needed. the non-parametric chi-square statistic, was used to analyze group differences because the dependent variable was measured at a nominal level (mchugh, 2013). for the chapter 5 multiple-choice questions, several statistical tests were run, including frequencies, shapiro-wilk normality, spearman rank order correlations, and a chi-square test. chapter 6 involved open-ended written responses and elaborations and did not require statistics. results six multiple choice questions formed the basis for our second and third research goals. frequency results for the question, “did you brush your teeth this morning?”, 15 students selected, “yes, i did!” and 3 selected “no, i did not”. for the question, “have you been to the dentist in the last year?”, 17 said “yes” and 1 selected “no”. a test of shapiro-wilk normality for all six variables was run from the chapter 5 multiple choice questions (n = 18). the 6 variables were: did you brush your teeth this morning?; have you been to the dentist in the past year?; gender; have you ever missed school because your teeth hurt?; how many cavities (or fillings) do you have in your teeth?; and do you like to read? one of the reasons that the data were not normally distributed was because it is difficult to have high variability when the number of participants were less than fifty. the normality test was run in order to determine the proper correlation test that should be used for further testing of the data. if the data were normally distributed, a pearson product moment correlation would have been run; however, the data were skewed, so a nonparametric spearman correlation was run. all the variables were considered not normally distributed as the significance values were all less than 0.05. according to a source, “the shapiro–wilk test is a more appropriate method for small sample sizes (<50 samples) although it can also be used on larger sample size while kolmogorov– smirnov test is used for n ≥50” (mishra et al., 2019, p. 70). table 1 shows the results of the spearman rank order correlations for chapter 5 multiple choice questions with six variables. the two questions, “did you brush your teeth this morning?” and “have you been to the dentist in the last year?” were significant (p = 0.020). the correlation coefficient was 0.542 which shows that there was a moderate, positive relationship between the two variables, r(16)= 0.542, p = 0.020, because the higher the correlation coefficient, the stronger 149 use of an ebook for oral health literacy© curriculum / ubbes & whitesel the relationship. no other values had significant correlations because their p-value was ≥ 0.05. table 2 represents chi-square results for the variables, did you brush teeth in the morning? and have you been to the dentist in the last year? the pearson chisquare test was run in order to test if the variables were associated with each other. results for the pearson chi-square test indicated a 5.294 value with a two-sided p-value of 0.021. because the p-value is less than the significance level of p<0.05, the relationship between these two variables was significant. in other words, morning teeth brushing and going to the dentist in the last year were associated with each other. table 1. spearman rank order correlations for chapter 5 multiple choice questions multiple choice questions did you brush your teeth this morning? have you been to the dentist in the last year? how many cavities (or fillings) do you have in your teeth? do you like to read? have you ever missed school because your teeth hurt? gender did you brush your teeth this morning? correlation coefficient 1.000 .542* .052 -.123 .108 -.100 significance (2-tailed) .020 .839 .626 .668 .693 have you been to the dentist in the last year? correlation coefficient .542* 1.000 -.168 -.301 .059 -.217 significance (2-tailed) .020 .504 .224 .817 .387 how many cavities (or fillings) do you have in your teeth? correlation coefficient .052 -.168 1.000 -.070 .168 .155 significance (2-tailed) .839 .504 .781 .504 .539 do you like to read? correlation coefficient -.123 -.301 -.070 1.000 -.351 .278 significance (2-tailed) .626 .224 .781 .153 .264 have you ever missed school because your teeth hurt? correlation coefficient .108 .059 .168 -.351 1.000 -.271 significance (2-tailed) .668 .817 .504 .153 .276 gender correlation coefficient -.100 -.217 .155 .278 -.271 1.000 significance (2-tailed) .693 .387 .539 .264 .276 *correlation is significant at the 0.05 level (2-tailed); n=18 table 2. chi-square results for brush teeth in the morning? and been to the dentist in the last year? chi-square tests value df asymptotic significance (2-sided) exact significance (2-sided) exact significance (1-sided) pearson chisquare 5.294a 1 .021 continuity correctionb .847 1 .357 likelihood ratio 3.905 1 .048 fisher’s exact test .167 .167 linear-by-linear association 5.000 1 .025 n of valid cases 18 a. 3 cells (75%) have expected count less than 5. the minimum expected count is .17. b. computed only for a 2x2 table. 150 december 2022, volume 15, issue 2, 145-159 figure 3 shows the results for the spearman rank order correlation between: did you brush your teeth this morning? and have you been to the dentist in the last year? (n = 18). all the students that brushed their teeth in the morning had been to the dentist in the past year. only one student did not go to the dentist in the last year, and that student did not brush their teeth in the morning. this correlation was significant with a p-value of 0.02. figure 3. bar graph of spearman rank order correlation between “did you brush your teeth this morning?” and “have you been to the dentist in the last year?” figure 4 displays the spearman rank order correlation for the variables, do you like to read? and gender? these questions were asked because students had to read and listen to the chapter from the ebook for oral health literacy© curriculum to learn about their oral health. when answering the attitudinal question, “do you like to read?”, 18 participants indicated “yes, all of the time” (4 girls, 2 boys); “sometimes” (5 girls, 3 boys); and “no, not really” (1 girl, 3 boys). in general, girls (n = 10) liked to read more than boys (n = 8), but it was not significant. figure 4. spearman rank order correlation between “gender?” & “do you like to read?” figure 5 shows the frequency results for the number of cavities that the students self-reported. when asked the question “how many cavities (or fillings) do you have in your teeth?”,12 students reported that they did not have any cavities, 4 students reported one cavity, 1 student reported two cavities, and 1 student reported 8 cavities in their teeth. although not indicated in figure 5, when students were asked, “have you ever missed school because your teeth hurt?”, only one student answered “yes” and seventeen students answered “no”. figure 5. bar graph for “how many cavities (or fillings) do you have in your teeth?” results for chapter elaborations students were encouraged to write their thoughts about the chapter 5 and chapter 6 ebook stories. written responses included vocabulary from the stories and certain phrases they recalled from reading a story one time. the written elicitations from students provided a form of qualitative responses that go beyond the objective multiple-choice responses that they also provided. the results below will explain the chapter 5 and chapter 6 vocabulary words and phrases that represented their functional health knowledge, including what elaborations that students remembered about the pictures and words from the book, what they liked about the book, and what they learned about their teeth. chapter 5 elaborations after reading the chapter 5 ebook, some of the most frequently reported vocabulary words were: calcium, healthy, dentist, goal, and yogurt. some students not only listed a singular word but wrote the vocabulary word in a phrase similar to what they read in the book and recalled from their memory. some examples are “setting goals for brushing and flossing teeth”, “going to the dentist every 6 months”, “it said i set a goal to eat yogurt which has calcium because calcium helps my teeth to grow strong.”, “calcium helps my teeth grow.” “set a goal to visit and talk with my dentist every 6 months.”, “that fiber and calcium help your teeth stay healthy.” these self-reported vocabulary words and phrases were important because the students only read the one-minute story once then recalled and wrote down words that stood out to them or seemed meaningful to them. the data showed that 151 use of an ebook for oral health literacy© curriculum / ubbes & whitesel students were able to draw upon their memory and use a working vocabulary which represented their functional health knowledge from the curriculum. chapter 6 elaborations after reading chapter 6 only once, some of the most frequently written vocabulary words were: eating, healthy, dentist, calcium, and yogurt. other written answers included: decisions, choices, floss, broccoli, brush teeth, and clean teeth. when asked what they learned from reading chapter 6, the children wrote the following phrases and tips: “do not eat as much sugar”; “eat an apple instead of candy”; “drink lots of water”; “drink milk everyday”; “eat fruits and vegetables.” others said, “when you go to the store buy a toothbrush” and “brush them every day”. by asking more open-ended questions for elaborations, students were able to remember key words and phrases which may help them make healthier choices to benefit both their nutrition and oral health. elaborations indicated that students were able to recall and comprehend basic health information in the form of nutrition and oral health vocabulary words and phrases, which represented their functional health knowledge from the curriculum. remembering the pictures in the book when second and third graders (n = 39) were asked “what is something you remember about the pictures in the book?”, the students wrote: “they all contain kids/adults making good and healthy choices”, “lettuce is good for your teeth.”, “a boy eating an apple.”, “brushing teeth and flossing.”, “girl drinking water.”, “they all include kids doing healthy choices.”, “someone smiling”, “they include nutritious pictures.”, “every picture had a healthy food in it.”, “eating yogurt.”, and “they were all doing something about their teeth.” remembering the words in the book when second and third graders (n = 39) were asked, “what is something you remember about the words in the book?”, the students wrote: “floss.”, “fiber.”, “calcium helps my teeth grow strong.”, “i buy toothpaste with my mom and floss.”, “it said i set a goal to eat yogurt which has calcium because calcium helps my teeth to grow strong.”, “set a goal to visit and talk with my dentist every 6 months.”, “setting goals for brushing and flossing teeth.”, “they talk about food that is healthy for your teeth.”, “they talk about fruit.”, “you need to keep your teeth healthy.”, “the words are all about teeth.”, “food goals.”, “milk is healthy for your teeth.”, “fruit is good.”, “nutritious beverages.”, “they told us about healthy food.”, “going to the dentist every 6 months.”, “healthy goals.”, “yogurt has calcium.”, “when you go to the store buy a toothbrush.”, “they all talk about kids making their choices about being more healthy.”, “they make healthy goals.”, and “they talk about healthy choices.” remembering what children liked about the book when second and third graders (n=39) were asked, “what did you like about the book that you read?”, the students wrote, “i love my teeth.”, “it had vegetables.”, “it has healthy tips for you.”, “it taught me to do healthy choices, keep my teeth healthy.”, “tips.”, “the girl and the mom buying toothpaste and floss.”, “the pictures.”, “it was cool.”, “to set healthy goals.”, “i like that it helps me remember brush your teeth.”, “it helps you learn how to keep your teeth clean.”, “all of it.”, and “to help us be healthy.” remembering what children learned about their teeth when second and third graders (n = 39) were asked, “what is something that you learned about your teeth today?”, they wrote: “brush every day.”, “calcium helps my teeth grow strong.”, “floss your teeth.”, “that fiber and calcium help your teeth stay healthy.”, “that you always need to brush your teeth.”, “to go to the dentist every 6 months.”, “always floss.”, “brush your teeth to keep them clean.”, “calcium makes your teeth strong.”, “fill in cavities.”, “clean my teeth.”, “go to the dentist every 6 months.”, “how to brush your teeth.”, “to not eat as much sugar.”, and “you can keep your teeth clean if you eat certain foods sometimes.” discussion there is an ongoing need to offer more preventative oral health education for u.s. children, especially since 51 million hours of school time are missed annually because of dental-related problems (usdhhs, 2000), and children with fair or poor oral health are 2.8 times more likely to lose greater than one hour of school time compared to children with very good oral health (naavaal & kelekar, 2018). moreover, there are limited empirically tested school-based caries programs with adequate training support for teachers and school staff to ensure fidelity to program implementation (sharma, et al., 2021). in the current pilot study to investigate the feasibility of introducing the ebook for oral health literacy© curriculum to second and third graders, a dietitian serving as a guest nutrition educator found it easy to implement two chapters in the classroom and build additional content around the stories. because a lack of oral hygiene and inadequate nutrition are risk factors for systemic diseases that begin in the mouth (nidcr, 2021), school curricula should promote the integrative connections between oral health, nutrition, and disease prevention in order to boost children’s functional health knowledge, skill development, and health behaviors across the lifespan. 152 december 2022, volume 15, issue 2, 145-159 the ebook for oral health literacy© curriculum is aligned to the national health education standards with a specific focus on developing decision making and goal setting skills. the national health education standards focus on what students should know and be able to do with an emphasis on demonstrating health-enhancing skills and behaviors to reduce risks, promote health, and prevent disease (joint committee on national health education standards, 2007). results from the 2016 school health policies and practices study (cdc, 2017) indicated that 77.1% of school districts had adopted a policy stating that their elementary schools will teach decision making, and 69.2% of school districts had adopted a policy to teach goal setting. decision making is defined as “the ability to select between two or more alternatives to reach the best outcome in a specified time frame. decision making requires the use of accurate and reliable information while progressing through a set of steps intended to take deliberate actions to enhance health” (national consensus for school health education, 2022, p. 20). goal setting is defined as “the process of determining a desired health-related behavior or practice to achieve over a specific time period. this process is more deliberate than desires and momentary intentions and involves committed thoughts, emotions, and behaviors toward attaining the goal” (national consensus for school health education, 2022, p. 21). the national health education standards (joint committee on national health education standards, 2007, p. 63) suggested that students in prekindergarten to grade two receive 40 hours of instructional time per academic year which is approximately one hour per week. instructional time for students in grades three to twelve were recommended to receive 80 hours of instructional time per academic year, which is approximately two hours per week. in the current study, health education was not occurring at all. therefore, the second and third-grade children in the current study would benefit greatly from 1 to 2 hours of health instruction each week (joint committee on national health education standards, 2007). the fact that the ebook for oral health literacy© curriculum only requires one-minute intervals to read in order to instruct children on both their oral health and nutrition behaviors could be a perk for children so they learn more functional health knowledge while also learning health-related skills. the ebook for oral health literacy© curriculum combines declarative sentences and a vocabulary about oral health and nutrition to teach elementary children about health education concepts and skills. the curriculum focuses specifically on cognitive skill development by underlining the words ‘decision making’ (chapter 5) and ‘goal setting’ (chapter 6) on each page for promoting children’s awareness of these skill-based verbs to help them take personal and social actions toward practicing their oral health hygiene and consuming healthy food and beverages. questions, cues, and advanced organizers are known to be an evidence-based instructional strategy (clemons et al., 2010). on each page, underlined cues for the words “decision making” and “goal setting” can also be described as a priming effect, which involves the activation of a mental concept in memory to increase the likelihood that the concept will be used again when processing information (farrar et al, 2022). priming effects have been successfully applied in early childhood vocabulary (avila-varela et al., 2021). additionally, the concluding page in each story is intended to be a form of skill-based cuing to provoke further action. chapter 5 reads “how about you? will you set a food goal for healthy teeth too?” whereas chapter 6 reads “i’ve made the decision to keep my teeth healthy and strong! how about you? have you made the decision too?”. these question cues are an example of interactive health literacy between the writer (narrator) and the reader in first-person narrative. future research will have to determine the effects of priming on food goals and oral health hygiene behaviors among children exposed to the oral health literacy curriculum. in the current study, health education instruction by the classroom teacher was limited so the health literacy intervention occurred when a dietitian visited the classroom. in this situation, expertise from the nutrition community helped to promote the importance of nutrient-dense food and beverages in tandem with oral health education which is an example of how to advance community involvement and health education using the whole school, whole community, and whole child model (lewallen et al., 2015). finding ways to implement the ebook for oral health literacy© curriculum in other places in the school day could be advantageous since children’s elaborations showed that they learned and recalled content knowledge in the form of vocabulary and health-related phrases after reading only one chapter (grade 2) or two chapters (grade 3). the vocabulary and linguistic readiness of children to learn a specific domain of health within their academic studies is paramount. health-related words like decay, infection, cavities, flossing, rinsing, and brushing need to be named and understood (e.g., comprehension) through functional health literacy skills (e.g., reading, writing, and speaking) before children can operationalize or demonstrate their functional health knowledge and skill development toward healthy behaviors. by repeating daily health behaviors each day and practicing the skills leading to the behavior based on feedback from teachers and parents, children can form important habits of health. research shows that feedback both regulates and is 153 use of an ebook for oral health literacy© curriculum / ubbes & whitesel regulated by motivational beliefs (nicol & macfarlanedick, 2006). external feedback has been shown to influence how students feel about themselves (positively or negatively), and what and how they learn (dweck, 1999). students need to practice oral health patterns every day to meet standards for brushing, rinsing, and flossing their teeth (thornton et al., 2019; dos santos et al., 2011). oral health requires children to learn functional knowledge of what hygiene is (and what it is not) while also demonstrating procedural knowledge of how to do it. contextual knowledge is also necessary so students know when, where, and to what extent they should practice the oral health behaviors of brushing, rinsing, flossing, and eating and drinking nutrient-dense food and beverages for healthy teeth. written feedback from second and third graders indicated that the ebook for oral health literacy© curriculum may have provided opportunities for students to increase their functional knowledge and vocabulary about oral health hygiene and healthy eating. students were able to access visual role models on each page to increase their awareness of social norms that lead to healthy behaviors. students also experienced auditory cues when the story was read to them which aimed to build their understanding of words and how they are pronounced. children benefit from multimodal educational approaches that integrate visual, textual, auditory, and gestural representations when learning about their health (ubbes, 2008). the design of the ebook for oral health literacy© even goes beyond multimodal approaches to include multisensory, multigenre, and multidisciplinary elements for teaching about oral health hygiene, nutrition, and dental checkups (ubbes et al., 2018). health educators who learn how to adopt a constructivist pedagogical perspective (ubbes et al., 2009) will be able to seek out how students are thinking about a topic and reflect on how they make meaning from the lesson. health education interventions have traditionally focused on knowledge, attitudes, and behaviors in a didactic manner. however, child-centric approaches that adopt elicitations through oral conversations and open-ended written responses may help educators to uncover the motivations and attitudes of children when they are learning skill-based health education. the national health education standards (national consensus for school health education, 2022) focus on skills like interpersonal communication, decision making, and goal setting, to name a few. so, asking students to talk and write about their learning experiences after reading chapters from the ebook for oral health literacy© curriculum can provide insights into how children practice cognitive skills leading to oral health behaviors. such elicitations can also be aimed at helping children to elaborate on their attitudes, beliefs, and feelings about specific health behaviors like oral health and nutrition. elaborations through written and spoken conversations can help children to make progress in practicing health-related behaviors which are the key outcome of any health education curriculum, but children can also be taught to explicitly work on two behaviors at once by realizing similar patterns between oral hygiene and nutrition for building healthy teeth, mouth, and gums. with limited time to teach health education in the school day, the curriculum can also extend to homebased implementation around mealtime, nap time, and bedtime routines. although the main outcome of a skill-based health education curriculum is healthy behavior (joint committee on the national health education standards, 2007), schools can also use literacy time across the curriculum to construct meaning about health. in second and third grade, children build their literacy and reading proficiency by “learning to read”. however, by fourth grade, children should have strong literacy skills as they shift toward “reading to learn” more disciplinary content knowledge (wright, 2019, p.4). according to the ohio department of education (2020), “language and literacy acquisition and achievement [are] foundational knowledge that supports student success”. as such, literacy skills should be fully developed at the second and third grade levels so that they may serve as a foundation for further literacy development. the national public health initiative, healthy people 2030 (usdhhs, 2020), indicates that only 48% of fourth graders read at their grade level in the united states. functional health literacy depends on the basic skills of reading, writing, and speaking about health (ubbes et al., 2018b), so functional health literacy serves as an integrated pathway for learning two disciplines, e.g., literacy and health, at the same time. although health educators are not responsible for the direct teaching of reading, writing, and speaking in a literacy sense, health educators would be using these functional forms of literacy, albeit functional health literacy, when providing examples of individuals learning about health and how to practice healthy behaviors. children need to experience and practice these functional forms of literacy across the school curriculum so they can communicate and make meaning when learning about health. hence, learning to read and reading to learn become symbiotic processes from early childhood into middle childhood and beyond. therefore, the ebook for oral health literacy© curriculum may be helpful in increasing functional knowledge, vocabulary, and reading proficiency about health, which could result in higher student achievement and lifelong learning. the narration of the chapter stories help children on a developmental 154 december 2022, volume 15, issue 2, 145-159 level to be able to learn health education at their own pace as they advance to the next page as assisted by teachers, peers, and/or parents as guides on the side. goal 1. to determine what children learned about their teeth from two chapters of the ebook for oral health literacy© curriculum and what they remembered from reading the oral health and nutrition stories this study not only aims to educate students about oral health habits, but also help to build their healthy literacy skills. one of the multiple-choice questions asks students, “do you like to read?” about 20% of students said that they do “not really like to read”. this could be for several reasons, one of which being that they are not comfortable reading due to a lack of literacy skills. by implementing the ebook for oral health literacy© curriculum, students can learn to build both their health literacy and self-efficacy which would hopefully promote a liking for reading and increase the amount of time spent reading. when looking further into the results if the students like to read, girls generally liked to read more than boys did. when answering the question, “do you like to read?”, four girls versus two boys selected “yes, all of the time”; five girls versus three boys selected “sometimes”; and only one girl versus three boys selected “no, not really.” although these findings were not statistically significant when the spearman rank order correlation was run, there is prior research to build on the idea that girls generally enjoy reading more than boys (ubbes et al, 2018b). by allowing children to respond to open-ended questions rather than only multiple-choice questions, they were able to recall what stood out to them as they read the ebook. open-ended questions also allow teachers and parents better insights into what the children know, feel, and show interest in. written language is an expressive language that affords teachers and parents insights into what children know and think about, especially when the curriculum is used as a vehicle for sharing new vocabulary words that lead to functional health knowledge. expressive written language also gives teachers and parents insights into what children are feeling and how motivated they are to practice daily oral hygiene and eating nutritious foods. for the chapter 5 ebook, some of the most frequently reported vocabulary words were calcium, healthy, dentist, fruit, vegetable, and yogurt. for chapter 6 of the ebook, some of the most reported vocabulary words were: eating, healthy, dentist, calcium, and yogurt. other words frequently reported were goals, decisions, choices, floss, broccoli, brush teeth, and clean teeth. based on the children’s responses, they were able to recall multiple vocabulary words as well as other words that were mentioned throughout the ebook chapters around the topics of oral health and nutrition. some of the words used in the chapters might not have been familiar to the children. this forced the children to recall from memory what they had heard, as well as remember the context in which the word was used, in order to spell it out in their open-ended answers. this chain of thought relates to higher order thinking skills. in a different study it was said that “…in its simple form, hots (higher order thinking skills) in storytelling was developed in young learners through open-ended question[ing], a strategy which enables students to practice speaking through giving opinion, comment, and imagination while analyzing and evaluating the story” (setyarini et al., 2018). there are many advantages to using open-ended questions, because questions elicit children to tell a story and practice various ways of thinking. children who wanted to communicate their thoughts via lists and sentences, had to spell out their ideas (this study) and/or drew an illustration (our first study). a couple of students drew pictures to express what stood out to them; for example, one student drew a picture of a toothbrush. hence, higher order thinking skills can prompt children to practice functional health literacy as an expressive language for communication. goal 2. to determine whether children liked to read and what they liked about the words and the pictures of the oral health stories this study focused on oral health literacy and a more specific type of functional health literacy that emphasized reading and writing about health. results showed that 80% of students said that they liked to read, which supported the value of the ebook for oral health literacy© as a vehicle for supporting literacy skills and oral health behaviors in tandem. the ebook narrates the story to children which can help them learn new vocabulary words leading to reading fluency. fluency is a critical predictor of literacy. by implementing the ebook for oral health literacy© curriculum, students learned to build their functional health literacy which would hopefully promote an ease and liking for reading and increase the amount of time they spend reading to learn facts, topics, and concepts about oral health. the open-ended questions in this study, prompted students to elaborate on their liking of two chapters in the ebook for oral health literacy© curriculum. for the purposes of this study, liking was conceptualized as a plain language word that helped children to identify their feelings, attitudes, or preferences for something. often children do not prefer things that are new or foreign to them (e.g., vegetables), which has been coined “neophobia” (chawner & hetherington, 2021). 155 use of an ebook for oral health literacy© curriculum / ubbes & whitesel for example, when children say they don’t ‘like’ milk, nutrition educators can talk about and differentiate between dairy and nondairy milk examples and help them to understand that each serving of milk provides 25% of the calcium needed each day (national dairy council, 2019). such information allows educators to examine children’s preferences so they can reframe their thinking on why an item they disliked may actually be important, helpful, or healthy for them; ultimately this interactive health literacy process can help children build their functional health knowledge even more. after understanding why some children may not “like” milk, health educators can talk about the benefits of milk to reframe the children’s beliefs; this interactive health literacy process of talking and/or reading about milk also increases children’s functional health knowledge as long as educators use valid and reliable health information and concepts. when children were asked the question, “do you like to read?” (n = 18), six children said that they liked to read all the time and eight students said that they sometimes liked to read. these children were able to recall and write about what they liked and learned from the book after a short one-time exposure, as well as being able to recall words and pictures that stood out to them from the ebook for oral health literacy© curriculum. these findings are supported by a study by zullig et al (2017) who found that oral health behaviors were significantly related to reading behaviors among 7th and 8th graders. if health educators teach about oral health topics earlier in the elementary school curriculum using a meaningful multimodal intervention that is easily recalled (ubbes et al, 2018c), children will have a stronger foundation for their oral health and overall general health since the two are inextricably linked (national institute of dental and craniofacial research, 2021). functional health literacy is the ability to read, write, and speak about health (ubbes & ausherman, 2018). even if children can read and write at requisite levels for schooling, few children have been exposed to enough health-related topics and concepts to have a strong vocabulary about health. low vocabulary in health education limits the transition from basic literacy to health literacy. studies have shown that vocabulary knowledge is requisite to reading comprehension and meaning making (dong et al., 2021). an important educational goal would be to focus on functional knowledge about nutrition and oral health that includes valid and reliable information from which to function in everyday situations and routines. as such, the process of reading about valid and reliable health information can support children in their development of more background knowledge from which to communicate and act on health information, set goals, and make decisions about health. goal 3. to determine whether children brushed their teeth in the morning; had any cavities; had teeth or mouth pain; and visited the dentist in the last year for a dental checkup eighteen third grade students answered four multiple choice questions and one open-response question regarding their oral health habits. all but one student replied “yes” when asked, “have you been to the dentist in the last year?” although it is positive that only one student had not “been to the dentist in the last year”, everyone is recommended to see a dentist every six months. twelve students reported that they did not have any cavities. specifically, four students reported that they had one cavity, one student had two cavities, and one student had eight cavities. based on the small sample of students, it can be concluded that one in three third grade students had at least one cavity. with oral caries being a completely preventable infectious disease, this statistic is troubling. despite the high number of students with at least one cavity (n=6), only one student reported missing school because their teeth hurt. other results showed that fifteen third grade students brushed their teeth in the morning whereas three students replied “no”. results showed that one in five students (20%) brushed their teeth in the morning. guidelines suggest that this number can and should be much higher due to the recommendation that children should brush their teeth twice a day with fluoridated toothpaste (thornton-evans et al., 2016). brushing one’s teeth is an important habit for students to adopt, which should be added to their daily routine in the morning and at night. research indicates that there are several barriers that keep children from brushing their teeth (duijster et al., 2015), including lack of knowledge, poor accessibility to dental products (e.g., toothbrush, toothpaste, and floss), and limited role models. although dental products are often distributed to children during a dental checkup, not all children have access to dental checkups due to geography, transportation, and/or financial barriers. school-based health clinics may be one of the most important and easily accessible ways that children can be supported with their overall health and oral health hygiene, specifically (knopf et al, 2016). the placement and use of the ebook for oral health literacy© curriculum in school-based health clinics should be studied in the future. limitations the main limitation of the study was a small sample size (n < 50). future studies will need to be conducted beyond a convenience sampling in order to have more participants and a control group used in the research. the second limitation was that reading or writing were not directly assessed in this study. the acquisition of literacy leads to functional changes in 156 december 2022, volume 15, issue 2, 145-159 several cortical structures in the brain, including white matter interconnections that are predictive of reading performance (lopez-barrosoet al., 2020; thiebaut de schotten et al., 2014). moreover, the ability to gain access to reading and writing scores of children for health education research is fraught with challenges. child-centric surveys that measure recognition of oral health words and reading of health-related terminology in different contexts should be developed to address student gains in functional health literacy in oral health, nutrition, and other health behaviors. the third limitation was that we were not privy to classroom conversations that occurred between the dietitian and the elementary children. as such, we only have written answers from students to determine how much learning occurred. since functional health literacy addresses the ability to “read, write, and speak about health” (ubbes & ausherman, 2018; ubbes et al., 2018b), future research should attempt to transcribe verbal and nonverbal communication of children in the classroom when teaching other multimodal chapters of the curriculum. understanding how much reading, writing, and speaking occurs in the health education classroom by children may be helpful for establishing instructional parameters for how functional health knowledge contributes to the development of functional health literacy (and vice versa). investigative studies what assess the synergy between health literacy and functional health literacy may ultimately lead children to make decisions and set goals for their health once they understand the noun and verb vocabularies that inform their actions to do so. as such, noun and verb cues like ‘decision’ and ‘to decide’, or ‘goal’ and ‘to set a goal’, respectively, become key lexical milestones for that informs the development of functional health literacy. conclusion an ebook for oral health literacy© curriculum was used as a brief intervention to help school-aged children use their functional health literacy skills of reading, writing, and speaking to learn about oral health hygiene and the importance of choosing healthy food and beverages for their teeth. the curriculum focused on building functional knowledge about oral health hygiene and nutrition behaviors through a health literacy intervention that highlighted the need for reasoned actions when setting food goals for healthy teeth and making decisions to keep teeth healthy and strong. the visual-textual-gestural narrative helped children to build a vocabulary about oral health when reading and writing about health. children wrote about words and phrases that they recalled after reading each chapter. their elaborations exemplified the importance of using a constructivist theoretical framework to elicit children’s functional health knowledge about oral health and nutrition. chi-square results showed that there was a moderate significant correlation between children who reported brushing their teeth in the morning and having been to the dentist in the past year (p = .021). the use of the ebook for oral health literacy© curriculum can help children to learn functional health knowledge about brushing, flossing, and rinsing teeth, eating nutrientdense foods and beverages, and going to the dentist for a checkup, including skill-based vocabularies about making decisions and setting goals for health. references american academy of pediatrics, section on pediatric dentistry. (2003). policy statement: oral health risk assessment timing and establishment of the dental home. pediatrics, 11, 1113-1116. arenson, m., hudson, p. j., lee, n., & lai, b. (2019). the evidence on school-based health centers: a review. global pediatric health, 6, 1-10. https:// doi.org/2333794x19828745 avila-varela, d. s., arias-trejo, n., & mani, n. (2021). a longitudinal study of the role of vocabulary size in priming effects in early childhood. journal of experimental child psychology, 205, 105071. https://doi.org/10.1016/j.jecp.2020.105071 brown, a., lowe, e., zimmerman, b., crall, j., foley, m., & nehring, m. (2006). preventing early childhood caries: lessons from the field. pediatric dentistry, 28(6), 553–560. butera, a., maiorani, c., morandini, a., simonini, m., morittu, s., trombini, j., & scribante, a. (2022). evaluation of children caries risk factors: a narrative review of nutritional aspects, oral hygiene habits, and bacterial alterations. children, 9(2), 262. https://doi.org/10.3390/ children9020262 cardina c. (2014). academic majors and subject-area certifications of health education teachers in the united states, 2011-2012. journal of health education teaching, 5(1), 35–43. https://eric. ed.gov/?id=ej1085288 chawner, l. r., & hetherington, m. m. (2021). utilising an integrated approach to developing liking for and consumption of vegetables in children. physiology & behavior, 238, 113493. https://doi.org/10.1016/j.physbeh.2021.113493 cook-harvey, c. m., darling-hammond, l., lam, l., mercer, c., & roc, m.(2016). equity and essa: leveraging educational opportunity through the every student succeeds act. learning policy institute. 157 use of an ebook for oral health literacy© curriculum / ubbes & whitesel dong, y., tang, y., chow, b. w. y., wang, w., & dong, w. y. (2020). contribution of vocabulary knowledge to reading comprehension among chinese students: a meta-analysis. frontiers in psychology, 11, 1-15. https://doi.org/10.3389/ fpsyg.2020.525369 dos santos, a.p.p., nadanovsky, p., & de oliveira, b. h. (2011). inconsistencies in recommendations on oral hygiene practices for children by professional dental and paediatric organisations in ten countries. international journal of paediatric dentistry, 21(3), 223-231. https://doi.org/10.1111/j.1365-263x.2011.01115.x duijster, d., de jong-lenters, m., verrips, e., & van loveren, c. (2015). establishing oral health promoting behaviours in children–parents’ views on barriers, facilitators and professional support: a qualitative study. bmc oral health, 15(1), 1-13. http://doi.10.1186/s12903-015-0145-0 dweck, c. (1999). self-theories: their role in motivation, personality and development. psychology press. eklund, k., burns, m. k., oyen, k., demarchena, s., & mccollom, e. m. (2022). addressing chronic absenteeism in schools: a meta-analysis of evidence-based interventions. school psychology review, 51(1), 95-111. https://doi.org/ 10.1080/2372966x.2020.1789436 farrar, s. t., plagnol, a. c., & tapper, k. (2022). the effect of priming on food choice: a field and laboratory study. appetite, 168, 105749. https:// doi.org/10.1016/j.appet.2021.105749 ferrer-cacales, r., sánchez-sansegundo, m., ruizrobledillo, n., albaladejo-blazquez, n., lagunaperez, a., & zaragoza-marti, a. (2018). eat or skip breakfast? the important role of breakfast quality for health-related quality of life, stress and depression in spanish adolescents. international journal of environmental research in public health, 15, 1781. https://doi.org/10.3390/ ijerph15081781 giovannini, m., verduci, e., scaglioni, s., salvatici, e., bonza, m., riva, e., & agostonic, c. (2018). breakfast: a good habit, not a repetitive custom. journal of internal medicine research, 36, 613624. https://doi.org/10.1177/147323000803600401 griffin, s.o., naavaal, s., scherrer, c.r., patel, m., & chattopadhyay, s. (2017). evaluation of schoolbased dental sealant programs: an updated community guide systematic economic review. american journal of preventive medicine, 52(3), 407-415. https://doi.org/10.1016/j. amepre.2016.10.004 griffin, s.o., wei, l., gooch, b., weno, k., & espinoza, l. (2016). changes in dental sealant and untreated tooth decay prevalence and the estimated impact of increasing school-based sealant program coverage. mmwr, 65, 1141-1145. griffin, s.o., barker l.k., wei, l., chien-hsun, l., albuquerque, m.s, & gooch, b.f. (2014). use of dental care and effective preventive services in preventing tooth decay among us children and adolescents—medical expenditure panel survey, united states, 2003–2009 and national health and nutrition examination survey, united states, 2005–2010. mmwr, 63(2), 55–61. jackson, s.l., vann jr, w.f., kotch, j.b., pahel, b.t., & lee, j.y. (2011). impact of poor oral health on children's school attendance and performance. american journal of public health, 101(10), 19001906. https://doi.org/10.2105/ajph.2010.200915 jenssen, b.j., kelly, m.k., powell, m., bouchelle, z., mayne, s.l., & fiks, a.g., (2021). covid-19 and changes in child obesity. pediatrics. 147 (5). https://doi.org/10.1542/peds.2021-050123 joint committee on national health education standards (2007). national health education standards: achieving excellence (2nd ed). atlanta, ga: american cancer society. knopf, j. a., finnie, r. k., peng, y., hahn, r. a., truman, b. i., vernon-smiley, m., ... & community preventive services task force. (2016). school-based health centers to advance health equity: a community guide systematic review. american journal of preventive medicine, 51(1), 114-126. https://doi. org/10.1016/j.amepre.2016.01.009. lastrucci, v., lorini, c., caini, s., florence health literacy research group, & bonaccorsi, g. (2019). health literacy as a mediator of the relationship between socioeconomic status and health: a cross-sectional study in a population-based sample in florence. plos one, 14(12), e0227007. lewallen, t. c., hunt, h., potts-datema, w., zaza, s., & giles, w. (2015). the whole school, whole community, whole child model: a new approach for improving educational attainment and healthy development for students. journal of school health, 85(11), 729739. https://doi-org/10.1111/josh.12310 lópez-barroso, d., de schotten, m. t., morais, j., kolinsky, r., braga, l. w., guerreiro-tauil, a., dehaene, s., & cohen, l. (2020). impact of literacy on the functional connectivity of vision and language related networks. neuroimage, 213, 116722. https://doi.org/10.1016/j.neuroimage.2020.11672 158 december 2022, volume 15, issue 2, 145-159 mchugh, m.l. (2013). the chi-square test of independence. biochemical medicine, 23(2), 143-149. https://doi.org/10.11613/bm.2013.018 mcnamara, j., mena, n.z., neptune, l., & parsons, k. (2021). college students’ views on functional, interactive and critical nutrition literacy: a qualitative study. international journal of environmental research in public health, 18, 1124. https://doi.org/10.3390/ijerph18031124 mishra, p., pandey, c. m., singh, u., gupta, a., sahu, c., & keshri, a. (2019). descriptive statistics and normality tests for statistical data. annals of cardiac anesthesia, 22(1), 67. https://doi. org/10.4103/aca.aca_157_18 mobley, c., marshall, t. a., milgrom, p., & coldwell, s. e. (2009). the contribution of dietary factors to dental caries and disparities in caries. academic pediatrics, 9(6), 410-414. https://doi. org/10.1016/j.acap.2009.09.008 morrison, a. k., glick, a., & yin, h. s. (2019). health literacy: implications for child health. pediatrics in review, 40(6), 263-277. https://doi.org/10.1542/ pir.2018-0027 naavaal, s. & kelekar, u. (2018). school hours lost due to acute/unplanned dental care. health behavior policy review, 5(2), 66-73. https://doi. org/10.14485/hbpr.5.2.7 national consensus for school health education. (2022). national health education standards: model guidance for curriculum and instruction (3rd edition). www.schoolhealtheducation.org national dairy council. (2019). discover milk’s nutritional value. https://www.usdairy.com/news-articles/ is-milk-nutritious national institute of dental and craniofacial research (nidcr). (2021). oral health in america: advances and challenges. bethesda, md. national institutes of health. nicol, d.j. & macfarlane-dick, d. (2006). formative assessment and self-regulated learning: a model and seven principles of good feedback practice. studies in higher education, 31(2), 199218. https://doi.org/10.1080/03075070600572090 nielsen-bohlman, l., panzer, a. m., & kindig, d. a. (2004). health literacy: a prescription to end confusion. institute of medicine. nutbeam, d. (2000). health literacy as a public health goal: a challenge for contemporary health education and communication strategies into the 21st century. health promotion international, 15(3), 259–267. phantumvanit, p., makino, y., ogawa, h., rugg-gunn, a., moynihan, p., petersen, e., evans, w., feldens, c.a., lo, e., khoshnevisan, m.h., baez, r., varenne, b., vichayanrat, t., songpaisan, y., woodward, m., nakornchai, s., & ungchusak c. (2018). who global consultation on public health intervention against early childhood caries. community dentistry and oral epidemiology, 46, 280–7. https://doi.org/10.1111/cdoe.12362 protudjer, j.l.p., marchessault, g., kozyrskyj, a.l., & becker, a.b. (2010) children’s perceptions of healthful eating and physical activity. canadian journal of dietetic practice and research, 71,19– 23. https://doi.org/10.3148/71.1.2010.19 ramos-gomez, f., kinsler, j., & askaryar, h. (2020). understanding oral health disparities in children as a global public health issue: how dental health professionals can make a difference. journal of public health policy, 41, 1-11. https:// doi.org/10.1057/s41271-020-00222-5 ruff, r. r., senthi, s., susser, s. r., tsutsui, a. (2019). oral health, academic performance, and school absenteeism in children and adolescents: a systematic review and meta-analysis. the journal of the american dental association, 150(2): 111-121. https://doi.org/10.1016/j. adaj.2018.09.023 schunk, d. h., & hanson, a. r. (1985). peer models: influence on children's self-efficacy and achievement. journal of educational psychology, 77(3), 313. setyarini, s., muslim, a. b., rukmini, d., yuliasri, i., & mujianto, y. (2018). thinking critically while storytelling: improving children’s hots and english oral competence. indonesian journal of applied linguistics, 8(1), 189-197. https://doi. org/10.17509/ijal.v8i1.11480 sharma, s. v., kelder, s., yamal, j., chuang, r., byrdwilliams, c., bona, g., bajaj, n., brito, f., & neumann, a. s. (2022). development and feasibility testing of catch healthy smiles, an oral health promotion intervention for prevention of dental caries among elementary school children. journal of school health, 92(1), 20-30. https://doi.org/10.1111/josh.13100 159 use of an ebook for oral health literacy© curriculum / ubbes & whitesel tappe, m. k., wilbur, k. m., telljohann, s. k., & jensen, m. j. (2009). articulation of the national health education standards to support learning and healthy behaviors among students. american journal of health education, 40(4), 245-253. https://doi.org/10.1080/19325037.2009.10599100 thiebaut de schotten, m., cohen, l., amemiya, e., braga, l. w., & dehaene, s. (2014). learning to read improves the structure of the arcuate fasciculus. cerebral cortex, 24(4), 989-995. https://doi.org/10.1093/cercor/bhs383 thornton-evans, g., junger, m. l., lin, m., wei, l., espinoza, l., & beltran-aguilar, e.(2019). use of toothpaste and toothbrushing patterns among children and adolescents united states, 20132016. mmwr. morbidity and mortality weekly report, 68(4), 87–90. https://doi.org/10.15585/ mmwr.mm6804a3 tinanoff, n., baez, r.j., diaz-guillory, c. donly, k. j., feldens, c. a., mcgrath, c., phantumvanit, p., pitts, n. b., seow, w. k., sharkov, n., songpaisan, y., & twetman, s. (2019). early childhood caries epidemiology, etiology, risk assessment, societal burden, management, education, and policy: global perspective. international journal of pediatric dentistry, 29, 238–48. https://doi. org/10.1111/ipd.12484 ubbes, v.a. & miami university libraries. (2022). digital literacy partnership at miami university. https:// dlp.lib.miamioh.edu ubbes, v.a. & witter, a. (2021). parental influences on children’s oral health behaviors, reading behaviors, and reading attitudes associated with the sharing of a digital story from the ebook for oral health literacy© curriculum, children and teenagers, 4(3), 26-55. https://doi. org/10.22158/ct.v4n3p26 ubbes, v.a., witter, a.m., kraska, c.m., justus, e.e. (2020a). evaluation of an ebook for oral health literacy© to promote child health: readability, suitability, understandability, actionability, and gist-based message design. children and teenagers, 3(1), 54-80.https://doi.org/10.22158/ ct.v3n1p54 ubbes, v.a. & ausherman, j. (2018). a historical comparison of reading materials originating in the 19th & 20th centuries that contributed a language and vocabulary for functional health literacy. the health educator, 50(2), 26-40. ubbes, v.a., dillhoff, r., & maldonado, w. (2018b). reading and writing attitudes of children: conceptual implications for health education and health literacy. journal of health education teaching, 9(1), 49-67. ubbes, v.a., coyle, j., & tzoc, e. (2018c). evaluation of an oral health curriculum: design feedback from three audiences. the international journal of health, wellness, & society, 8(4), 1-10. https:// doi.org/10.18848/2156-8960/cgp/v08i04/1-10 ubbes, v.a., black, j.m., & ausherman, j.a. (2009). teaching for understanding in health education: the role of critical and creative thinking skills within constructivism theory. in black, j.m., furney, s.r., graf, h.m. & nolte, a.e. (eds.). philosophical foundations of health education. san francisco, ca: jossey-bass. ubbes, v.a. (2008). educating for health: an inquirybased approach to prek-8 pedagogy. champaign, il: human kinetics. u.s. department of health and human services. (2000). oral health in america: a report of the surgeon general. rockville, md: usdhhs, national institute of dental and craniofacial research, national institutes of health. https:// www.nidcr.nih.gov/sites/default/files/2017-10/ hck1ocv.%40www.surgeon.fullrpt.pdf. velardo, s., & drummond, m. (2017). emphasizing the child in child health literacy research. journal of child health care, 21(1), 5-13. https://doi. org/10.1177/1367493516643423 videto, d. m., & dake, j. a. (2019). promoting health literacy through defining and measuring quality school health education. health promotion practice, 20(6), 824-833. https://doi. org/10.1177/1524839919870194 vygotsky, l.s. (1978). mind in society (m. cole, trans.). cambridge, ma: harvard university press. international electronic journal of elementary education vol. 3, issue 2, march, 2011. issn:1307-9298 copyright © iejee www.iejee.com implementing a science-based interdisciplinary curriculum in the second grade: a community of practice in action* meredith park rogers** indiana university, bloomington indiana, united states abstract the purpose of this study was to explore the role that a collaborative teaching approach, referred to as a community of practice (cop), had on a team of four second grade teachers’ implementation of a science-based interdisciplinary curriculum. data was collected in the form of extensive observation notes gathered over 10-weeks of twice weekly team meetings and two 45 minute interviews with each participant. from the field notes developed two vignettes for the purpose of illustrating the members cop in action. combining my analysis of the vignettes and the interviews resulted in three emergent themes: 1) benefits, 2) contributions, and 3) their commitment to professional development. from this study i learned that establishing a cop was viewed as a necessary component of the team’s implementation of their science-based interdisciplinary curriculum. implications for encouraging preservice and inservice elementary teachers to develop cops to support science teaching, specifically interdisciplinary teaching, are discussed. keywords: elementary education; science; community of practice; interdisciplinary teaching introduction elementary teachers’ avoiding the teaching of science is not a new issue. tilgner (1990) commented that the situation had not changed in 20 years, and in the decade since, there have been continuing reports along similar lines across the world” (as cited in appleton, 2007, p. 496). reports such as taking science to school: learning and teaching science in grades k-8 * this manuscript is from a dissertation study and was also presented at the 2007 national association for research in science teaching. ** correspondence: dr. meredith park rogers, w. w. wright education building 201 north rose avenue bloomington, indiana 47405-1006 812-856-8168 email: mparkrog@indiana.edu http://www.iejee.com/ mailto:mparkrog@indiana.edu international electronic journal of elementary education vol.3, issue 2, march, 2011 84 (duschl, schweingruber & shouse, 2007) and several project 2061 publications american association for the advancement of science, 1993) (rutherford & ahlgren, 1990; describe the need for consistent science learning in the elementary grades to begin develop students’ scientific literacy. yet, the problem remains, science is considered a second class subject in most elementary classrooms (roden, 2000). some researchers have suggested that to promote the importance of elementary science, studies need to be conducted that examine different approaches elementary teachers use to teach science, such as an integrated curriculum (raizen & michelsohn, 1994; roden, 2000; tilgner, 1990) or collaborative teaching (silva, 2000; supovitz, 2002). it is both of these approaches that serve as the rationale for this study. through extensive observations and discussions with a team of four second grade teachers, i have developed vignettes to illustrate their community of practice (lave & wenger, 1991; wenger, 1998) and to understand the role their community served them in using science as an organizer for achieving a coherent curriculum. to explore this phenomenon further, the following four questions guided my research process: 1. what does this team’s community of practice look like? 2. what role does the community of practice have in implementing their science-based interdisciplinary curriculum? 3. what does the community of practice offer each member with regards to their own professional growth? 4. what are the contributions of each member to the community of practice? conceptual framework lave and wenger’s notion of communities of practice (wenger, 1998) guided my understanding of what it means when groups of people work collaboratively toward a common goal. in this case the goal was to achieve a coherent curriculum using science as an organizer (park rogers & abell, 2006) and what i examined for this study was the inner workings of the team’s cop with respect to how it supported their enactment of this process. the term “community of practice” (cop) is grounded in social learning theory, but as palincsar, magnusson, marano, ford, and brown (1998) pointed out lave and wenger never made any claims about the implications of their studies for constituting communities of practice; in fact they are probably bewildered by the ways in which we and others in education have appropriated their ideas in the service of implementing or developing such communities. however, regardless of lave and wenger’s intention for introducing the notion of cop there is a clear connection between its relevance in studying collaborative practices in educational settings in order to better understand teachers’ instructional decisions and curricular implementation (silva, 2000; palincsar et al., 1998). community of practice in action / park rogers 85 as supovitz (2002) indicated, there are three key components to establishing a successful cop in an education setting. “first, communities of practice mutually engage on the task at hand. second, they communally negotiate the contours and focus of their joint enterprise. and third, they develop a set of shared repertoires to effectively address their work” (supovitz, 2002, p. 1598). regardless of the amount of time a cop has been developed or the experience the members of a cop have, these three components constantly are being refined and honed because each new situation presents new challenges. throughout our daily lives we move in and out of various communities that follow specific practices. these cops provide the “ideal situated contexts through which implicit and explicit meanings are appropriated and negotiated by members of the community” (hung, chee, hedberg, & seng, 2005, p. 160). the social networks of a cop “[form] naturally and are informally bound by the work that people engage in together; they are selforganized, and memberships are based on participation rather than on official status” (wenger as cited in foulger, 2005, p. 3). when needed, meanings are negotiated among members based on the assumed understandings of the culture. therefore, within the structure of a cop, knowledge is constructed according to the group’s explicit and tacit understandings. wesley’s and buysse’s (2001) comparison of a cop to that of a learning organization provides support for how the concept of a cop could be used within a classroom setting. wesley and buysse explained that a learning organization “emerges from a common desire among its members to achieve change (i.e., improve existing practices) [and] it provides regular opportunities for collaborative reflection and inquiry through dialogue” (p. 118). because ongoing reflection and inquiry are also common practices of cops it can be said that cops can often naturally form within educational settings. wenger (1998) referred to participation as the process in which identities are constructed in relation to the community. thus, the notion of a cop provided me with a framework to understand each team member’s identity for participation and how their role contributed to the overall dynamics of the team in implementing an inquiry-based curriculum; an approach that was grounded in their beliefs and knowledge of teaching science as inquiry. literature review there is a growing interest across many education disciplines with using the construct of cop] (lave &wenger, 1991; wenger, 1998) as a means to study the nature of establishing collaborative experiences in various teaching and learning situations. however, for the most part studies examining cops within science education are limited to studying the learning that occurs international electronic journal of elementary education vol.3, issue 2, march, 2011 86 through the use of cops in research teams (e.g., science laboratories) and professional development contexts (palincsar et al. 1998). for example, in a research team setting feldman, divoll, and rogan-klyve (2009) explored characteristics of communities of practice through an apprenticeship experience between graduate and undergraduate studies working on an interdisciplinary scientific research project. what they observed were communities where participants had designated responsibilities thus each member played a pivotal role in the success of the community’s scientific practice and ultimately their learning. within science education professional development contexts, cops are generally used as a method for trying to sustain changes in ideas and practice once teachers return to the classroom (akerson, cullen & hanson, 2009; lumpe 2007). for example, akerson et al. (2009) employed the idea of a cop to support teachers learning about nos and the transfer of this learning to their instructional practice. what they learned was that “while developing a cop is not sufficient on its own to improve teachers’ views and practice related to nos, it provides key supports to allow changes in nos to be continued beyond professional development activities” (p. 22). in both contexts, the purpose of cops was to structure support for learning a new idea and applying that idea to practice (teaching or the lab). silva (2000) explained however, that educational research needs to move beyond using cops simply as a design method, but studying how it can be enacted in a practical sense at the classroom level as a part of teachers’ reflective practice. manouchehri’s study (2001) investigated this idea with two pairs of middle school mathematics teachers. she was interested in understanding what contributions each member of the pair brought to the cop and how the peers felt their partner’s contributions improved their teaching practice. one pair indicated some change in their professional practice after seven months of working together, the other pair did not. manouchehri learned that an effective cop requires effort from all members and that perhaps there needs to be some support or guidance during its initial development from an outside source (e.g., a lead teacher or principal). she also suggested the roles participants seem to naturally take on when participating in such a professional community are critical to the cop’s success and sustainability. silva’s study (2000), while not specifically focusing on development of cops in science, provides a deeper understanding of the dynamics of team planning with elementary teachers. her study looked at three teams of elementary teachers with very different demographics and experiences with designing an integrated language arts and social studies curriculum while under the leadership of a curriculum specialist. the purpose of silva’s study was to share the experiences these teachers encountered and describe how community of practice in action / park rogers 87 each team made sense of the new integrated curriculum. evidence from silva’s (2000) study suggests that “teams do not enact curriculum…instead, teams become vehicles for curriculum decision making” (p. 292). therefore, to develop a better understanding of team teaching at the elementary level, silva suggested the need for gaining a deeper appreciation of the essence of teachers’ experiences as part of a team, their beliefs, and their actions; in other words, their community of practice. from this review of the literature one could conclude that when teachers are afforded the opportunity to work with colleagues, the quality of their teaching improves (lumpe, 2007). through the use of a cop, teachers reflect with one another and are more willing to take risks in their teaching (foulger, 2005). however, it is clear that further exploration is needed into the design and implementation of elementary cops at the classroom level. therefore, studying the design and use of cops at the elementary level where science plays a critical role in the overall curriculum design would not only contribute to a scant literature base but may also help to address roden’s (2000) claim that change must occur at the elementary level to make science a first-class subject. my study approaches this issue from much the same perspective as silva’s (2000) study – to gain the essence of this team’s cop experience through observing their interactions, and eliciting their own thoughts about their collaborative process. therefore, the objective of this study was to record, interpret and share the experiences of four second grade teachers cop; and in particular, the role of their cop in helping them to achieve curricular coherency that is rooted in their knowledge and beliefs about teaching science as inquiry. research design this study employs both a case study approach and method of analyze. according to creswell (1998), “some case studies generate theory, some are simply descriptions of cases, and other are more analytical in nature and display cross-case or inter-site comparisons” (p. 186). for the purpose of this study, a descriptive case study was adopted as the goal was to discuss the four participants as a collective whole in order to understand the dynamics of the team’s community of practice with regards to supporting their use of science for designing curriculum coherency. context of study this study took place at an elementary situated in a growing midwest community. at the time of data collection, the total school population for this school was 465 with 86 students split among the four 2nd grade classrooms involved in this study. the total minority population was 24.5%. in grades k-3 at this school the classroom teacher was responsible for teaching the core content areas of communication arts (literacy), international electronic journal of elementary education vol.3, issue 2, march, 2011 88 mathematics, science, and social studies. students went to teacher specialists for art, music, and physical education and it was during this specialist time period that the teachers scheduled their bi-weekly team meetings. teacher participants’ background the four second grade teachers participating in this study were given the pseudonyms of tracy, brenda, heather, and nancy. i purposively selected these four teachers because of my various past professional experiences with three of them. although i had not had any prior interactions with nancy, she agreed to participate because of the focus on the team approach to interdisciplinary design. in addition to the four participating teachers, the school principal also contributed to the study by providing information on the dynamics of this teaching team in comparison to other teachers in the school, and the school’s overall educational objectives. at the time of this study, tracy was in her 16th year of teaching. over those 16 years she taught grades k-4, with the majority of her teaching time (11 years) at the second grade level. she explained that using an integrated approach to teaching had always played a significant role in her teaching practice, especially with her curricular design experiences during drake’s early years of following the basic school (boyer, 1995) model. during the data collection period for this study, brenda was in her 13th year of teaching. similar to tracy, brenda had experience teaching several of the primary grade levels, although the majority of her teaching was split between two different schools teaching second or third grade. brenda explained that the key to her teaching was to use an inquiry approach across all disciplines. although she felt science and math lent themselves most easily to this approach, she also stated that the more comfortable she became with inquiry, she also found ways to apply inquiry-based practice to her teaching of reading and writing. heather had 14 years of teaching experience, all of which were in the second grade at drake elementary. she admitted that at the beginning of her teaching career that she preferred to teach mathematics, but over the years she grew to love and appreciate teaching science. the fourth teacher of the team, nancy had nine years of teaching experience at the time of this study with seven years at the second grade level at a school other than drake, and two years as a title 1 reading teacher at drake. due to a cut in funding, the title 1 position at drake was removed, but the principal offered her a regular classroom position on the second grade team instead. nancy believed that her main contribution to the team was her strengths in reading and writing, which was why she was selected to run the pullout reading program for the second grade students needing additional literacy support. this meant that she did not teach any of the science, but she still contributed to the team planning sessions and community of practice in action / park rogers 89 taught other disciplines that were observed (reading, writing, and mathematics). data collection i observed and collected field notes in two different settings: 1) during team planning sessions, and 2) in individual teachers’ classrooms. the team meetings were observed for 1½ hours each week for 10 weeks. they took place on tuesday and thursday afternoons and served two purposes. the tuesday meeting afforded the teaching team the opportunity to reflect on what they had taught at the end of the previous week, how they had carried the learning forward during the current week, and what considerations they needed to make in adapting their teaching for the remainder of the week. the thursday meeting acted as a checkpoint for the teachers; they often shared anecdotes about things their students said or did that may have shifted their thinking about their lessons for that week. copies of handouts and schedules were collected at these meetings to support my analysis of these field notes. the second setting for observation data was the teachers’ individual classrooms. the purpose of these observation periods was to gather data on how the teachers connected the ideas discussed in their team meetings and implemented them into their individual teaching practice. i observed the teachers’ classrooms during the same length of time as the team meetings (10 weeks). overall, i gathered observational data on two and a half science units, but focused the majority of her data gathering on the first 6 weeks with the changes unit, which examined changes in properties of matter and changes of state. during the final three weeks i spent the majority of my time observing the teachers’ classrooms during reading, writing, and mathematics lessons. it was during these last few weeks that i had the opportunity to observe nancy teach. finally, i used a standardized open-ended interview protocol (patton, 2002) as a second source of data. i conducted a single open-ended interview protocol with the principal before beginning the 10 weeks of observation for the purpose of gathering background information and to establish the context for the study. i also conducted two interviews with the teachers, one at the beginning and one at the end of the 10-week observation period. the questions in the first interview asked the teachers to describe past teaching experiences, their goals and methods for designing their curriculum, and the role of science in this design. the second interview protocol focused on how their approach to teaching science influenced their teaching of other subjects, and the role that their team teaching approach had on enacting their interdisciplinary design. international electronic journal of elementary education vol.3, issue 2, march, 2011 90 data analysis as a result of having employed case study method for data analysis, various themes emerged in response to the response to the four research questions. an integrated mode of examining these themes across participants’ stories resulted in developing a rich description of their shared experience (patton, 2002). using a single case study approach for data analyze afforded me the opportunity to examine and report on these teachers experiences as a unit rather than individually. the unit of analysis for this particular study was the team, with each member of the team contributing to my understanding of their community of practice. this method of data analysis assisted us in staying focused on the purpose of the study, which was to examine the role of team planning in developing the team’s shared understanding (supovitz, 2002) of what it means to teach inquiry-based science and use this understanding to design a coherent curriculum. following the observations of the team meetings and the changes unit, i wrote preliminary thoughts about the emerging themes observed in tracy’s, brenda’s, and heather’s instruction. this act of reflection (wolcott, 1995) allowed me to begin bracketing my personal views about developing a community of practice and made me aware of the team’s unique dynamics and the specific roles each member contribute to their community of practice. i employed a content analysis process on both the field notes and the interviews, a technique often associated with case studies. the content analysis process was inductive in nature and involved two phases: 1) aligning the teachers’ responses from what i observed and what the teachers stated in both sets of interview questions to the four research questions and 2) reviewing the teachers’ responses for patterns that we could then develop into assertions to answer the research questions. because this paper is part of a larger study, i focused my coding of interview data to comments that focused only on the role of team planning and i focused my analysis of the field notes to the team meetings mainly, using the observations of individual classroom teaching as a confirmation (through implementation) for what was discussed in the meetings. from my content analysis of the field notes and the interview data, three themes emerged: benefits, contributions, and commitment to professional development. the vignettes described in the next section address research question one mainly, as they depict two sample team meetings illustrating the team’s planning strategies and interactions with one another. following the vignette section, in the findings and interpretations, i discuss how the theme of benefits addresses research question two and three and the theme of contributions refer to research question number four. the final theme, commitment to professional development is also discussed with respect to community of practice in action / park rogers 91 how the teachers’ view the role of professional development in developing their community of practice, but is not related to one specific research question. setting the stage: a window into team meetings brenda, tracy, heather and nancy met regularly twice a week for an hour during their shared planning time on tuesday and thursday afternoons. the following two vignettes are representative of a typical tuesday and thursday meeting. they illustrate the role of each member and the kinds of conversations one could expect to hear during these meetings. a tuesday afternoon it is 2:30 and the second grade teachers are gathered in nancy’s classroom at her small group meeting table. they have their planning binders laid out in front of them and they are looking over what they have scheduled for the week. brenda initiates conversation with the question, “so tell me what you have been doing with writing?” nancy is the first to respond, saying that she used the read-aloud book they have been discussing in class to look at the detailed style of writing the author used. the team had participated in a book study the previous year that looked at lucy mccormick calkins’ and abby oxenhorn’s (2003) book small moments: personal narrative writing. each teacher was using the strategies from this book with their students. for example, their students select something they do in their daily life and write about that event. the goal is to have the students to go from a broad discussion of the daily event to a narrowed and detailed description of a brief moment within the event. nancy directs the conversation to publishing. for this piece of writing she wants the students to focus most of their time on revision writing rather than rushing to illustrate. so she is considering having her students complete a page that is folded in thirds instead a full booklet. she believes this will make the students focus on writing concisely and will leave less blank room. heather says that she likes that idea because she was also thinking of making the illustrations more of a side item in order to keep the students’ focus on improving their writing. however, she was thinking of having her students publish their small moment into a small booklet instead. brenda says she is still in the brainstorming phase of writing with her students and that they have not really caught on to the significance of the detailed writing that is needed to go from a broad concept to a small moment. she has not yet thought about how they are going to publish their writing but asked to see some examples from heather’s and nancy’s students when they are finished. as everyone else talks and shares their ideas tracy writes in her planning binder on this week’s schedule. she takes advantage of a brief international electronic journal of elementary education vol.3, issue 2, march, 2011 92 pause in the conversation to say to her colleagues that she is having similar difficulties as brenda and that she feels better knowing she is not alone. she too wants to place less emphasis on illustrations and have the students focus more on the publication of their writing. she asks nancy and heather if she can photocopy the booklet formats they are considering using with their students to help her think about what she might want her students to do. they all take a moment to write some notes down on their weekly plans. during this time the conversation starts to go off topic from planning their writing lessons to stories about their students. but brenda brings them back on task by asking nancy, “so what are you going to do again in writing tomorrow?” brenda asks heather the same question and heather looks back and forth between tracy and brenda as she explains how she is helping her students to move their small moment revisions forward. nancy interjects with a question about the writing prompt assessment that they need to give their students next week. tracy suggests doing it on monday so they can get it over with at the beginning of the week and not have it interfere with the rest of their week. brenda, heather and nancy all agree that this is a good idea. they block off the writing period for the assessment that day. tracy has to leave to pick her students up from the counselor. nancy, brenda and heather stay for another 20 minutes to talk about some other lessons they have used since last thursday and how they plan on building from those lessons for the remainder of this week. next the teachers start talking about the strategies they are working on with their students during making words. this is equivalent to spelling time in traditional classrooms. these teachers pull words from the content areas that follow similar spelling patterns and that students frequently encounter in their reading, writing, and speech. heather shares a lesson where she used the story bubbles popping as a word study about combinations of long ā sounds. she says that she selected this book because it discussed a lot of the same ideas that the students were experiencing in their science unit. in particular she described an activity with an alkaseltzer tablet where the students observed different ways to dissolve the tablet at different rates. she explains that the book reinforced some ideas about dissolving while also introducing students to a more extensive vocabulary they can use when recording evidence in their science journals. this connection between science and reading leads the teachers into a conversation about predictions. they share with each other different strategies that they are using in science to help the students develop predictions. they want to extend this beyond science and find ways to help their students become more comfortable with taking risks in making predictions in other content areas as well (e.g., reading response journals community of practice in action / park rogers 93 and math discussions). after a few minutes of sharing different techniques that each of them use, they pause to write some ideas down in their planning binders. heather asks brenda how her science class went that morning because she remembered from their last meeting that she was having some difficulties getting her students to develop questions. heather asks, “did they ask any questions? i am thinking i want to do the guided inquiry on thursday prior to the ice experiment because my kids have started asking some interesting questions and i think they ready to begin a more open inquiry approach.” brenda explains that her students’ questions are starting to get better, but that she ran out of time to ask them about their questions so they will not be ready for a more student-directed inquiry on thursday. heather says she might go ahead and start a more open approach to inquiry with them on thursday anyway, rather than doing the next lesson in their changes science unit. at 3:20pm they start to wrap things up because brenda, nancy and heather have to go pick up their students from the specialists and get them ready to for dismissal at 3:45pm. they each make some last minute notes in their planning binders. on their way out of the room, they discuss different materials that they would like to borrow from each other for the remainder of this week. a thursday afternoon it is 2:40pm and once again the four second grade teachers are gathered in nancy’s classroom around a small group discussion table. the conversation begins with brenda saying that she is planning on doing her writing prompt preparation with her students tomorrow for their assessment on monday. nancy chimes in, saying that they started some of this preparation today. she describes the team the mini-lesson that she did with their students. brenda then asks heather what she did in writing today, heather explains that she started some prompt writing today, but she is going to focus more on it tomorrow. tracy directs the conversation back to the small moment writing that they were doing at the beginning of the week. she explains that she has not had a chance to start this writing with her students. she is having difficulties getting her students to think from the broad concept to the more narrowed topic of a small moment. she has been thinking about how the others are approaching this writing style and that she is going to take a slightly different approach next week. for example, nancy had her students focus on the sequence of writing what occurs in a small moment, but tracy does not want to separate the show and tell part of the writing from the sequencing because this may be where some of the problems are coming from. she wants to try incorporating both sequencing and show and tell writing together. international electronic journal of elementary education vol.3, issue 2, march, 2011 94 after a brief pause, tracy changes the topic from writing to math. she explains to the others that she wants to start working with some of the ideas from chris confer’s (1994) book math by all means: geometry, grades 1-2 to supplement the district’s text. tracy says she really likes the hold and fold activity that confer suggests because it helps to develop students’ math vocabulary. brenda says that one of her favorite activities is rocket discovery because it deals with shapes within a shape. tracy concurs. heather and nancy like the idea of using this book. they suggest some other books that they could connect the literacy and math pieces. one of them raises the idea of using the book cloak for the dreamer (friedman, 1995) because of the discussion about shapes in the cloak design. the math planning conversation ends with talk about using ideas from confer’s book to decorate their classroom bulletin boards with a geometry theme. while everyone takes a moment to write in their planners, tracy changes the topic to science. she initiates this discussion with an explanation that the changes unit they have been studying in science is meshing well with their reading she is having her students look for changes in story lines. brenda reminds tracy to save the water from the ice melting activity from the changes unit to use for the evaporation activity next week. tracy responds, “o.k. are you planning on moving forward with some discussion on the water cycle for a couple of days next week?” brenda replies that she thinks they will take all of next week to cover the water cycle. brenda, tracy and heather discuss different books they can use to connect to the water cycle ideas they are going to be studying in science. tracy says that she wants to begin her reading with fictional books that have elements of the water cycle in them. as they progress with their study of the water cycle in science, she will draw connections between the stories and the science concepts. at this point all four teachers examine their reading books and begin to brainstorm how they can connect the books with the remainder of the changes unit. they find a couple of books that focus on character change. they talk for a few moments about how the idea of character change could be incorporated into reading and the small moments writing. this conversation carries on for about 10 minutes, until one of them realizes it is 3:30pm. they quickly pack up and go their separate ways to pick up their students to get ready for school dismissal at 3:45pm. findings and interpretations according to these teachers, regular team planning sessions were a necessary part of their teaching practice. based on our analysis of the data, we assert the following three claims as critical pieces to this team’s vision and development of their community of practice. first, the teachers believed the benefits of team planning outweighed the time spent; second, they community of practice in action / park rogers 95 valued the unique skills each member contributed to the group and as a result felt their teaching was stronger as a collective unit rather than individually; and third, each of them was committed to the idea that teachers need to be continually involved in professional development. for this team of teachers the bi-weekly team meetings were one way of ensuring they met this need for continuous professional growth. these three assertions are elaborated on below with embedded data provided to support each claim. benefits for some teachers having only a couple of planning periods a week is not enough time to do all they have to do, so using the little planning time that they have to meet with other teachers may seem counterproductive. however, the members of this teaching team said just the opposite. for example, tracy (interview 1) explained that “without the team our approach to teaching would look very different, because i would be responsible for pulling everything together myself”. tracy seemed to suggest that the team approach actually saved her time and helped her to implement the inquirybased curriculum she felt fit her teaching philosophy. the team meetings were not a burden on these teachers’ time, but the most efficient way for them to gather new ideas and resources for their teaching. besides the time factor, these teachers described the support they give each other as another benefit of their twice weekly team planning sessions. they described the purpose of the planning sessions as a constant check-in for them to make sure that they were staying true to their curriculum, meeting their objectives, and addressing the needs of their students. according to brenda, having the opportunity to meet regularly with her grade level colleagues ensured that she was reflective in her teaching practice. it is very beneficial whenever you can sit down together and brainstorm and figure out – o.k., this is working but this is not working out. ask each other “did this happen to you when you were doing this?” “think about trying this whenever you are doing this lesson”. just having the time to talk things out is important. (brenda, interview 1) in addition to encouraging reflection on their practice, the regular meeting times gave teachers the support they needed to take risks in their teaching and refine their ideas before putting them into practice. heather commented on this when she said, “having the team support allows you to try different things and take risks in our teaching. if you are alone you want to feel safe and secure, so instead of branching off with different ideas you may resort back to the manual more often”. brenda (interview 1) noted, “teachers get better at teaching when they work as a team. learning goes up when you are asking questions, talking and problem-solving with others”. international electronic journal of elementary education vol.3, issue 2, march, 2011 96 in summary, these teachers felt that their weekly tuesday and thursday meetings provided them with benefits that they could not get on their own. the meetings encouraged them to be reflective about their teaching practice, they provided an outlet to talk through problems and share strategies that worked, and increased their accessibility to resources (e.g., materials and teaching ideas). overall, these teachers viewed their scheduled time together as a benefit rather than a detriment to their teaching practice. contributions as the newest member of the team, nancy described the collaborative atmosphere of the second grade teachers as a vital part of her success in returning to the classroom after several years as a reading specialist. nancy acknowledged that, “without this team i’d be struggling more and would feel isolated. i wouldn’t be as reflective with my teaching nor would i be as willing to experiment with different teaching practices”. she went on to say, “this team is rare. we are well matched with respect to skills, we value each other’s strengths, and our personalities get along; we believe in each other professionally and personally” (nancy, interview 1). i asked each teacher to describe her contribution to the team. in each case they identified a different attribute. however, each of them explained that the reason they valued their planning time together was not because of what they offered but because of what they gained. as brenda (interview 1) noted, “just having the time to sit with three other experts that will help me plan things out is invaluable.” because of the respect they showed for each others’ expertise, i asked them to describe the contributions they felt each of their teammates offered. i learned that each person plays a specific role on the team. for example, brenda’s teammates described her as a manager, because she often initiated the discussion at the team meetings, kept the conversation on task, and was the first to provide suggestions when a teammate had an instructional question or problem. tracy was identified as the person who made curricular connections across the content areas. yet similar to brenda, she often would initiate the team’s conversation with curriculum questions. serving a slightly different role, heather was acknowledged as the organizer of the group, because she often took notes about their discussions and reminded them of special dates they needed to mark in their calendars (e.g., test dates and grandparents day). under heather’s title as organizer, she was also a resource person for different lesson ideas, especially those that integrated the disciplines. nancy’s expertise was undoubtedly her experience as a title 1 reading specialist. therefore, brenda, tracy, and heather all agreed that since science and mathematics were their strengths, nancy’s literacy background was a much-welcomed addition to the team. community of practice in action / park rogers 97 looking back at the two vignettes, one can see why these teachers identified each other with those particular characteristics. for example, in both vignettes brenda initiated the conversation and managed the conversation to ensure that everyone had an opportunity to share what they were doing, ask questions, or simply comment on someone else’s story. from time to time she also took responsibility for bringing the conversation back on task. for example, in the tuesday vignette, when the team started to go off task about planning their writing, brenda redirected the conversation with a question to nancy. brenda asked, “so what are you going to do again in writing tomorrow?” tracy’s role as curriculum connector was illustrated in the thursday vignette when she shared the connection between idea of changes they were studying in science and how she was having her students look for changes in storylines in reading. in this case, tracy drew from two different experiences to share with her teammates how her students were grasping the concept of change because of the connections she made in both disciplines. during the first vignette, heather’s role as team organizer focused more on her position as a resource provider. for example, heather shared a lesson with her teammates in which she used the book bubbles popping. throughout her description of this lesson, she explained how she used that book to connect to experiences the students had with the alka-seltzer activity in science, as well as how students used the vocabulary from that story in their science journals. a little later on in the tuesday vignette, heather’s organization skills were revealed once again when she referred back to a previous meeting and asked brenda if her students were starting to develop any of their own inquiry questions. this question served two purposes for heather: 1) she wanted to check back in with brenda to see how she was progressing with her students, and 2) heather planned ahead for her own lessons and wondered about division of materials with brenda’s and tracy’s classes. not only did nancy’s teammates view her expertise in literacy as a valuable contribution to the team, but nancy explained that it gave her a different perspective with which to consider how an inquiry-based approach to teaching meshed with disciplines other than science. she said, because i am not a scientist when i think of inquiry i see it through the lens as a reader or a writer. i see the same [inquiry] skills used in science also used in literacy, but i look at it from a writer’s perspective. so for example with poetry, what does inquiry look like in poetry? so bringing out a question for the [students] and then having them go investigate what things they are noticing as a writer. then having them come back and collaborate and talk about it as a group, what things they are noticing and sharing these…so i guess my perspective is a little bit different from the others. (nancy, interview 2) international electronic journal of elementary education vol.3, issue 2, march, 2011 98 because of this different perspective, most of nancy’s participation during the team meetings involved asking questions. also, since she did not teach the science curriculum, she focused much of her discussion on reading, writing, and mathematics. she contributed to the geometry discussion in the thursday vignette when she and heather suggested using the book cloak for a dreamer (friedman , 1995) in reading at the same time they were doing the geometry unit in mathematics. i observed these teachers portray consistent roles throughout the 10 weeks that i visited their team meetings. when they discussed classroom and curriculum issues, brenda usually initiated the conversation, tracy made curricular connections, heather organized their plans for action with taking notes and distributing resources, and nancy probed her teammates for their ideas and suggestions, as well as offered her assistance with literacy connections. although their time together was informal and fun, it was also productive because each member came to the table prepared to ask questions and share ideas. commitment to professional development this team’s dedication to professional growth was something that was evident throughout each team meeting. there was several times throughout my 10 weeks at drake elementary that i heard these teachers refer to strategies they had read about in a professional book study or learned about in a workshop they had attended. for example, the team used writing strategies described in the calkins and oxenhorn book (2003) small moments: personal narrative writing when they were planning for their next writing lesson. all four teachers were familiar with this book because they had studied it in their school’s professional book club the previous year. the principal explained that participation in book clubs was voluntary, but often all four members of the second grade team took part. this reflects their orientation toward teaching as one of continuous learning. for these teachers, professional development was an integral part of their teaching practice. according to both tracy and brenda, it was important for all teachers to think of their own learning as much as their students’. for example, brenda (interview 2) said, i think that it is really helpful [for teachers] if [they] are doing some kind of, not necessarily coursework, but something where [they] are reading, and have a group of people that [they] can talk with. for me it was coursework because that is what i love, but you know a book study or something like that [also works]. in these teachers’ minds, the time they spent together was just another form of professional development. the twice weekly meetings gave them the opportunity to gather ideas and resources just as in any other professional development program outside of their school. because of their like-minded community of practice in action / park rogers 99 commitment to professional development, they viewed each other as professional resources for their teaching. discussion and conclusions for this team of teachers, team planning was not a requirement mandated from the outside; they deemed it necessary for successful implementation of their inquiry-based approach to a connected and coherent curriculum. they viewed second grade at drake elementary school not as four separate classrooms, but as a single unit which they facilitated as a teaching team. their team approach served an important role in their instructional approach and offered several benefits to their teaching practice. their community was defined by their individual contributions to the team’s collaborative practice, which manouchehri (2001) also noted in her study is a critical component to any cop’s success and sustainability. the third finding from this study, each member’s commitment to professional development, provided some explanation as to how the team’s understanding of inquiry-based science developed and was mutually agreed upon by each member of the community as the foundation for designing a coherent curriculum. the conceptual framework informing this study was wenger’s (1998) notion of communities of practice (cop). wenger’s description of a cop was comprised of four components: community, meaning, practice, and identity. for the purpose of this study we do not discuss all these aspects of cop. instead we have chosen to elaborate two of these components – the teaching team’s practice and how they generate meaning within their community. these two components were most clearly illustrated through the findings of this case study. first, we focus on wenger’s description of practice in a cop. wenger stated that as “we interact with each other and with the world and we tune our relations with each other and with the world accordingly. in other words, we learn” (p. 45). wenger explained that a cop cannot withstand time if it is solely developed because a job requires it. brenda, tracy, heather, and nancy came together to learn from one another, and therefore formed their own cop. for them teaching was not only a way to earn a living, but a passion. it was this team’s sense of passion for teaching that constituted their community of practice. the teachers’ concept of practice was an experience that included both explicit and tacit meaning. their views of how to design inquiry-based lessons was something they explicitly discussed at their twice weekly meetings, but their shared view of what constitutes inquiry-based instruction was implied within their community of practice. wenger (1998) stated, “practice is about meaning as an experience of everyday life” (p. 52). to elaborate on this statement he argued that 1) meaning is located in a process called the negotiation of meaning, and 2) this international electronic journal of elementary education vol.3, issue 2, march, 2011 100 negotiation involves the interaction of two processes called participation and reification, which form a duality fundamental to the nature of practice. the teachers in this study were engaged in the process of negotiating meaning each time they met, whether it was during their scheduled team meetings or impromptu conversations over lunch. any given school day, these teachers were faced with various questions or problems requiring a negotiation of meaning based on both the explicit and tacit understandings of their cop. wenger’s second argument about participation and reification looked at the process of how a cop negotiates meaning. he described participation as sharing an experience with others in an activity or enterprise; therefore suggesting participation requires “both action and connection” (p. 55). reification is a process that wenger claimed is central to every cop. he defined reifying as taking something (e.g., an experience) that is abstract and making it into something (e.g., a meaning) that is concrete. using a wide range of reification processes (e.g., making, designing, representing, describing, perceiving, and interpreting) “human experience and practice are congealed into fixed forms and given the status of object” (p. 59). wenger described the participation and reification as a duality rather than opposites. with respect to this study, the teachers’ participation and processes of reification played an integral part in the negotiation of meaning for their cop. for example, based on their participation (action and connection with one another) and reification of such abstract concepts as inquiry and curricular connections, they developed a set of practices unique to their community. regarding silva’s (2000) findings on team teaching, the duality between participation and reification in the negotiation of meaning for a cop plays an important role in how a curriculum is implemented. with regards to the team in this study, the characteristics of their cop (e.g., commitment to professional development and individual expertise) guided their processes of participation and reification. in turn this led to both explicit and tacit negotiation of meaning about designing and implementing an inquiry-based coherent curriculum. implications the findings from this study have implications for educators working with both preservice and inservice elementary teachers. from this study we have learned that regardless of the number of years of teaching experience, all teachers need to seek ways to develop professional collaborative relationships as they can play a critical role in their own reflective practice. this can be especially important for elementary preservice teachers just learning to teach science as they often do not feel confident teaching science. requiring them to establish these kinds of relationships as students in their science methods classes may encourage them to look for similar community of practice in action / park rogers 101 relationships with colleagues during their induction years and onwards. the development of a cop that includes both new and experienced teachers may help to foster a more consistent inquiry-based science program throughout an elementary school. with regards to inservice teachers, many of the benefits discussed by the teachers in this study may benefit any experienced teacher. it is always important for teachers to challenge their own learning, and this is especially true for teaching science. the national science education standards (national research council, 2000) refers to quality science teaching as being inquiry-based. this kind of curriculum requires teachers to think beyond the cookbook steps of a textbook and engaging in questioning and exploration with their students. as the teachers in this study explained, a cop offers a safety net for teachers to question, debrief, and reflect with colleagues so they will be more willing to take the risks that an inquiry-based approach to teaching science sometimes requires. finally, there are also implications from this study for school administrators. for science to be valued in elementary classrooms, administrators need to learn first-hand the instructional strategies teachers are learning in professional development so they will give them the support (e.g., time to meet collaboratively) that they need to design and implement quality curricula that includes science. this case study contributes to the current body of literature on improving the quality of elementary science instruction. according to roden’s (2000) statement that science is viewed as a second-class core subject in elementary classrooms, it is clear that there is a need for providing classroom teachers with practical solutions for incorporating more science into their curriculum by drawing from the resources around them. the cop these four teachers had developed demonstrates the possibility for quality science in elementary classrooms when planning and implementing instruction for all subject areas that mirrors inquiry-based science teaching. • • • received: 22 july 2010 / revised: 22 october 2010 / accepted: 27 october 2010 biographical statement meredith park rogers received her ph.d. in curriculum and instruction, with an emphasis in elementary science education, from the university of missouri in 2006. she is currently an assistant professor of science education at indiana university and oversees the elementary science education program. her research interests include teacher knowledge, pedagogical content knowledge specific for teaching elementary science, and approaches to improving teacher professional development. international electronic journal of elementary education vol.3, issue 2, march, 2011 102 references akerson, v. l., cullen, t. a., & hanson, d. l. (2009). fostering a community of practice through a professional development program to improve elementary teachers’ views of nature of science and teaching practice. journal of research in science teaching, 46, 1-25. american association for the advancement of science. (1993). benchmarks for science literacy. new york: oxford university press, inc. appleton, k. (2007). elementary science teaching. in s. k. abell & n. g. lederman (eds.), handbook of research on science education (pp. 493-535). oxford, uk: taylor & francis. boyer, e. l. (1995). the basic school: a community for learning. princeton, nj: the carnegie foundation for the advancement of teaching. calkins, l. m., & oxenhorn, a. (2003). small moments: personal narrative writing. portsmouth, nh: heinemann. confer, c. (1994). math by all means: geometry, grades 1-2. sausalito, ca: math solutions publications. creswell, j. w. (1998). qualitative inquiry and research design: choosing among five traditions. thousand oaks, ca: sage publications, inc. duschl, r. a., schweingruber, h. a., & shouse, a. w. (eds.). (2007). taking science to schools: learning and teaching science in grades k-8. washington, dc: national academies press. feldman, a., divoll, k., & rogan-klyve, a. (2009). research education of new scientists: implications for science teacher education. journal of research in science teaching, 46, 442-459. foulger, t. (2005). innovating professional development standards: a shift to utilize communities of practice. essays in education, 14, 1-14. friedman, a. (1995). a cloak for the dreamer. new york: scholastic, inc. hung, d., chee, t. s., hedberg, j. g., & seng, k. t. (2005). a framework for fostering a community of practice: scaffolding learners through an evolving continuum. british journal of educational technology, 36, 159-176. lave, j., & wenger, e. (1991). situated learning: legitimate peripheral participation. cambridge, uk: cambridge university press. lumpe, a. (2007). research-based professional development: teachers engaged in professional learning communities. journal of science teacher education, 18, 125128. manouchehri, a. (2001). collegial interaction and reflective practice. action in teacher education, 22(4), 86-97. national research council. (2000). inquiry and the national science education standards: a guide for teaching and learning. washington, d.c.: national academy press. palincsar, a. s., magnusson, s. j., marano, n., ford, d., & brown, n. (1998). designing a community of practice: principles and practices of the gisml community. teaching and teacher education, 14, 5-19. community of practice in action / park rogers 103 park rogers, m., & abell, s. (2006, april). achieving a coherent curriculum in second grade: science as the organizer. national association for research in science teaching (narst), san francisco, ca. patton, m. q. (2002). qualitative research and evaluation (3rd ed.). thousand oaks, ca: sage publications. raizen, s. a., & michelsohn, a. m. (eds.) (1994). the future of science in elementary schools: educating prospective teachers. san francisco, ca: jossey-bass. roden, j. (2000). primary science: a second-class core subject? in j. sears and p. sorensen (eds.), issues in science teaching (pp.31-40). london: routledgefalmer. rutherford, j. f., & ahlgren, a. (1990). science for all americans. new york: oxford university press, inc. silva, d. y. (2000). collaborative curriculum encounters. journal of curriculum and supervision, 15, 279-299. supovitz, j. a. (2002). developing communities of instructional practice. teachers college record, 104, 1591-1626. tilgner, p. j. (1990). avoiding science in the elementary school. science education, 74, 421431. wenger, e. (1998). communities of practice. new york: cambridge university press. wesley, p. w., & buysse, v. (2001). communities of practice: expanding professional roles to promote reflection and shared inquiry. topics in early childhood special education, 21, 114-123. wolcott, h. f. (1995). the art of fieldwork. walnut creek, ca: altamira press. microsoft word 13_iejee_4_1_mcnamara_ozuru_floyd international electronic journal of elementary education, 2011, 4(1), 229-257. issn:1307-9298 copyright © iejee www.iejee.com comprehension challenges in the fourth grade: the roles of text cohesion, text genre, and readers’ prior knowledge danielle s. mcnamara ∗∗∗∗ arizona state university, united states yasuhiro ozuru university of alaska anchorage, united states randy g. floyd the university of memphis, united states abstract we examined young readers’ comprehension as a function of text genre (narrative, science), text cohesion (high, low), and readers’ abilities (reading decoding skills and world knowledge). the overarching purpose of this study was to contribute to our understanding of the fourth grade slump. children in grade 4 read four texts, including one high and one low cohesion text from each genre. comprehension of each text was assessed with 12 multiple-choice questions and free and cued recall. comprehension was enhanced by increased knowledge: high knowledge readers showed better comprehension than low knowledge readers and narratives were comprehended better than science texts. interactions between readers’ knowledge levels and text characteristics indicated that the children showed larger effects of knowledge for science than for narrative texts, and those with more knowledge better understood the low cohesion, narrative texts, showing a reverse cohesion effect. decoding skill benefited comprehension, but effects of text genre and cohesion depended less on decoding skill than prior knowledge. overall, the study indicates that the fourth grade slump is at least partially attributable to the emergence of complex dependencies between the nature of the text and the reader’s prior knowledge. the results also suggested that simply adding cohesion cues, and not explanatory information, is not likely to be sufficient for young readers as an approach to improving comprehension of challenging texts. keywords: comprehension, fourth grade slump, cohesion, genre, domain knowledge, reading, individual differences, coherence, construction integration ∗ danielle s. mcnamara, department of psychology senior research scientist, learning sciences institute arizona state university, p.o. box 872111, tempe, arizona 85287-2111, united states. e-mail: danielle.mcnamara@asu.edu international electronic journal of elementary education 230 introduction a good deal of research has been conducted and has contributed to our understanding of how children learn to decode words and the factors that influence young readers’ ability and inability to decode words (cain, oakhill, & bryant, 2000; ehri, 1991, vellutino, scanlon, & spearing, 1995). reading decoding represents the ability to apply letter-sound correspondence rules when reading words and non-words. scholars have postulated that slow or inaccurate decoding skills tax working memory resources, using up working memory capacity needed for other comprehension processes such as integrating information across sentences (cain, oakhill, & bryant, 2004; hannon & daneman, 2001; perfetti, 1985). indeed, slow or inaccurate word decoding has a profound impact on the reading comprehension success (lyon, 2002; vellutino, 2003). there has also been a growing realization that children’s ability to decode the words in text does not paint a complete picture of children’s ability to comprehend text (e.g., cain et al., 2004; oakhill, cain, & yuill, 1998). the ability to decipher a word is not the same as the ability to interpret a sentence, understand the relationship between sentences, and to interpret the global meaning of a text (oakhill, cain, & bryant, 2003). successful comprehension also requires the reader to integrate individual word meanings into a coherent sentence level representation and to integrate sentences to create a global understanding. as such, successful reading comprehension requires the efficient coordination and integration of a number of underlying processes. these processes include not only word decoding and parsing sentences, but also integrating information within a text and with prior world knowledge (kintsch, 1988, 1998; perfetti, 1985). more research now turns to developing a better understanding of children’s comprehension processes (cain et al., 2004; cote & goldman, 1999; kendeou, van den broek, white, & lynch, 2009) as opposed to decoding processes. this project is intended to contribute to this understanding by examining the effects of both person-related and textrelated factors on children’s text comprehension. thus, we examine two factors related to children’s abilities (i.e., decoding skill and knowledge) and two factors related to text (i.e., cohesion and genre). our research targets children in grade 4 because there is some evidence that children at that age are at a critical period in reading development characterized by an emergence of comprehension difficulties. this has been referred to as the fourth grade slump (meichenbaum & biemiller, 1998; sweet & snow, 2003). our goal here is to examine the relative impact of the four factors related to children’s ability and text characteristics so that we can more fully understand the potential problems leading to a fourth grade slump. the guiding premise of this research is that world knowledge deficits, which are the negative gaps between the reader’s actual knowledge and the knowledge demanded by a text to understand the text, are significant contributors to potential problems occurring when children reach the fourth grade. thus, we examine here effects of knowledge on text comprehension, particularly in concert with characteristics of the text that influence the amount of knowledge required to understand the text. we further assume that genre and cohesion are two aspects of text that principally contribute to the degree of knowledge required to understand texts. based on these premises, we expect that readers generally understand (a) narrative text better than science text and (b) text with high cohesion better than text with low cohesion. in addition, we hypothesize the presence of an interaction between level of knowledge and text characteristics such that the benefit of knowledge is more pronounced for text with higher demands on knowledge. thus, our specific predictions for the interactions are that comprehension challenges in the fourth grade / mcnamara, ozuru & floyd 231 the benefits of knowledge were expected to be more pronounced for science texts and for texts with more conceptual gaps (i.e., low cohesion texts). we explain these assumptions in greater detail in the following sections. text comprehension our expectations are primarily based upon the construction-integration (ci) model of text comprehension (kintsch, 1988, 1998). according to this theory, and indeed most theories of text comprehension (graesser, singer, & trabasso, 1994; van den broek, rapp, & kendeou, 2005), a critical process of successful comprehension is the retrieval of information from knowledge that is not explicitly stated in the text. according to the ci model, text comprehension has multiple levels, including a surface level representation of the words and syntax, and a textbase level that represents the meaning of the text. we hypothesize that, ultimately, the most important level of representation for comprehension that these children often struggle to construct is the situation model, which involves the integration of the textbase with knowledge. comprehension is assumed to be more successful and deeper if the reader activates relevant knowledge and integrates that knowledge with the information explicitly stated in the text. in essence, text comprehension is more successful when the reader generates inferences while reading (vidal-abarca, martinez, & gilabert, 2000; wolfe & goldman, 2005). of course, successful comprehension is also largely dependent on the first two levels of comprehension. if the reader does not successfully form a surface-level representation, then the reader will be highly unlikely to form a coherent textbase. that is, if the reader does not decode the words or parse the sentences, the reader’s surface level of comprehension will be deficient, and by consequence, the textbase will likely to be incoherent or malformed. if that is the case, then the activation of relevant knowledge and a coherent situation model representation are unlikely. in sum, the situational model generally builds upon the textbase and surface representations (unless, of course, the reader’s understanding comes solely from prior knowledge and not from the text). thus, the integration of knowledge with the textbase understanding requires sufficient decoding skill for a textbase to be formed. however, what is critical here is the notion that the contribution of decoding skill and knowledge work differently. that is, whereas decoding is fundamental to comprehension of texts across all genres with different features, the contribution of knowledge to comprehension is likely to vary depending on text genre and text features (in particular, text cohesion). subsequent sections describe how knowledge contributes to comprehension depending on text genre and text cohesion. text genre and world knowledge our focus regarding the influence of text genre is on the distinction between narrative and expository texts, in particular science texts. as discussed in the previous section, world knowledge plays a critical role in deep-level comprehension of texts because readers must use knowledge to integrate meanings of individual sentences into a coherent representation of situations or events depicted by the overall text (kintsch, 1988, 1998). as such, whether readers can develop a deep-level comprehension of the overall text meaning is likely to be affected by text genre. narrative texts usually present reoccurring topics (e.g., friendship, love, and parting with a friend) in a specific context involving particular characters, settings, and times. readers often have extensive experience and knowledge (i.e., schemas) regarding the events and situations described in typical narrative texts. although narrative texts may contain new information (i.e., unfamiliar location, characters, and specific actions), most children have, from first-hand experience, well-developed schemas about the settings, actions and events described by narrative texts (nelson, 1996; olson, 1985). thus, most international electronic journal of elementary education 232 children possess adequate event related knowledge to comprehend narrative texts. moreover, many narrative texts also follow a simple structure—a sequence of casually related events for which many elementary school children are familiar (williams et al., 2005). in contrast to narrative texts, expository texts often place greater processing demands on the reader due to their increased structural complexity and increased demands for domainspecific information. expository texts often contain abstract and logical relations that can be difficult to interpret, especially for children in the third to fifth grades (kamberelis & bovino, 1999). perhaps most importantly, expository texts introduce many concepts that are new or only partially understood by the reader. indeed, expository texts are used for the purpose of acquiring new information, and thus, they often contain novel content for young school children who are beginning to learn about those content domains, such as science. if children lack previous knowledge about a particular domain, comprehension will be limited because they do not possess the knowledge structures to which the new information can be integrated and assimilated (langer, 1986). the link between knowledge and expository text comprehension is well supported by previous research with adolescents and adults (afflerbach, 1986; chi, feltovich, & glaser, 1981; mcnamara & kintsch, 1996) and elementary school children (best, floyd, & mcnamara, 2008; rupley & wilson, 1996). thus, one possible interpretation for the emergence of comprehension difficulties around the fourth grade is that children lack sufficient prior knowledge to comprehend expository texts that are introduced during this period. whereas early elementary school reading instruction focuses on the development of fundamental reading skills (i.e., learning to read), reading goals shift toward reading to learn in the third and fourth grades. thus, up until the fourth grade, children tend to read narrative texts for the purpose of learning to read. however, as they transition from narrative text to expository texts to move to reading to learn during the third and fourth grades, and particularly in the fourth grade, knowledge levels may become the most critical influence on their comprehension. text cohesion and world knowledge the effect of world knowledge on reading comprehension is also likely to be regulated by the manner in which reading materials are written. this issue is very important because both narrative and science texts can be written in different ways that might affect comprehension. the notion of text cohesion is one of the most useful concepts to systematically represent text characteristics that affect comprehension in a theoretically meaningful way. text cohesion represents the extent to which a text explicitly provides background information and cues to help readers relate information distributed across different parts of the text (britton & gulgoz, 1991; graesser, mcnamara, & louwerse, 2003). cohesive elements in a text are grounded in explicit linguistic elements (i.e., words, features, cues, signals, constituents) and their combinations (graesser & mcnamara, 2010). texts are considered to be low cohesion when constructing a coherent representation from the text requires many inferences based on reader’s knowledge. texts are considered high cohesion when elements within the text provide more explicit clues to relations within and across sentences (mcnamara, louwerse, mccarthy, & graesser, 2010). as such, lowcohesion texts place greater processing demands on the reader, in particular for readers with low levels of background knowledge. previous research indicates that many expository materials written for school children have low levels of cohesion. for example, beck, mckeown, and gromoll (1989) performed an extensive analysis of four elementary school social studies texts and found that the texts comprised unclear goals and poor explanatory comprehension challenges in the fourth grade / mcnamara, ozuru & floyd 233 links and assumed too much knowledge on the part of readers. thus, deficits in prior knowledge are likely compounded by exposure to low-cohesion texts. in support of that hypothesis, a series of studies conducted by mcnamara and colleagues (mcnamara, 2001; mcnamara & kintsch, 1996; mcnamara, kintsch, songer, & kintsch, 1996; o’reilly & mcnamara, 2007) indicates that the effects of domain knowledge in the comprehension of expository materials are moderated by text cohesion for middle school children and adults. across these studies, the authors modified texts so that participants either read low-cohesion or high-cohesion versions of the same text. to form the highcohesion texts, the low-cohesion texts were modified by adding surface-level indicators of relations between ideas in the text. such modifications range from adding low-level information, such as identifying anaphoric referents, synonymous terms, connective ties, or headers, to supplying background information left unstated in the text (beck, mckeown, omason, & pople, 1984; beck, mckeown, sinatra, & loxterman, 1991; britton & gulgoz, 1991; for a review see mcnamara et al., 2010). when consecutive sentences overlap conceptually, the reader is more likely to be successful in forming a coherent representation linking the meaning of the two or more sentences. likewise, when relationships between ideas in the text are explicit by using connectives such as because, consequently, therefore, and likewise, the reader is more likely to understand the text content better. these studies (mcnamara, 2001; mcnamara & kintsch, 1996; mcnamara et al., 1996; o’reilly & mcnamara, 2007) indicated that increased cohesion consistently facilitated comprehension for readers, in particular those with low levels of background knowledge. it was concluded that low-knowledge readers cannot easily fill in gaps in low-cohesion texts because they do not have the knowledge to generate the necessary inferences. therefore, these readers need high-cohesion text to understand and remember the content. these studies also demonstrated a reverse cohesion effect, showing benefits from low-cohesion text for readers with high level of knowledge. demonstrations of reverse cohesion supported the assumption that less cohesive texts force high-knowledge readers to generate knowledge-based inferences to bridge cohesion gaps present in the text, thus resulting in further integration of text information with pre-existing knowledge. we would like to emphasize that this gap-filling process can be successful only if readers have the sufficient amount of background knowledge that can be accessed or triggered based on limited textual information. in this study, this gap-filling inference based on pre-existing knowledge would be most likely to occur for high knowledge readers’ reading low cohesion narrative texts. it would be unlikely to occur for low-cohesion science texts because the level of background knowledge is still too low to afford such gap-filling inferences even among relatively high-knowledge students. we hypothesize that (most) grade 4 children will not have a sufficient knowledge base in science to automatically generate inferences when reading low cohesion science text. however, some grade 4 students are expected to have sufficient world knowledge relative to narrative texts. thus, we expected that high-knowledge students would show a reverse cohesion effect for the narrative texts. in all other cases, we expected to find an advantage for the higher cohesion text. present research the overarching goal of this research is to further the understanding of the factors that lead to comprehension difficulties among elementary school children entering the period associated with the fourth grade slump. in light of this goal, we examine the roles of reading decoding skill and world knowledge among children in the fourth grade when exposed to texts from different genres (narrative and expository) and different levels of cohesion. international electronic journal of elementary education 234 our study examined elementary-school children’s comprehension of narrative and expository texts used in the classroom. we examined separately the effects of knowledge and word decoding skill, and how the effects of text characteristics (i.e., genre and cohesion) depend on knowledge or decoding skill. we expected to find significant effects of both knowledge and decoding skill on comprehension across different texts. we further expected that children would encounter greater difficulty comprehending the science texts and low cohesion texts, both of which are more knowledge demanding. most importantly, we hypothesized that comprehension would depend on both knowledge and the characteristics of the text. we predicted that comprehension of science texts, in contrast to narrative text, would depend on world knowledge. in contrast, we did not expect such an interaction between children’s decoding skills and text types. specifically, an interaction was expected between text genre and the readers’ level of knowledge, wherein knowledge has a greater effect on expository text comprehension than narrative text comprehension. decoding skill was expected to benefit comprehension, but a differential effect of decoding skill as a function of text genre was not predicted. we also hypothesized that the effects of text cohesion would depend on both knowledge and text genre. specifically, we predicted an advantage for low cohesion texts (i.e., a reverse cohesion effect) when the texts were relatively familiar (i.e., narratives) and when the reader had sufficient knowledge to fill in the cohesion gaps (i.e., high knowledge readers). in all other cases, we expected to find an advantage for higher cohesion text. method participants participants included 65 children enrolled in the fourth grade at four public schools in a large metropolitan school district. children ranged in age from 9 years, 2 months to 11 years, 2 months (m = 118.30 months, sd = 5.35 months). girls composed 52.3% of the sample (n = 34), and boys composed 47.7% (n = 31). of the sample, 54% were caucasian (n = 35), 40% of the children were african american or black (n = 26), and 3% were hispanic (n = 2). all children but two spoke english as their primary language. using parent education level as an index of socioeconomic status, 2% of fathers did not complete high school; 49% of mothers and 55% of fathers graduated from high school, completed some college, or completed technical school; and 49% of mothers and 35% of fathers obtained at least a college degree. on two screening measures, children in this sample demonstrated vocabulary knowledge and listening comprehension skills that were somewhat above average for their age. the average performance on the woodcock–johnson iii (wj iii) tests of achievement (ach) picture vocabulary test (woodcock, mcgrew, & mather, 2001) was 108.2 (sd = 10.7), and average performance on the wj iii ach oral comprehension test was 108.0 (sd = 9.5). because the population means and standard deviations for these tests are 100 and 15, respectively, these results also indicate that the participating children displayed, on average, somewhat less variability than expected of the population of children this age. design the experimental design of the study was a 2 x 2 within-subjects design. the within-subjects factors were text genre (narrative and expository) and text cohesion (low cohesion and high cohesion). materials texts. there were eight texts used in this study. four texts were original texts obtained from basal readers and science textbooks. these texts were considered low-cohesion texts. the comprehension challenges in the fourth grade / mcnamara, ozuru & floyd 235 four remaining texts were revised versions of the four low-cohesion texts that were manipulated to increase their cohesion (see appendix 1). the four low-cohesion texts included two science and two narrative texts. they were drawn from a pool of 127 texts collected from elementary-school-age basal readers and science textbooks. these texts included 67 science texts, 53 narrative texts, and 7 science texts written in narrative format. the average number of words per text was 388 (sd = 167, min = 115, max = 991; science = 388; narrative = 399; mixed format = 298), and the average flesch-kincaid reading level was 3.86 (sd = 1.79, min = 0, max = 7.8; science = 4.96; narrative = 2.52; mixed format = 3.53). from this pool of 127 texts, we selected two narrative texts and two expository texts. these four texts were chosen because they were representative of the text pool and closely equated in terms of number of words and flesch–kincaid grade level. these indices were derived using coh-metrix, version 1.0 (graesser, mcnamara, & louwerse, & cai, 2004). preliminary selection criteria for inclusion in this study were for flesch–kincaid grade level to be between 2.0 and 5.0 and text length to be within the range of 304 and 471 words. from the text pool, we selected two science texts, heat (sra’s real science, grade 2: elementary science) and needs of plants (mcgraw–hill’s science, grade 2), and two narrative texts, moving (mcgraw-hill reading, grade 3) and orlando (addison wesley’s phonics take-home reader, grade 2). the cohesion of the four original texts was then manipulated to increase their cohesion. each of the two texts within each genre included one original, lower-cohesion version and one higher-cohesion version. the aim of the cohesion manipulations for the high-cohesion versions was to increase cohesion between concepts and ideas such that they created a clear situation model for the child. the basic concepts discussed in the low-cohesion and highcohesion texts were the same (e.g., heat moves through objects). however, the understandability of the high-cohesion versions increased by cohesion cues. there were a number of cohesion manipulations made to the high-cohesion texts that were fitting for each of the four texts. we increased cohesion using methods previously found to enhance comprehension (e.g., beck et al., 1991; mcnamara et al., 1996), including manipulations to referential, temporal, causal and explanatory cohesion. specifically, there were seven aspects of the text that were modified to increase cohesion: (a) replacing pronouns with noun phrases, (b) adding descriptive elaborations, (c) adding sentence connectives, (d) replacing or inserting words to increase conceptual overlap, (e) adding topic headers, (f) adding theme sentences, and (g) moving or re-arranging sentences to increase temporal or referential cohesion. for example, if events were not presented in chronological order, the presentation of the events was altered to match the chronological order of the events in the world. the aim was to alter the texts so that they approximated equivalent levels of cohesion as measured by the coh-metrix, version 1.1 (graesser et al., 2004). the tool automatically analyzes texts on over 50 types of cohesion relations and over 200 measures of language and discourse by applying modules that use lexicons, classifiers, syntactic parsers, shallow semantic interpreters, conceptual templates, latent semantic analysis, and other components widely used in computational linguistics. the high-cohesion versions of the texts included explicit information about the meanings of particular terms, a greater number of noun phrases, and a greater number of causal connections. the following example taken from the plant texts illustrate the ways in which cohesion was added. in this example, the high-cohesion version adds a sentence explaining that a mineral is not a plant or an animal. the third sentence includes a connective term “instead.” low cohesion. “plants also need minerals. a mineral is a naturally occurring substance that is neither plant nor animal.” international electronic journal of elementary education 236 high cohesion. “plants also need minerals. a mineral is not a plant or an animal. instead, a mineral is a substance in the ground that occurs naturally.” cohesion manipulations also involved creating a context so that the child could more easily interpret the situations described in the text. the following example taken from the opening sections of the orlando texts illustrates an instance in which a context was created for the high-cohesion version of the orlando text. it is also important to point out that the order in which information was presented was changed for the orlando text such that the high-cohesion version provided greater temporal cohesion, that is, information was presented in the order in which events occurred; the low-cohesion version on the other hand presented information in a non-temporal order and thus the reader was to infer that information presented at the start of the story was not the first event to occur. low cohesion. “salvador was upset. he told is mama he was going out. he didn’t want to be worried or sad.” high cohesion. “once upon a time there was a boy. his name was salvador. salvador adored his pet pig named orlando.” a further method for increasing text cohesion was to integrate information across sentences in the low-cohesion text to provide a clearer depiction of the situations described by the text. the following example taken from the heat texts illustrates an instance in which information was integrated. low cohesion. “most metals are good conductors. metal pots are used for cooking. heat from the stove quickly moves through the metal. the heat warms the food.” high cohesion. “most metals are good conductors. for example, metal pots are used for cooking because heat from the stove quickly moves through the metal pots and the heat from the pot warms the food.” table 1. select characteristics of the narrative and expository texts narrative expository orlando moving plant heat low high low high low high low high argument overlap 0.35 0.62 0.53 0.85 0.78 0.86 0.66 0.84 number of words 451 547 437 584 466 637 404 521 average sentence length 6.44 9.77 12.49 14.60 10.13 10.98 7.21 11.33 grade level 2.20 3.86 4.02 4.82 3.76 3.77 2.67 4.93 word frequency (min log) 1.85 1.54 1.86 1.83 1.46 1.48 1.50 1.40 table 1 presents some of the main text characteristics for the eight texts. these features are argument overlap, number of words, average sentence length (i.e., average number of words in a sentence), flesch-kincaid grade level, and average of word frequency of the lowest word frequency word in each sentence (logarithm). the argument overlap scores, which relate to the proportion of adjacent sentences that share one or more arguments (e.g., pronoun, noun or noun phrase), were higher for the high-cohesion versions of the texts. incorporating words and phrases to increase cohesion has an effect on text length, such that the high-cohesion versions of the texts comprised more words than the low-cohesion versions. also, adding cohesion changed the number of sentences and sentence length as indicated in the table. in addition, manipulating texts for cohesion affected the grade level comprehension challenges in the fourth grade / mcnamara, ozuru & floyd 237 scores, such that the high-cohesion versions were estimated as having higher grade-level scores than low-cohesion versions. grade level assignments are primarily based on factors such as the number of words in the sentences and the number of letters or syllables per word (i.e., as a reflection of word frequency). thus, adding words and sentence length (which is generally necessary to increase cohesion), increases grade level scores. however, the cohesion manipulation did not greatly influence content-based difficulty as indicated by word frequency. comprehension measures. comprehension was assessed using a combination of recall tasks and multiple-choice questions. multiple measures have the benefit of providing a more thorough evaluation of the breadth and depth of comprehension. we collected free recall and cued recall data to assess children’s comprehension of the texts. the free recall task required children to report what they remembered about the passage they had just read. each child was provided the following directive: “tell me everything you can remember about what you have just read. give me as many details as possible, like you were trying to tell a friend about what you just read.” all responses were recorded on an audiotape and later transcribed. cued recall, which assessed major themes in the text, was used to evoke richer content from the children (zinar, 1990). the cued recall task required children to respond to three directives. the directives were designed to assess comprehension of three major sections of the texts, and they essentially covered the entire text. for example, for the orlando text, children were directed to (a) “tell me everything that salvador did after his mama told him they would have to sell orlando,” (b) tell me everything that salvador’s mama said at first about what they needed to do with orlando,” and (c) “tell me everything that happened to salvador and what he did after the storm began.” all responses were audiotaped and later transcribed. twelve multiple-choice questions were constructed for each text to assess students’ comprehension. six of the questions were designed to tap local-level comprehension, and the other six were designed to tap global-level comprehension of the text. whereas locallevel questions requested information that was within five or fewer clauses (mostly within 2 sentences), global-level questions requested information that was located across six or more clauses. each multiple-choice question had four answer options with only one being the correct answer. examiners read the questions orally while the questions were presented in text form. the children were required to vocalize the correct answer. comprehension scores for each text were obtained for each child by calculating the proportion of correct responses to total questions (i.e., 12). reading competency measures as part of a larger battery of assessment instruments, children completed two tests from the wj iii ach (woodcock et al. 2001). the tests from the wj iii ach included the word attack test and the academic knowledge test. word attack measures reading decoding skills. examinees must pronounce phonologically regular non-words. the test has a median internal consistency reliability coefficient of .94 for ages 8 to 10 (mcgrew & woodcock, 2001). academic knowledge measures knowledge about the biological, physical, and social sciences and the humanities. examinees must provide information about the biological and physical sciences; about history, geography, government, and economics; and about art, music, and literature. the test has a median internal consistency reliability coefficient of .84 for ages 8 to 10 (mcgrew & woodcock, 2001). for both wj iii tests, age-based standard scores international electronic journal of elementary education 238 (m = 100, sd = 15) were obtained. for word attack, the standard score represents decoding skills. the standard score for the academic knowledge tests represents world knowledge. procedures recruitment. children were recruited by sending letters of invitation to parents through the children’s school classrooms. the letters provided information about the study and requested that parents contact researchers to schedule a testing session. testing sessions were conducted on five saturdays during fall 2003. after completion of the testing, children were provided a gift card to a department store, coupons from merchants, and school supplies. testing. an assessment battery was completed in approximately a 2-hour testing session. graduate students who had successfully completed a graduate course covering the administration of standardized tests completed all testing. children first silently read each text within a 5-minute period. after reading the first text, the text was removed from view before answering the free and cued recall questions and 12 multiple-choice questions. this process was repeated with the remaining four texts. the cohesion manipulation was organized such that children either read the high-cohesion or low-cohesion version of each text. the cohesion manipulation was counterbalanced so that an equal number of children read high-cohesion and low-cohesion versions of each of the four texts. the order of texts was counterbalanced using a latin-square design. finally, after reading the texts and answering the recall and multiple-choice questions, children completed the battery of reading competency tests. coding recall data. the analysis focused on the amount of information children recalled about information in the text by counting the number of propositions recalled for each text. it is important to note that we matched all recall to ideas contained in the low-cohesion versions of the texts so we could evaluate whether reading the high cohesion versions of a texts that contained the same ideas in a more understandable way, increased recall for the main ideas contained in the low-cohesion versions of the text. thus, the number of propositions in the low-cohesion texts provided a benchmark for which we could compare the amount of recall that children generated from the low-cohesion and high-cohesion versions of the texts. there were two steps to the recall analysis. first, the low-cohesion versions of narrative and expository texts were propositionalized using a conventional method in which the information contained in each sentence was broken into main propositions and subpropositions (see kintsch, 1998). main propositions consisted of the main idea, whereas the sub-propositions contained details pertaining to the main idea. for instance, the main proposition for the sentence “plants need sunlight, air and water to live” consisted of the notion that plants need things to live. the sub-proposition consisted of the notion that plants need sunlight, and water. the number of main propositions mapped on to the number of sentences contained in each low-cohesion text (e.g., 45 propositions for the plant text). second, the children’s transcribed recall data were divided into idea units. idea units were classified as utterances that contained a subject, verb, and direct object. idea units were separated by connectives, such as so, and, but, and because. every idea unit was matched to the propositions. in cases where children repeated information, each idea unit was counted only once. our initial analysis also focused on inferences children generated about information in the text (i.e., information that was extrapolated from but not directly specified in the texts). using previous research as a guide (e.g., kintsch, 1993), inferences were classified as textbased, elaborative, global, or irrelevant. however, we did not further analyze inference data comprehension challenges in the fourth grade / mcnamara, ozuru & floyd 239 because too few inferences were produced (inferences comprised only 3% of cued recall question answers and 1% of free-recall question answers). all the free recall and cued recall was coded by two trained raters. half the data was coded by a third trained rater. inter-rater reliability was evaluated for all dimensions of coding between the third rater and each of the two raters. simple agreement and kappa analyses indicated that agreement reached 90% or above on all dimensions, which indicates a high level of agreement. disagreements were resolved by discussion between raters. proposition analysis. the recall analyses assessed the number of propositions recalled. to account for the completeness of information recalled, a value of 1.0 was assigned to recall that contained the main proposition and more detailed information cited in the subproposition. a value of 0.5 was assigned to recall data that contained the main proposition but that did contain the detailed information in the sub-proposition. a value of 0 was assigned when the proposition was not recalled. for example, a value of 0.5 was assigned for the sentence “plants need sunlight, water, and air to live” when a child stated that plants need water to live. a value of 1 was assigned when the child stated that plants need water, sunlight, and air. because children’s propositional recall sometimes contained erroneous information, recall that contained such information received a score of 0.5. for example, a child may have erroneously stated, “plants do not need sunlight.” once recall scores were totaled for a text, they were transformed into a proportion because there were an unequal number of propositions contained in the four texts. free recall and cued recall propositions were summed into separate scores to focus on different dimensions of comprehension across tasks. thus, it was possible for children to recall the same information in the free recall task and in the cued recall task. for the free recall analysis, proportion scores were calculated by dividing the free recall score by the number of propositions that could potentially be recalled for each text. for the cued recall analysis, the recall scores were summed for each of the free recall questions and divided by the number of directly relevant main propositions that could be potentially recalled (for each question). to determine which sentences contained directly relevant information, two experimenters identified propositions that directly related to the cued-recall directives from three categories: directly related sentences, indirectly related sentences, and irrelevant sentences. there were 32 main propositions directly relevant to the plant directives, 52 to the heat directives, 28 to the moving directives, and 35 to the orlando directives. note that the same proposition was sometimes classified as directly relevant for more than one cued-recall question; in such cases, the same proposition was counted for each of the relevant questions. kappa analyses showed that there was a high level of agreement between raters across both texts (weighted kappa = .85 expository texts and .85 for the narrative texts). disagreements were resolved by discussion. results preliminary analyses performance on the comprehension questions for the two narrative texts (orlando and moving) and for the two expository texts (plants and heat) were compared to verify whether there was an effect of text within genre. there were no statistically significant differences in children’s performance comparing the two narrative texts on the multiple-choice, free recall, or cued recall tasks. there were no differences on children’s performance between the two expository texts for the free recall and cued recall tasks. however, on the multiple-choice questions, children performed better on the plant questions (m = .60, sd = .22) than on the international electronic journal of elementary education 240 heat questions (m = .52, sd = .21), t(64) = 3.96, p < .01. we conducted item analyses to examine whether these effects were due to a subset of the multiple-choice questions, but superior performance was evident for the majority of questions for the plants text. we further confirmed that this effect may have been due to the ease of the questions (and not due to differences between the texts) because the flesch–kincaid grade level for the multiple-choice questions for the heat text (m = 2.99, sd = 1.77) was somewhat, though not significantly, higher than the plant text (m = 1.94, sd = 1.29), f(1, 22) = 2.751, p = .111, indicating that questions for the heat text are somewhat more difficult than questions for the plant text. these effects are unfortunate; however, the results from the recall analyses were not affected by the ease of the questions and thus provide confirmatory validity to the overall results. relations between comprehension measures, decoding skill, and world knowledge as can be expected, there was a significant, moderate correlation between decoding skill and world knowledge (r = .52, p <.01). among dependent measures, the two recall measures (free and cued) correlated highly (r = .81, p <.01). by contrast, where the tasks differed more, the correlations between performance on the multiple-choice questions and free recall (r = .46, p <.01) and cued recall (r = .50, p<.01) were significant but moderate in magnitude. effects of reader abilities, genre, and cohesion we performed a median split on the standardized individual difference scores resulting in high and low groups for each knowledge and decoding skill measure. the mean scores for the high and low groups on the two individual difference measures are presented in table 2. table 2. mean scores on aptitude measures for children assigned to the low and high groups ability group individual difference test low m (sd) high m (sd) wj iii academic knowledge 92.97 (5.04) 113.06 (6.80) wj iii word attack 97.39 (4.04) 114.17 (7.86) we conducted separate mixed anovas on performance on each of the three dependent measures: multiple-choice questions, free recall questions, and cued recall questions. in these anovas, within-subjects factors were genre (narrative vs. expository) and cohesion (low vs. high cohesion). the between-subjects factor was either knowledge level (high vs. low knowledge) or reading decoding skill (high vs. low decoding skill). the anovas were conducted separately for knowledge and decoding skill because including both factors reduced some cell sizes to unacceptably low numbers. to ensure that the statistically significant effects identified from the anovas were not distorted by entering knowledge or decoding skill as a categorical variable (high or low), we conducted multiple regression analyses in which either knowledge and reading decoding skill was entered as a continuous variable. children’s scores on the academic knowledge test or scores on the word attack test were entered as a separate independent variable, and scores on the comprehension measures were the dependent variables. we also ran multiple regression analysis with the academic knowledge and word attack scores as well as the centered interaction between the academic knowledge and word attack scores as independent variables to assess the robustness of these effects comprehension. significant main and interaction effects generated from the anovas were replicated in the regression comprehension challenges in the fourth grade / mcnamara, ozuru & floyd 241 analyses. we report only the anovas because this method mimics prior research on knowledge and cohesion interactions, and is most easily interpreted. world knowledge analysis we begin by describing the results when knowledge levels are entered as the betweensubjects factor. a 2 x 2 x 2 mixed anova was conducted for each of the three dependent measures. these data are presented in table 3. the first section below describes the effects of genre and knowledge, and the second describes the effects of cohesion. table 3. proportion correct for the three reading comprehension measures as a function of knowledge, genre, and cohesion knowledge level text type reading comprehension measure genre/cohesion multiple-choice m (sd) cued recall m (sd) free recall m (sd) low narrative/low .62 (.16) .33 (.15) .16 (.12) low narrative/high .70 (.12) .36 (.15) .18 (.11) low expository/low .45 (.17) .06 (.05) .06 (.05) low expository/high .42 (.18) .06 (.05) .04 (.03) high narrative/low .77 (.14) .39 (.14) .22 (.11) high narrative/high .81 (.11) .30 (.12) .19 (.09) high expository/low .68 (.17) .12 (.08) .08 (.06) high expository/high .71 (.17) .15 (.09) .09 (.07) table 4. proportion correct for three reading comprehension measures for the narrative and science texts narrative text science text anova m (sd) m (sd) df1, df2 f fourth-grade children multiple-choice .73 (.12) .57 (20) 1,63 68.20*** cued recall .34 (.12) .10 (.07) 1,50 226.72*** free recall .19 (.09) .07 (.05) 1,52 113.92*** note. *** = p < .001 table 5. proportion correct for three reading comprehension measures for low and highknowledge readers low knowledge high knowledge anova m (sd) m (sd) df1, df2 f multiple-choice .34 (.10) .74 (.10) 1,63 59.27*** cued recall .20 (.07) .24 (.07) 1,50 2.88 free recall .11 (.06) .15 (.07) 1,52 3.68 note. * = p < .05, *** = p < .001. international electronic journal of elementary education 242 table 6. interaction between genre and knowledge narrative expository low k m (sd) high k m (sd) t low k m (sd) high k m (sd) t interacti on f multiple-choice .66 (.12) .79 (.87) -5.20*** .43 (.14) .70 (.15) -7.36*** 11.06** cued recall .34 (.13) .34 (.11) -0.50 .04 (.03) .09 (.05) -5.21*** 2.99 free recall .17 (.10) .20 (.08) -1.22 .04 (.04) .08 (.05) -2.85** <1 note. k= knowledge. ** = p < .01, *** = p < .001. genre and knowledge. as shown at the top of table 4, there was a main effect of genre for all three dependent measures, indicating that children’s comprehension was better for the narrative texts than for the expository texts. as shown at the top of table 5, the effect of knowledge was significant for the multiple-choice questions but marginal for the cued recall and recall measures. thus, high-knowledge readers showed better comprehension than did low-knowledge readers, and this finding was most apparent on the multiple-choice questions. as shown in table 6, there was a significant interaction between genre and knowledge using scores from the multiple-choice questions, and there was a marginal interaction using the cued recall measure. follow-up tests confirmed that the effect of knowledge was greater for the comprehension of expository texts than for the narrative texts. text cohesion. there was a main effect of cohesion on the multiple-choice question performance. as shown in table 7, children comprehended the high-cohesion texts better than the low-cohesion texts. however, the effects of cohesion were not reliable for the two recall measures. table 7. proportion correct for low and high cohesion texts on the three dependent measures low cohesion m (sd) high cohesion m (sd) f multiple-choice .63 (.15) .67 (.15) 4.35* cued recall .22 (.09) .22 (.08) < 1 free recall .13 (.7) .12 (.7) < 1 note. * = p < .05. table 8. interaction between cohesion and genre narrative expository low coh m (sd) high coh m (sd) t low coh m (sd) high coh m (sd) t inter action f multiplechoice 0.69 (0.16) 0.76 (0.13) 3.09** 0.57 (0.21) 0.57 (0.23) 0.12 5.07* cued recall 0.35 (0.15) 0.33 (0.14) 1.36 0.09 (0.06) 0.10 (0.09) -1.26 5.66* note. coh = cohesion. ** = p < .01, *** = p < .001. comprehension challenges in the fourth grade / mcnamara, ozuru & floyd 243 as shown in table 8, there was a significant interaction between cohesion and genre on the multiple-choice question and cued recall question performance. follow-up analyses on the multiple-choice data indicated that for the narrative texts, children comprehended the highcohesion texts better than the low-cohesion texts, whereas there was no effect of cohesion for the expository texts. the interaction according to the cued recall measure is more difficult to interpret because none of the separate effects were significant. nonetheless, the trends indicated that there was a slight advantage for low cohesion narrative texts, compared to a slight advantage for high cohesion science texts. there was also a two-way interaction between cohesion and world knowledge on the cued recall task, f(1,50) = 5.67, p < .05, but not on the multiple-choice or free recall task (both f<1). the interaction indicated that high-knowledge readers recalled more information after reading the low-cohesion texts, m = 0.26, sd = 0.09, than the high-cohesion texts, m = 0.22, sd = 0.08; t(25) = -2.23, p < .05). however, there was no effect of cohesion for low-knowledge readers, mlow cohesion = 0.20, sd = 0.08; mhigh cohesion=0.21, sd = 0.08; t(25) = 1.10, p = .28. most importantly, there was a three-way interaction between genre, cohesion, and knowledge on the cued and free recall measures (cued recall: f(1,50) = 11.86, p < .01; free recall; f(1,52) = 6.52, p <.05). this interaction was marginal for the multiple-choice measure, f(1,63) = 2.61, p = .10. this interaction is displayed for the recall measures in figure 1 (see also table 3 for means). as already noted, there is a larger effect of knowledge for the science texts than for the narrative texts. moreover, we see in figure 1 that the effect of knowledge depends on cohesion, particularly for the narrative texts, such that there was a reversed cohesion effect for high-knowledge readers and a slight benefit for high cohesion for the low-knowledge readers. high-knowledge students tended to show better comprehension after reading the low-cohesion narrative text than the high cohesion narrative text (multiple choice, t(33) = -1.51, p = .14, cued recall, t(26) = -3.88, p < .05, free recall, t(26) = 1.65, p = .11). in contrast, this pattern did not emerge for the low-knowledge readers. there were trends indicating benefits of cohesion for low-knowledge readers on the narrative text and highknowledge readers on the science texts, but these effects were not reliable. it is notable that the scores for low-knowledge readers on the measures of expository recall were extremely low (see figure 1 and table 3), indicating that low-knowledge readers may not have understood the science text sufficiently in order for us to fully interpret the three-way interaction. 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 low k high k low k high k narrative science c u e d r e c a ll low cohesion high cohesion figure 1. interaction between knowledge, genre, and cohesion on the cued recall task and the free recall task. 0.00 0.05 0.10 0.15 0.20 0.25 low k high k low k high k narrative science r e c a ll low cohesion high cohesion international electronic journal of elementary education 244 decoding skill analysis our second set of analyses examined potential effects of decoding skill for the fourth grade children by conducting the same 2 x 2 x 2 anova as reported above, but this time with decoding skill entered as the dichotomous variable. we only report the effects that are not redundant with the effects reported above, that is, those that involve decoding skills. the mean scores on the comprehension measures are reported in table 9. table 9. proportion correct scores as a function of decoding skills, genre, and cohesion decoding skill genre/cohesion multiple-choice m (sd) cued recall m (sd) free recall m (sd) low narrative low .65 (.17) .32 (.14) .17 (.10) low narrative high .72 (.13) 31 (.12) .18 (.10) low expository low .50 (.19) .08 (.06) .06 (.04) low expository high .51 (.23) .10 (.07) .05 (.03) high narrative low .75 (.15) .41 (.15) .22 (.13) high narrative high .81 (.11) .35 (.16) .21 (.11) high expository low .66 (.20) .11 (.08) .09 (.06) high expository high .65 (.20) .11 (.10) .10 (.08) table 10. proportion correct for low and high decoding children on the three dependent measures low decoding m (sd) high decoding m (sd) f multiple-choice .59 (.14) .72 (.11) 15.54*** cued recall .20 (.07) .25 (.08) 4.28* free recall .11 (.05) .15 (.07) 5.05* note. * = p < .05, *** = p < .001. as shown in table 10, there was a main effect of decoding skill according to all three measures of comprehension indicating that skilled decoders performed better on the comprehension tasks than their less skilled counterparts. however, there were no significant interactions involving decoding skills. thus, decoding skill affected comprehension, but the effects of text genre and text cohesion did not depend on decoding skill. discussion the purpose of this study was to examine young children’s comprehension as a function of the characteristics of the text and the children’s knowledge and reading decoding skill levels. our overarching goal was to more fully understand factors that might contribute to reading problems that seem to emerge in the fourth grade, called the fourth grade slump. the most important prediction made in this study is that at the age when young children are expected to begin learning from text, successful comprehension will largely depend on the reader’s knowledge about the world and about specific domains. we made this prediction because this educational period is when children are exposed to a wider variety of texts, and moreover, successful comprehension of those texts demands the integration of knowledge with the text. the more that knowledge is required from the text, the more we can expect to see influences of the reader’s knowledge level. thus, we predicted that interactions between levels of world knowledge and characteristics of the text would emerge around grade 4. indeed, we observed complex interactions between knowledge, text genre, and text cohesion in this study with fourth grade students. findings confirmed that text comprehension is influenced by decoding skill as well as the availability of knowledge to the reader. better decoders showed better comprehension than comprehension challenges in the fourth grade / mcnamara, ozuru & floyd 245 less skilled decoders and children with more world knowledge performed better on the comprehension measures than did those with less knowledge. also, children showed better comprehension of narrative than science texts. this result is most likely because narrative texts are more familiar in structure and contain more familiar information than do science texts, and thus readers have more knowledge available to facilitate comprehension. we also found that the effect of knowledge was greater for science texts than for narrative texts. thus, knowledge was more important when the text was more knowledge demanding. further, a three-way interaction was expected between knowledge, text genre, and text cohesion. the children exhibited a reversed cohesion effect when the texts were familiar (i.e., narrative texts) and when they had sufficient knowledge to fill in the cohesion gaps in the text (i.e., high knowledge readers). this result replicates findings with adolescent readers’ (mcnamara et al., 1996) and adult readers (mcnamara, 2001; mcnamara & kintsch, 1996). it confirms that text understanding can be improved when the reader is induced to make more inferences and when those inferences are likely to be successful. we further predicted that there would be an advantage for high cohesion text. while there was a main effect of cohesion for multiple-choice question performance, the benefits of cohesion were moderate and inconsistent across measures, and did not occur for science texts. to some extent, this result was predicted in the sense that we predicted an interaction of cohesion, genre, and knowledge levels. nonetheless, the results suggest that our manipulations to the cohesion of these texts may not have been sufficient to fully support the children’s understanding of the text. prior research has clearly established a benefit of cohesion manipulations to texts for children of this age (beck, mckeown, omason, & pople, 1984; beck, mckeown, sinatra, & loxterman, 1991; loxterman, beck, & mckeown, 1994). however, the studies that have shown benefits of cohesion included manipulations to the explanatory content in the text. in contrast, in the current study, we were conservative in our approach to avoid adding large amounts of explanatory information to the high cohesion text that could not be inferred in the low cohesion text. this approach is motivated by the goal of not confounding cohesion manipulations with the addition of extra information. however, it seems that the young readers in this study needed more background information in the texts to support their understanding. simply adding cohesion cues, and not explanatory information, is not likely to be sufficient for young readers as an approach to improving comprehension of challenging texts. limitations and future directions the interpretation of these findings should be tempered by at least two limitations. first, although we believe that we selected the two most consistent, powerful, and theoretically valid aptitudes for reading comprehension, other reading aptitudes deemed important in previous research, such as oral language-based abilities (carver & david, 2001; storch & whitehurst, 2002), metacognitive strategy knowledge (baker, 1985; wong, 1985), and knowledge of text structure (williams et al., 2005), were omitted from this study. second, although our sample was quite diverse in terms of gender, race, and socio-economic status and although they were recruited from four public schools in a large metropolitan school district, these samples do not well demonstrate representativeness of all readers in fourth grades. in fact, the children demonstrated somewhat above average performance and somewhat restricted range on the standardized, norm-based screening measures and on the standardized, norm-based reading decoding and world knowledge measures. subsequent studies should ensure that there are sufficient readers with low normative levels on such measures. international electronic journal of elementary education 246 conclusions the results of this study further our understanding of factors that may lead to comprehension problems for children in the fourth grade. essentially, at this age, children are expected to be and generally are developmentally ready to make inferences while reading text and to learn from text. it is at this age that decoding skills are expected to be largely in place and it is at this age when these young children are increasingly expected to read and learn from expository texts. however, generating inferences depends on prior knowledge. if the reader does not possess sufficient knowledge, inferences are likely to be unsuccessful. thus, comprehension problems will become more evident (in comparison to their peers), for children in the fourth grade who have not gained sufficient knowledge about the world. decoding skills are clearly important, but deficits in decoding skills are likely to be evident on all types of texts. thus, decoding skill problems are likely to be detected far before the fourth grade. in contrast, different problems will occur for readers who are expected to learn from text. when readers are able to make inferences, and the text calls for those inferences, differences in levels of knowledge will be a principle factor contributing to comprehension problems. • • • acknowledgment the research was supported in part by the institute for education sciences (ies r305g020018-02). any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the ies. this project was a collaborative endeavor; hence we are also grateful to all of the members of the coh-metrix project who contributed to this study in various ways. we are particularly grateful to rachel best who helped to conduct and analyze this study. we are also grateful to max louwerse and art graesser who were co-investigators on this project. danielle s. mcnamara is a professor in the psychology department and senior scientist in the learning sciences institute at arizona state university. her academic background includes a linguistics b.a. (1982), a clinical psychology m.s. (1989), and a cognitive psychology ph.d. (1992). her research involves the study of learning and cognitive sciences and the development of educational technologies (coh-metrix, istart, writing pal; see soletlab.com). one focus of her work is on developing methods to improve success for struggling students. she has served on numerous funding panels, governing boards, editorial boards, and as associate editor for three journals. dr. yasuhiro ozuru is an assistant professor of psychology at the university of alaska anchorage. he received a ph.d. in experimental psychology with focus on cognitive psychology from new school university in 2003. he has worked on various projects related to computer-based tutoring for reading comprehension strategies and projects to develop assessment tools for elementary school students’ digital literacy. his current research interests are in the areas of discourse processing and argumentation. randy g. floyd is an associate professor of psychology at the university of memphis. he is the editor of the journal of school psychology, and he has served on the editorial boards of journal of psychoeducational assessment, journal of school psychology, school psychology review, and psychology in the schools. his research interests include the structure, measurement, and correlates of cognitive abilities; aptitudes predicting reading comprehension; the technical properties of early numeracy measures; and the process of professional publication. comprehension challenges in the fourth grade / mcnamara, ozuru & floyd 247 references afflerbach, p. (1986). the influence of prior knowledge on expert readers' importance assignment processes. national reading conference yearbook, 35, 30-40. baker, l. (1985). how do we know when we don't understand? standards for evaluating text comprehension. in d. l. forrest-pressley, g. e. mackinnon, & t. g. waller (eds.), metacognition, cognition, and human performance (vol. 1, pp. 155-205). orlando, fl: academic. beck, i., mckeown, m., & gromoll, e. (1989). learning from social studies texts. cognition and instruction, 6, 99-158. beck, i. l., mckeown, m. g., omason, r. c., & pople, m. t. (1984). improving the comprehensibility of stories: the effects of revisions that improve coherence. reading research quarterly, 19, 263277. beck, i. l., mckeown, m. g., sinatra, g. m., & loxterman, j. a. (1991). revising social studies text from a text processing perspective: evidence of improved comprehensibility. reading research quarterly, 26, 251-276. best, r. m., floyd, r. g., & mcnamara, d. s. (2008). differential competencies contributing to children’s comprehension of narrative and expository texts. reading psychology, 29, 137-164. britton, b. k., & gulgoz, s. (1991). using kintsch's computational model to improve instructional text: effects of repairing inference calls on recall and cognitive structures. journal of educational psychology, 83, 329-345. cain, k., oakhill, j. v. & bryant, p. e. (2000). phonological skills and comprehension failure: a test of the phonological processing deficit hypothesis. reading and writing, 13, 31-56. cain, k., oakhill, j., & bryant, p. e. (2004). children's reading comprehension ability: concurrent prediction by working memory, verbal ability, and component skills. journal of educational psychology, 96, 31-42. carver , r. p., & david, a. h. (2001). investigating reading achievement using a causal model. scientific studies of reading, 5, 107–140. chi, m. t. h., feltovich, p. j., & glaser, r. (1981). categorization and representation of physics problems by experts and novices. cognitive science, 5, 121-152. coté, n., & goldman, s. r. (1999). building representations of informational text: evidence from children’s think-aloud protocols. in h. van oostendorp & s. r. goldman (eds.), the construction of mental representations during reading (pp. 169-193). mahwah, nj: erlbaum. ehri, l. c. (1991). development of the ability to read words. in r. barr, m. l. kamil, p. b. mosenthal, & p. d. pearson (eds.), handbook of reading research (vol. ii, pp. 384-417). new york: longman. graesser, a.c. & mcnamara, d.s. (2010). computational analyses of multilevel discourse comprehension. topics in cognitive science. graesser, a. c., mcnamara, d. s., & louwerse, m. m. (2003). what do readers need to learn in order to process coherence relations in narrative and expository text. in a.p. sweet and c.e. snow (eds.), rethinking reading comprehension (pp. 82-98). new york: guilford publications graesser, a. c., mcnamara, d. s., louwerse, m. m., & cai, z. (2004). coh-metrix: analysis of text on cohesion and language. behavioral research methods, instruments and computers, 36, 193-202. graesser, a.c., singer, m., & trabasso, t. (1994). constructing inferences during narrative text comprehension. psychological review, 101, 371-95. hannon, b. & daneman, m. (2001). a new tool for measuring and understanding individual differences in the component process of reading comprehension. journal of educational psychology, 93,103-28. kamberelis, g., & bovino, t.d. (1999). cultural artifacts as scaffolds for genre development. reading research quarterly, 34, 138–170. kendeou, p., van den broek, p., white, m.j., & lynch, j.s. (2009). predicting reading comprehension in early elementary school: the independent contributions of oral language and decoding skills. journal of educational psychology, 101, 765-778. kintsch, w. (1988). the use of knowledge in discourse processing: a construction integration model. psychological review, 95, 163-182. international electronic journal of elementary education 248 kintsch, w. (1993). information accretion and reduction in text processing: inferences. discourse processes, 16, 193–202 kintsch, w. (1998). comprehension: a paradigm for cognition. cambridge, ma: cambridge university press. langer, j. a. (1986). children reading and writing: structures and strategies. westport, ct: ablex publishing. loxterman, j. a., beck, i. l., & mckeown (1994). the effects of thinking aloud during reading on students’ comprehension of more or less coherent text. reading research quarterly, 29, 353367. lyon, g.r. (2002). reading development, reading difficulties, and reading instruction: educational and public health issues. journal of school psychology, 40, 3‑6. mcgrew, k. s., & woodcock, r. w. (2001). technical manual. woodcock-johnson iii. itasca, il: riverside publishing. mcnamara, d. s. (2001). reading both high-coherence and low-coherence texts: effects of text sequence and prior knowledge. canadian journal of experimental psychology, 55, 51-62. mcnamara, d. s., louwerse, m. m., mccarthy, p. m., & graesser, a. c. (2010). coh-metrix: capturing linguistic features of cohesion. discourse processes, 47, 292-330. mcnamara, d. s., & kintsch, w. (1996). learning from texts: effects of prior knowledge and text coherence. discourse processes, 22, 247-288. mcnamara, d. s., kintsch, e., songer, n., & kintsch, w. (1996). are good texts always better? interactions of text coherence, background knowledge, and levels of understanding in learning from text. cognition and instruction, 14, 1-43. meichenbaum, d., & biemiller, a. (1998). nurturing independent learners: helping students take charge of their learning. cambridge, ma: brookline. nelson, k. (1996). language in cognitive development: the emergence of the mediated mind. new york: cambridge university press. oakhill, j. v, cain, k., & bryant, p. e. (2003). dissociation of single-word reading and text comprehension skills. language and cognitive processes, 18, 443-468. oakhill, j. v., cain, k., & yuill, n. m. (1998). individual differences in children's comprehension skill: towards an integrated model. in c. hulme & r. m. joshi (eds.), reading and spelling: development and disorder. mahwah, nj: erlbaum. olson, m. w. (1985). text type and reader ability: the effects on paraphrase and text-based inference questions. journal of reading behavior, 3, 199-214. o'reilly, t., & mcnamara, d. s. (2007). reversing the reverse cohesion effect: good texts can be better for strategic, high-knowledge readers. discourse processes, 43, 121-152. perfetti, c. (1985). reading ability. new york: oxford university press. rupley, h. r., & wilson, v. l. (1996). content, domain, and world knowledge: relationship to comprehension of narrative and expository text. reading and writing, 8, 419-432. storch, s. a., & whitehurst, g. j. (2002). oral language and code-related precursors to reading: evidence from a longitudinal structural model. developmental psychology, 6, 934–947. sweet, a. p., & snow, c. e. (eds.). (2003). rethinking reading comprehension. new york: guilford press. van den broek, p., rapp, d. n., & kendeou, p. (2005). integrating memory-based and constructionist processes in accounts of reading comprehension. discourse processes, 39, 299–316. vellutino, f. r. (2003). individual differences as sources of variability in reading comprehension in elementary school children. in a. p. sweet & c. e. snow (eds.), rethinking reading comprehension (pp. 51-81). new york: guilford press. vellutino, f. r. scanlon, d. m., & spearing, d. (1995). semantic and phonological coding in poor and normal readers. journal of experimental child psychology, 59, 76 -123. vidal-abarca, e., martinez, g., & gilabert, r. (2000). two procedures to improve instructional text: effects on memory and learning. journal of educational psychology, 92, 107–116. williams, j. p., hall, k. m., lauer, k. d., stafford, k. b., de sisto, l. a., & decani, j. s. (2005). expository text comprehension in the primary grade classroom. journal of educational psychology, 97, 538550. wong, b. y. l. (1985). issues in cognitive-behavioral interventions in academic skill areas. journal of abnormal child psychology, 13, 425-442. comprehension challenges in the fourth grade / mcnamara, ozuru & floyd 249 woodcock, r. w., mcgrew, k. s., & mather, n. (2001). woodcock–johnson iii. itasca, il: riverside. wolfe, m. b. w. & goldman, s. (2005). relations between adolescents’ text processing and reasoning. cognition and instruction, 23, 467-502. zinar, s. (1990). fifth-graders' recall of propositional content and causal relationships from expository prose. journal of reading behavior, 22, 181-199. international electronic journal of elementary education 250 appendix a eight texts used in study effects of heat: low cohesion moving heat heat can move from one object or place to another. heat moves from warm objects to cooler ones. you can warm your hands by holding a cup of warm soup. heat moves from the soup through the cup to your hands. you can feel warm air rising above the cup. heat moves through some materials more easily than others. heat moves easily through conductors. most metals are good conductors. metal pots are used for cooking. heat from the stove quickly moves through the metal. the heat warms the food. other materials are not good conductors. but they may be good insulators. insulators help keep heat from passing through. most plastics are good insulators. so are clothes you wear, like sweaters and coats. you wear these clothes to keep warm when it is cold outside. changing matter adding or taking away heat can change matter. matter is something that takes up space. matter can change from one state, or form, to another. an ice cube is solid water. solid is one state of matter. heat can melt an ice cube. the ice cube changes into liquid water. liquid is another state of matter. when heat is taken away, the water can change back. liquid water turns into solid water. heat can make liquids boil. water boils when it is heated. when the water boils, it turns into a gas. this gas is called water vapor. solid, liquid and gas are three states of matter. heat from the sun causes liquid water to turn into water vapor. water vapor mixes with the air. this is called evaporation. sometimes heat causes changes that cannot be changed back. bread can change into toast when you heat it. eggs change when you cook them in a pan. you cannot untoast a piece of toast. you cannot uncook an egg. changing air heat can warm air, too. a balloon is filled with air. when heat warms the air in the balloon, the air changes. the air takes up more space. heat from the sun warms objects all around you, like rocks, streets, and buildings. these objects then warm the air. warm air is lighter than cold air. warm air goes up. cold air takes its place. you can tell how hot or cold the air is. temperature is a measure of how hot something is. people use thermometers to measure the temperature. comprehension challenges in the fourth grade / mcnamara, ozuru & floyd 251 effects of heat on objects, matter, and air: high cohesion heat moves heat can move from one object to another object, or it can move from one place to another place. heat moves from warm objects to cooler ones. for example, you can warm your cold hands by holding a cup of warm soup. your hands become warmer because heat moves from the soup, through the cup, to your hands. the heat from the soup also moves above the cup, so you can feel warm air rising above the cup. heat moves through some materials more easily than other materials. conductors are materials through which heat moves easily. most metals are good conductors. for example, metal pots are used for cooking because heat from the stove quickly moves through the metal pots and the heat in the pot warms the food. other materials are not good conductors, but instead are good insulators. insulators are materials that help keep heat from passing through. for example, most plastics are good insulators. other good insulators are the clothes you wear, especially sweaters and coats. you wear these insulating clothes in order to keep warm when it is cold outside. heat changes matter adding heat or taking away heat can change matter. matter is something that takes up space. matter can change from one state to another state, or from one form to another form. three states of matter are solid, liquid and gas. for example, an ice cube is solid water. heat can melt an ice cube, causing the ice cube to change into liquid water. when heat is taken away, the liquid water can change back into solid water (ice). heat can make liquids boil and change into a gas state. for example, water boils when it is heated. as the water boils, it turns into a gas state that is called water vapor. heat from the sun causes liquid water to turn into water vapor. water vapor then mixes with the air in a process called evaporation. however, sometimes heat causes changes that cannot be changed back. as one example, bread can change into toast when you heat the bread. however, you cannot untoast a piece of toast by taking away heat. as another example, eggs change when you cook them in a pan, but of course you cannot uncook an egg by taking away the heat. heat changes air just as heat can warm liquids, it can also warm the air. air is changed when it is heated. for example, if heat warms air in a balloon, the air changes by taking up more space. heat from the sun warms objects all around you, including rocks, streets, and buildings. these objects then warm the air. warm air is lighter than cold air. therefore, warm air moves upward. when the lighter, warm air goes up, the heavier, cold air moves downward. this cold air takes the place of the air that was warmed. you can measure how hot or cold the air is by using temperature. temperature is a measure of how hot something is. people use thermometers to measure temperature. international electronic journal of elementary education 252 the needs of plants: low cohesion what are the needs of plants? like all living things, plants have certain needs. plants need sunlight, water, and air to live. plants also need minerals (min·uhr·uhlz). a mineral is a naturally occurring substance that is neither plant nor animal. the parts of plants help them to get or make what they need. all plants get water and minerals from the soil. the root is the part of the plant that grows underground. roots help hold the plant in the ground. roots also help take in water and minerals that the plant needs. the stem is the part that supports the plant. it helps the plant stand upright. it carries minerals and water from the roots. it also carries food from the leaves to other parts of the plant. some plants, such as mosses, are simple plants. they don’t have real roots or stems. these plants do not grow tall. instead, they form low-growing mats in damp places to get water directly from the soil. other plants, such as the redwood tree, have many roots and a large stem. they can grow very tall. why does a plant need leaves? the leaves (singular, leaf) are the main food-making part of the plant. many leaves have broad, flat surfaces that help them take in sunlight. leaves are green because of chlorophyll. chlorophyll traps the energy (en·uhr·jee) in sunlight for the plant. the leaf also helps the plant get the air it needs. each leaf has tiny holes that take in air for the plant. the leaf uses a gas in the air called carbon dioxide. the plant uses the sun’s energy to combine carbon dioxide and water to make food. the stem then carries the food to the other parts of the plant. plants use the food they make to stay alive. when we eat plants or other animals that eat plants, we use this food, too. when leaves make food for a plant, they give off oxygen (ahk·suh·juhn). oxygen is a gas that is in air and water. people and animals need oxygen to live. you inhale the oxygen made by a plant with each breath you take. how do plants respond to their environment? have you ever seen a plant leaning toward a sunlit window? this is one way that plants respond to their environment. a plant responds to light by growing toward it. some trees and shrubs drop their leaves as the days grow shorter and colder. these plants respond to the change in seasons from summer to winter. plants also respond to other things in the environment. for example, roots may grow toward water. the ability to respond to the environment helps a plant to live, grow, and meet its needs. comprehension challenges in the fourth grade / mcnamara, ozuru & floyd 253 the needs of plants: high cohesion what plants need plants have certain needs, just like all living things have needs. for example, plants need sunlight, water, and air to live. plants also need minerals (pronounced as min·uhr·uhlz). a mineral is not a plant or an animal. instead, a mineral is a substance in the ground that occurs naturally. there are three parts of plants that help plants get what they need or help plants make what they need. the three parts of a plant the three parts of the plant are the roots, stems, and leaves. 1. the root the root is the part of the plant that grows underground. all plants get water and minerals from the ground, which is sometimes called soil. roots help the plant take in water and minerals that the plant needs from the soil. roots also help hold the plant in the ground. 2. the stem the stem is the part that supports the plant. the stem helps the plant stand upright. it carries minerals and water from the roots of the plant to other parts of the plant. the stem also carries food from the leaves to other parts of the plant. some plants, such as the redwood tree, can grow very tall because they have many roots and a large stem. other plants don’t have real roots or stems. these plants are simple plants. an example of these simple plants is mosses. these simple plants do not grow tall. instead of having roots and stems, they form low-growing mats in damp places. simple plants get water directly from the soil through these mats. 3. the leaves the leaves help the plant make its food. the leaves need sunlight, air, and water to make food. many leaves have broad, flat surfaces. these surfaces are broad and flat in order to help the leaves take in lots of sunlight. the energy in sunlight is trapped by the leaf by a substance called chlorophyll (pronounced klo ro fill). leaves are green because of chlorophyll. the leaf also helps the plant get the air it needs to make food. this process is helped by tiny holes in each leaf. these holes take in air for the plant. the leaf only uses a gas in the air called carbon dioxide (car bun di ok side). however, the plant needs both carbon dioxide and water to make food. the plant uses the sun’s energy to combine the carbon dioxide and water to make food. the stem then carries this food to the other parts of the plant. plants use the food they make to stay alive. when we eat plants or when we eat animals that have eaten plants, we also use this food. that means that the food that keeps the plant alive, keeps us alive too. we also need oxygen (ahk·suh·juhn) to stay alive. the leaves give off oxygen when they make food for the plant. oxygen is a gas that is in air and in water. all people and animals need oxygen to live. therefore, we need plants because we inhale oxygen made by plants with each breath that we take. how plants respond to their environment consider a plant leaning toward a window on which the sun shines. a plant responds to light by growing toward it. this is one way that plants respond to their environment. but plants also respond to other things in the environment. for example, roots may grow toward water. and, some trees and shrubs drop their leaves as the days grow shorter and colder in the fall. these plants are responding to the change in seasons from summer to winter. the ability of a plant to respond to the environment helps a plant to live, to grow, and to meet its needs. international electronic journal of elementary education 254 moving: low cohesion john was very upset when his family moved to another state last month. he knew he would miss all his friends from the neighborhood – the twins, charlie and bob; lisa, debbie, mike, and jimmy. he would miss the playground two blocks from his home where they played baseball. he knew he would also miss the lake where he and his father went fishing, his friends and teachers at school, his aunt lucy who lived three blocks away, and the stores where he went shopping with this family. and he knew he would miss his home: the nights in the den watching sports, the barbecue parties in the backyard, his hideout in the attic and, of course, his room. on moving day, john sat on the steps of his front porch and barely moved. “come on,” his mother said. “help us bring some things into the car.” john could not even hear her. he was thinking of all the good times he had in the neighborhood. when some of his friends came by to say good-bye, tears flowed down his face, and he could barely look at them. “are you going to write to me?” he asked. “of course,” they said. they each made sure they had the other’s address and telephone number. john and his dog, ralph, were the last to get into the car. as his father drove away, john looked back at his house as it got smaller and smaller. when they made a left turn onto another street, he stared out the window at the familiar houses, the same ones he saw when he rode his bicycle through the neighborhood. “don’t be upset,” his mother said. “you’ll meet new friends where we are going.” “just think of it as an opportunity to meet new people,” his father said. “life is full of changes.” “and you can still keep in touch with your old friends,” his mother said. “i do not want any new friends!” john said. john was upset for almost a week after he moved. but then he started meeting other children in his neighborhood. his neighbors had two children, samantha and tom, that were john’s age. he met them when he played outside with ralph. they liked a lot of the same things john liked: baseball, bicycle riding, playing tag. they introduced him to other children. in the fall, john met even more children at school. he liked his new teachers. but he still could not wait for thanksgiving to visit his aunt and uncle in his old neighborhood, where he would see his old friends again. comprehension challenges in the fourth grade / mcnamara, ozuru & floyd 255 moving: high cohesion last month john and his family moved to another state. john was very upset because he knew he would miss all his friends from the neighborhood. he had many friends: the twins, charlie and bob; lisa, debbie, mike, and jimmy. he would miss many other people too: his friends and teachers at school and his aunt lucy who lived three blocks away. he would miss places too. he would miss the playground two blocks from his home where he and his friends played baseball. he knew he would also miss the lake where he and his father went fishing and the stores where he went shopping with this family. and he knew he would miss his home: the nights in the den watching sports, the barbecue parties in the backyard, his hideout in the attic and, of course, his own room. on moving day, john sat on the steps of his front porch and he was so upset that he barely moved. his mother said, “come on, help us bring some things into the car.” but, john could not even hear his mother because he was thinking of all the good times he had in the neighborhood. later, when some of his friends came by to say good-bye, tears flowed down john’s face. because he was so sad, he could barely look at them. john wanted to keep in touch with his friends, so he asked, “are you going to write to me?” his friends said: “of course, we will write to you.” he and his friends each made sure they had each other’s address and telephone number, so they could write and call. then it was time to leave. john and his dog, ralph, were the last to get into the car. as his father drove away, john looked back at his house. while they were driving away it seemed that the house got smaller and smaller as it got further away. when they made a left turn onto another street, john stared out the window at the familiar houses. they were so familiar because these houses were the same ones he saw when he rode his bicycle through the neighborhood. his parents knew he was upset and tried to comfort him. his mother said: “don’t be upset, you’ll meet new friends where we are going.” his father said: “just think of this move as an opportunity to meet new people. we know that this move is a change for you. but changes can be good. life is full of changes.” his mother said: “you can make new friends, and you can still keep in touch with your old friends.” john disagreed, “i do not want any new friends!” john was upset for almost a week after he moved to the other state. but then he started meeting other children in his neighborhood. his neighbors had two children, samantha and tom, who were john’s age. he met samantha and tom when he played outside with his dog, ralph. samantha and tom liked to do a lot of the same things john liked to do: baseball, bicycle riding, playing tag. john made more friends when samantha and tom introduced him to other children. in the fall, when school began, john met even more children at school. he also liked his new teachers. but he still could not wait for thanksgiving to visit his aunt and uncle in his old neighborhood, where he would see his old friends again. international electronic journal of elementary education 256 orlando: low cohesion salvador was upset. he told mama he was going out. he didn't want her to be worried or sad. he just needed to be alone. he hurried across the cornfield. he turned the corner of the fort and ran through the door. no one would see or hear him cry. the floor was just dirt, but salvador liked his fort. he had made it himself the year before. he had brought wood scraps from the old barn behind his house. salvador lay down on the dirt floor. he closed his eyes. he had a picture in his mind of his mama. he thought about what happened at home before he left. he cried some more. “we have to sell orlando,” mama had said. salvador had put down his fork. “what did you say, mama?” mama tried to explain. but she could not bring herself to say more. she just looked away. “why would we sell orlando?” salvador had asked. “you know we are moving, salvador. and we will not have room for him in our new place. i know he is a fine pig,” mama had said. “orlando is more than a pig!” salvador had shouted. “he is family.” “i know, son,” mama had said. “if anyone tries to take poor orlando, i won't let them!” salvador had shouted. then he had hurried from the house to his fort. salvador adored orlando. orlando was four years old. salvador had named him after a place in florida he had read about. there could not be a more perfect pet for salvador. as salvador lay in the fort crying, it began to rain. at first it was only a light rain. then it began to pour. next there was lightning and thunder. it was a very bad storm! next salvador heard a loud crack! lightning had struck a tree in the forest. the tree was on fire! salvador got up and ran out of the fort. it was dark! he couldn't see. he did not know which way was home. before long salvador heard a snorting sound. it was orlando! orlando had come to find him! salvador reached out for orlando. he felt the pig's ear. he held on. orlando led salvador home. they ran through the cornfield as fast as they could go. mama stood by the door on the porch. “oh, salvador. i was so worried!” she cried. “orlando saved me,” salvador said. “yes, i know,” said mama. “he is family! we will just have to find room for him at our next place.” this brought a smile to salvador's face. orlando was saved! “now he really is a member of our family!” said salvador. comprehension challenges in the fourth grade / mcnamara, ozuru & floyd 257 orlando: high cohesion once upon a time, there was a boy. his name was salvador. salvador adored his pet pig named orlando. orlando was four years old. salvador had named orlando after a place in florida that he had read about. there could not be a more perfect pet for salvador than his pet pig. one day, orlando was eating dinner at home with his mama. mama said, “we have to sell orlando.” salvador put down his fork. “what did you say, mama?” mama tried to explain, but she could not bring herself to say more. mama just looked away. salvador asked, “why would we sell orlando?” mama answered, “you know we are moving, salvador. but we will not have enough room for orlando in our new place. we cannot take him, even though i know he is a fine pig.” salvador shouted, “orlando is more than a pig! he is family.” mama said, “i know, son.” salvador shouted, “if anyone tries to take poor orlando, i won't let them!” salvador was upset, so he needed to be alone. salvador told mama he was going out because he didn't want her to be worried or sad. salvador left the house and hurried across the cornfield. he turned the corner to a fort that he had made and ran through the door. he went to the fort so no one would see him cry or hear him cry. the floor of the fort was just dirt, but salvador liked his fort. he had made it by himself one year ago. he had brought wood scraps from an old barn behind his house to build the fort. after salvador ran into his fort, he lay down on the dirt floor and closed his eyes. he had a picture in his mind of his mama. he thought about his mama telling him that they had to sell orlando, and that made him cry some more. as salvador lay in the fort crying, it began to rain. at first it was only a light rain. then it began to pour. next there was lightning and thunder because it was a very bad storm! then lightning struck a tree in the forest, so salvador heard a loud crack! the lightning had caught the tree on fire! because of the noise and fire, salvador got up and ran out of the fort. it was dark, so he couldn't see and did not know which way was home. salvador became lost in the dark. before long salvador heard a snorting sound. the snorting was from orlando! orlando had come to find him! when salvador reached out for orlando, he felt the pig's ear. salvador held on to the ear as orlando led salvador home. they ran through the cornfield as fast as they could go. when salvador and orlando got home, mama stood by the door on the porch. mama cried, “oh, salvador. i was so worried!” salvador explained, “orlando saved me.” mama said, “yes, i know. orlando must be family because he saved you! we will just have to find room for him at our next place.” so orlando was saved, which brought a smile to salvador's face. salvador said, “now orlando really is a member of our family!” international electronic journal of elementary education, 2013, 5(2), 125-142. issn:1307-9298 copyright © iejee www.iejee.com using first-grade teacher ratings to predict third-grade english language arts and mathematics achievement on a high-stakes statewide assessment dorinda j. gallant  the ohio state university, columbus, oh, usa received: 2 august 2012 / revised: 25 december 2012 / accepted: 10 january 2013 abstract early childhood professional organizations support teachers as the best assessors of students’ academic, social, emotional, and physical development. this study investigates the predictive nature of teacher ratings of first-grade students’ performance on a standards-based curriculum-embedded performance assessment within the context of a state accountability system. the sample includes 4292 elementary school students cross-classified by 131 first-grade and 137 third-grade schools attended. this study uses extant statewide assessment data for students located in a state in the southeastern part of the united states. controlling for student and school demographic variables in cross-classified random effects multilevel models, first-grade teacher ratings—as reflected by domain scores on a performance assessment—are found to positively and significantly correlate with students’ third-grade academic achievement. keywords: teacher ratings, predictive validity, curriculum-embedded performance assessment introduction research on the use of teacher-based judgment measures (e.g., measures that use teacher ratings or rankings to assess students’ knowledge or skills in specific academic content areas) to assess students’ academic achievement in core academic areas (i.e., math, reading, science, or social studies) span over four decades. one of the most comprehensive scholarly reviews of the use of teacher-based judgment measures to assess student achievement was completed by hoge and coladarci (1989). the authors presented a thorough review of 16 empirical studies from 1962-1988. with these studies, they examined the association between concurrently administered direct and indirect teacher-based judgment measures in which  dorinda gallant, the ohio state university, department of educational studies, college of education and human ecology, 211b ramseyer hall, 29 west woodruff avenue, columbus, oh 43210, united states, telephone: 614-247-8860. e-mail: gallant.32@osu.edu http://www.iejee.com/ using first-grade teacher ratings to predict third-grade english language arts and mathematics achievement on a high-stakes statewide assessment / gallant 126 teachers used ratings, rankings, grade equivalence, number correct, and item responses in reading, mathematics, social studies, and science, and norm-referenced measures of academic achievement. overall, hoge and coladarci found a moderate to strong association (mdn r = .66) between teachers’ judgments of students’ academic performance and their actual performance on standardized norm-referenced achievement tests. specific to the predictive value of teachers’ ratings, several studies have investigated the longitudinal nature of teacher ratings in relation to students’ performance on normreferenced tests (e.g., hecht & greenfield, 2001; meisels, liaw, dorfman, & nelson, 1995; quay & steele, 1998; stevenson, parker, wilkinson, hegion, & fish, 1976). stevenson et al. (1976), followed a cohort of students from kindergarten through third grade and found moderate to strong correlations (rs ranged from .41 to .71) between teacher ratings (i.e., instructions, vocabulary, reflective, retention, learning, independence, or attention) and students’ reading achievement, as measured by the wide range achievement test (jastak, bijou, & jastak, 1965), across four time periods (i.e., prior to kindergarten and in the spring of grades 1, 2, and 3), and moderate to strong correlations (rs ranged from .37 to .65) between teacher ratings (i.e., instructions, learning, vocabulary, retention, hardworking, independence, or reflective) and students’ arithmetic achievement across the same four time periods. using a sample of kindergartners in three michigan school districts to present predictive validity evidence for a performance assessment for young children, meisels, liaw, dorfman, and nelson (1995) reported high correlations between teacher ratings and students’ performance on a norm-referenced measure within a one-year period. specifically, the correlations between kindergarteners’ total scores on the work sampling system (wss; meisels, jablon, marsden, dichtelmiller, & dorfman, 1994) checklists—an authentic assessment measure in which students are observed in their natural settings to determine the extent to which they can demonstrate proficiency on defined local, state, or national curriculum standards—and a total score on the kindergarten achievement battery of the woodcock-johnson psychoeducational battery–revised (wj-r; woodcock & johnson, 1989), were .76. the fall/spring and winter/spring correlations between individual wss checklists and wj-r total scores were similar (ranging from .38 to .78 for fall/spring and ranging from .39 to .78 for winter/spring). the highest correlations were found between the concept and number and the language and literacy subscales of the wss and the wj-r (rs = .75 and .78, respectively, for fall/spring and rs = .78 for winter/spring). in 2001, hecht and greenfield found that first-grade teacher ratings, collected using the academic competence subscale of the social skills rating system (gresham & elliott, 1990) in a large urban public school system, explained approximately 50% of the variance in thirdgrade letter-word identification and passage comprehension for a predominantly minority population of children (i.e., 73% african american, 18% hispanic, and 2% asian american) exposed to poverty. the authors also reported that first-grade teacher ratings accurately classified third-grade students into good or impaired reader ability groups with an accuracy rate of at least 73%. hence, the finding provided support for the use of teacher ratings to predict students’ later performance. however, in quay and steele’s 1998 study in which the authors followed a cohort of students from pre-kindergarten to grade 2, the magnitude of the association between prekindergarten teacher ratings and students’ later achievement varied according to when the ratings occurred. the researchers found that pre-kindergarten ratings on the developmental profile ii (dpii; alpern, boll, & shearer, 1986) had only a small significant association with firstgrade teacher reports of reading comprehension (r = .16), but pre-kindergarten ratings on the international electronic journal of elementary education vol.5, issue 2, 125-142, 2013 127 developmental rating scale (drs) had significant moderate associations with first-grade teacher reports of reading comprehension (r = .42) and math achievement (r = .38) and significant small associations with second-grade teacher reports of reading comprehension (r = .24) and math achievement (r = .27). yet, the associations between pre-kindergarten ratings on either the drs or the dpii and students’ second-grade iowa tests of basic skills (itbs) total reading or math were small (rs ranged from -.02 to .12). moreover, the first-grade ratings on the dpii were small significant predicators of first-grade teacher reports of reading comprehension and math achievement and second-grade teacher report of reading comprehension (rs ranged from .18 to .29). conversely, first-grade ratings on the drs were moderate to high predictors of firstand second-grade teacher reports of reading comprehension and math achievement, and second-grade itbs total reading and math (rs ranged from .40 to .78) significance of this study although much research has been conducted on the use of teacher ratings, this study aims to contribute to research on teacher ratings specifically in two areas: (a) examining the predictive nature of teacher ratings within the context of a state’s education accountability system and (b) using multilevel methodology to account for both the nested nature of students within schools and the cross-classification that can occur in longitudinal studies when students do not attend the same schools at both time periods of investigation. there is minimal research literature on the connection between teacher ratings on a continuously administered performance-based assessment and a high-stakes group-administered assessment within the context of an education accountability system. meisels, atkins-burnett, xue, nicholson, bickel, and son (2003) proposed that instructional and high-stakes assessments can potentially be connected to create an education accountability system that relies on both information obtained in the classroom and tests on student achievement. this team of researchers found that in a large urban public school system in which at least 70% of the sample were african american and at least 87% received free or reduced-price lunch, students who had been exposed to the wss (meisels, jablon, marsden, dichtelmiller, & dorfman, 1994, 2001) for at least three years, prior to being administered the itbs, showed a greater change in mean grade 3 and grade 4 itbs developmental standard scores for reading and mathematics compared to a demographically matched comparison group and all other students within the school district. these findings suggest that, as teachers obtained continuous information on students’ academic performance, they could potentially use the information to improve students’ learning over time. this is especially true when viewing accountability as a “system” instead of as a stand-alone test. hence, as school districts and state departments of education attempt to meet national accountability mandates for students in grades 3 through 8 that are found in the no child left behind legislation of 2001, it is imperative that researchers investigate the use of teacher ratings in the early grades to predict students’ later performance on high-stakes tests. furthermore, from a methodological perspective, this study explores the predictive nature of teacher ratings from a multilevel modeling perspective, accounting for the variance associated with the cross-classification of students within schools. previous studies have primarily used correlations, linear regression analyses, or correlations and regression analyses (e.g., hecht & greenfield, 2001; meisels et al., 1995; stevenson et al., 1976) to investigate the predictive nature of teacher ratings. however, the nested nature of students within schools or classrooms has either been ignored or limited in teacher-rating studies. gallant (2005, 2009) explored the predictive nature of first-grade teacher ratings on a state’s standards-based performance assessment for children in a large urban public school district using a two-level using first-grade teacher ratings to predict third-grade english language arts and mathematics achievement on a high-stakes statewide assessment / gallant 128 multilevel regression model. the author found a positive significant association between firstgrade teacher ratings and students’ third-grade performance in mathematics and english language arts when partitioning the between-classroom and the within-classroom variances. however, the study did not account for situations in which students may have attended different schools at the two time points (i.e., grades 1 and 3) because of school configurations (e.g., k-2, k-3, k-5) and because the sample was limited to one urban school district within the state. purpose of this study therefore, this study aimed to expand the work of meisels et al. (1995) and gallant (2005, 2009) on the predictive nature of teacher ratings in the early grades to determine students’ later academic performance by including a larger representative sample of students nested within schools. furthermore, this study accounts for the cross-classification of students within schools. hence, the purpose of this study is to use a statewide, standards-based assessment program for students in grades 1 and 3 to investigate the predictive nature of first-grade teacher ratings, taking into consideration the nested nature of the data and the crossclassification of students across schools. specifically, this study investigated the association between first-grade teacher ratings, as reflected by domain scores, on the language and literacy and mathematical thinking domains of a standards-based curriculum-embedded performance assessment based on the developmental guidelines and checklist of work sampling system and third-grade students’ achievement in mathematics and english language arts, as indicated by scale scores, on a high-stakes standards-based criterionreferenced test. the research questions of interest were the following: 1. how well do first-grade teachers’ ratings of students’ language and literacy performance on a standards-based curriculum-embedded performance assessment predict students’ english language arts third-grade performance on a standards-based criterion-referenced test? 2. how well do first-grade teachers’ ratings of students’ mathematical thinking performance on a standards-based curriculum-embedded performance assessment predict students’ third-grade mathematics performance on a standards-based criterion-referenced test? context of this study in a state located in the southeastern part of the united states, an education accountability act was passed in 1998. the accountability act introduced comprehensive accountability measures to schools and school districts and further created an education oversight committee to monitor compliance with the legislation (education accountability act, 1998). among the accountability components included in the section of the legislation on academic standards and assessments, three major components relevant to this study were (a) adoption of grade-level specific educational standards in english language arts, mathematics, science, and social studies for kindergarten through grade 12; (b) development or adoption of a statewide assessment program to measure student performance on state standards in english language arts, mathematics, science, and social studies for grades 3 through 8; and (c) development, adoption, or selection of separate tests linked to adopted firstand secondgrade academic standards. while student performance in grades 3 through 12 were used as an accountability measure at the state level, firstand second-grade readiness tests were not. international electronic journal of elementary education vol.5, issue 2, 125-142, 2013 129 however, the second-grade readiness test was to serve as the baseline for the third-grade assessment. legislative compliance for the first-grade readiness test was met when the state board of education, through the state’s department of education, adopted the personal and social development, language and literacy, and mathematical thinking developmental guidelines and checklists of the work sampling system for students in first grade, with modifications to indicators for alignment with the state’s adopted standards for first grade (huynh, prior, & gallant-taylor, 2002). the work sampling system provided an alignment of curriculum and assessment along with compliance with prior legislation that mandated the development or adoption of developmentally appropriate assessment for children pre-kindergarten through grade 3. the first statewide administration of the first-grade readiness test occurred in spring of 2001. method data source and sampling this study was based on a cluster random sample of 27 school districts with student assessment records consisting of first-grade teacher ratings and third-grade achievement scores. originally, 30 school districts were randomly selected, but three districts did not have the necessary data. two extant data files were merged and provided to the author by the state’s department of education. the merged data file included first-grade teacher ratings on the personal and social development, language and literacy, and mathematical thinking domains of a standards-based, curriculum-embedded performance assessment administered in spring of 2002; third-grade scale scores on the english language arts and mathematics subscales of a standards-based, criterion-referenced test administered in spring of 2004; student demographic variables; and school and district codes. specific to this study, the english language arts and mathematics domains and subscales on the curriculum-embedded performance assessment and the achievement tests, respectively, were selected because of the national focus on reading and math achievement, as reflected in the no child left behind legislation. furthermore, only spring ratings of firstgrade students’ performance on the indicators for language and literacy and mathematical thinking were selected to be consistent with the spring administration of the third-grade achievement tests in english language arts and mathematics. that is, using spring ratings instead of fall or winter ratings on the performance assessment represented one year of academic growth of students in language and literacy and mathematical thinking. a brief description of each assessment measure as it relates to this study follows. first-grade performance assessment. teacher ratings were obtained using a low-stakes, standards-based, curriculum-embedded performance assessment (hereafter referred to as performance assessment) based on the work sampling system. the language and literacy domain consisted of 12 indicators and the mathematical thinking domain consisted of 14 indicators (huynh, prior, & gallant-taylor, 2002). specific examples of each indicator can be found in using work sampling guidelines and checklists: an observational assessment (dichtelmiller, jablon, meisels, marsden, & dorfman, 1998). using the developmental guidelines and checklists for each domain, teachers observed and rated students’ performance on the language and literacy and mathematical thinking indicators as: 1 = not yet, 2 = in process, and 3 = proficient. ratings of not yet indicated that the skill, knowledge, or behavior had not been demonstrated; ratings of in process indicated that the skill, knowledge, or behavior was emergent and was not demonstrated consistently; and using first-grade teacher ratings to predict third-grade english language arts and mathematics achievement on a high-stakes statewide assessment / gallant 130 ratings of proficient indicated that the skill, knowledge, or behavior was firmly within the child’s range of performance (dichtelmiller et al., 1998). training sessions were provided to teachers at the district and the regional levels in 2001, and additional training sessions were conducted by individual school districts in subsequent years as needed (huynh, prior, & gallant-taylor, 2002). the internal consistency—cronbach’s alphas—of .98 were reported for the language and literacy and mathematical thinking domain scores for the spring observation period for all first-grade students in 2001. domain scores were computed by summing teachers’ ratings of each indicator on the language and literacy and mathematicalthinking domains. thus, performance assessment domain scores ranged from 12 to 36 on the language and literacy domain and from 14 to 42 on the mathematical thinking domain. for this study, cronbach’s alpha of teachers’ ratings was .93 for both the language and literacy domain and for the mathematical thinking domain. third-grade achievement tests. the third-grade achievement tests were untimed high-stakes standards-based criterion-referenced assessments administered to students in grade 3 through a statewide assessment program. the tests assessed mathematics and english language arts using multiple-choice and constructed-response items in the spring of an academic year. the english language arts subtest also consists of an extended writing item. test results were reported as total scale scores (i.e., theoretical minimum and maximum scores are computed as grade level times 100, plus or minus 64) and performance levels (i.e., below basic, basic, proficient, and advanced) for each of the academic areas (huynh, meyer, & barton, 2000). reliability indices (i.e., cronbach’s alpha and kr-21) for the 1999 administration of the english language arts and mathematics achievement tests were larger than .85 for all students and across gender and ethnicity (huynh, meyer, & barton, 2000). sample. student records meeting the following criteria were included in the study: (a) records contained both first-grade performance assessment domain scores for 2002 and third-grade achievement scale scores in english language arts and mathematics for 2004, (b) records indicated that students were tested at grade level for the achievement tests in english language arts and mathematics (i.e., students in third grade took the grade 3 version of the achievement test), and (c) performance assessment domain scores ranged from 12 to 36 for language and literacy and from 14 to 42 for mathematical thinking (the range of scores was the natural minimum and maximum for the performance assessment, based on the numerical values assigned for rating categories). hence, the sample of records for this study consisted of 4292 student records representative of 131 first-grade schools and 137 third-grade schools. classroom-level data were not included in the data file. descriptive characteristics of students, including descriptive statistics for performance assessment domain scores and achievement tests scores, are presented in table 1. as reflected in the table, the sample of records represents 50% female, 49% non-white, 58% eligible for subsidized lunch, and 11% on an individualized education plan. achievement tests were administered to third-grade students in spring of 2004. table 1. descriptive characteristics of students (n = 4292) characteristic n % m sd min max gender female 2146 50.0 race non-white 2104 49.0 international electronic journal of elementary education vol.5, issue 2, 125-142, 2013 131 individual education plan 472 11.0 eligible for free or reduced-price lunch 2489 58.0 eligible for free or reduced-price lunch 2489 58.0 using first-grade teacher ratings to predict third-grade english language arts and mathematics achievement on a high-stakes statewide assessment / gallant 132 table 1 (continue). descriptive characteristics of students (n = 4292) characteristic n % m sd min max performance assessment mathematical thinking 4292 37.61 5.28 14.00 42.00 language and literacy 4292 31.93 4.76 12.00 36.00 achievement tests mathematics 4292 308.71 11.94 265.00 344.00 english language arts 4292 311.35 14.26 257.00 352.00 note. performance assessments were administered to first-grade students in spring of 2002. selected descriptive characteristics of firstand third-grade schools students attended, available on the state’s report card web site, are presented in table 2. the author hypothesized that the selected school-level demographic variables influenced student achievement. due to the varied organizational structure of schools in this state (e.g., k-2, k-3, k-5), approximately 23% of students did not attend the same elementary school for grades 1 and 3. as reflected in the table, the mean student-teacher ratio and attendance rates for students and teachers were similar for firstand third-grade schools attended. however, third-grade schools, on average, spent more dollars per student. table 2. descriptive characteristics of the firstand third-grade schools first-grade school (n = 131) third-grade school (n = 137) characteristic m sd min max m sd min max dollars spent per student 5701.47 978.28 3939.0 9461.0 6230.04 1209.29 3601.0 9539.0 studentteacher ratio 17.74 3.25 3.40 25.60 18.65 2.73 9.70 26.90 student attendance rate 96.16 1.66 91.00 100.00 96.65 1.22 94.50 99.90 teacher attendance rate 94.76 1.50 89.20 99.10 94.83 1.41 90.80 99.30 note. descriptive characteristics of schools were obtained from school report cards posted on the state’s web site. the characteristics are not exhaustive but reflect a small sample of characteristics hypothesized by the author to affect student achievement. statistical analyses statistical analyses included computing the partial correlation coefficients for the association between first-grade performance assessment domain scores and third-grade achievement test scores and cross-classified multilevel regression analyses. for all analyses, the statistical significance level was set at  = .05. however, alpha levels of .001 were also reported. an indepth description of each statistical analysis follows. international electronic journal of elementary education vol.5, issue 2, 125-142, 2013 133 correlations. correlations between performance assessment domain scores and achievement tests scale scores were computed to determine the amount of shared variance between firstgrade teachers’ summed ratings on the performance assessments and students’ later performance on the third-grade achievement tests, controlling for student demographic variables (i.e., gender, race, individualized education plan status, and eligibility for subsidized meals). the effect of student demographic variables was partitioned out to mitigate the possible confounding that inclusion of the variables would have created in the association between measures. the magnitude of the associations were described using cohen’s conventions (1992) of small = .10, medium = .30, and large = .50. cross-classified random effects multilevel regression analyses. two-level multilevel regression analyses were used to partition the variance associated with the nested nature of the data (i.e., students nested within schools). specifically, cross-classification random effects models were specified because students were not purely nested within one school over the two-year time period. that is, students did not necessarily attend the same school in first and third grade. unlike traditional two-level multilevel regression models, cross-classification random effects models do not assume that the nesting is purely hierarchical (raudenbush & bryk, 2002; beretvas, 2008). thus, the cross-classification random effects models allow the researcher to account for the cross-classification of students across firstand third-grade schools attended. the cross-classified random effects models used in this study were estimated using maximum likelihood estimation with an independent z-structure and specified to address each research question. dependent variables were third-grade achievement tests scores in english language arts and mathematics. independent variables were student demographic variables and language and literacy and mathematical thinking domain scores, computed as the sum of teacher ratings for each indicator on the domain, on the performance assessment. the demographic variables were coded such that race (race) was coded 0 = white and 1 = non-white, gender (gender) was coded 0 = male and 1 = female, eligibility for subsidized meals (lunch) was coded 0 = ineligible for subsidized meals and 1 = eligible for free or reduced-priced lunch, and individual education plan status (iep) was coded 0 = no iep and 1 = iep. the software hlm 6.08: hierarchical linear and nonlinear modeling (raudenbush, bryk, & congdon, 2004) was used for the multilevel analyses. the unconditional and conditional cross-classification random effects models for this study were specified as follows. the mixed-effects unconditional model for predicting student achievement is: yijk = θ0 + b00j + c00k + eijk, (1) where yijk is the third-grade mathematics or english language arts achievement test scores of student i in grade 1 school j and grade 3 school k; θ0 is the overall grand-mean achievement score of all students; b00j is the random effect for grade 1 school attended [b00j ~ n(0, b00)]; c00k is the random effect for grade 3 school attended [c00k ~ n(0, c00)]; and eijk is the deviation of student ijk’s score from the cell mean [eijk ~ n(0, 2)]. the conditional models were specified the same at level 1 but differently at level 2 for mathematics and english language arts achievement. differences in the models occurred at level 2 because the intercept variability across first-grade school attended in the mathematics model was not statistically significant (p > .05), indicating that the effect be better modeled as fixed rather than random. student demographic variables and two of the four school demographic variables (i.e., dollars spent per student and teacher-student ratio) were included in the model to potentially account for some of the unexplained variances found in using first-grade teacher ratings to predict third-grade english language arts and mathematics achievement on a high-stakes statewide assessment / gallant 134 the unconditional model and to serve as controls in the analyses. only dollars spent per student and teacher-student ratio were included at the school levels because inclusion of student attendance rates and teacher attendance rates created extremely large standard errors, possibly due to multicollinearity. level 1 was as follows: yijk = 0jk + 1jk(x)ijk + 2jk(gender)ijk + 3jk(iep)ijk + 4jk(race)ijk + 5jk(lunch)ijk + eijk, eijk  n(0, 2), (2) where yijk is the third-grade mathematics or english language arts achievement test score of student i in grade 1 school j and grade 3 school k; x is the grand-mean centered mathematical thinking or language and literacy domain score; 0jk is the overall grand-mean math achievement score for students attending the same school both years; 1jk…5jk are the regression coefficients relating the predictors to the mathematics achievement; and eijk is a random effect that represents the deviation of student ijk’s outcome from the predicted outcome based on the predictors in the model. level 2 (for mathematics): 0jk = 0 + c00k + γ01(sch1dllrs) + γ02(sch1ratio) + 03(sch3ratio) + 04(sch3dllrs) (3) 1jk = 1 + b10j 2jk = 2 3jk = 3 4jk = 4 5jk = 5; and level 2 (for english language arts): 0jk = 0 + b00j + c00k + γ01(sch1dllrs) + γ02(sch1ratio) + 01(sch3ratio) + 02(sch3dllrs) (4) 1jk = 1 + b10j 2jk = 2 3jk = 3 4jk = 4 5jk = 5, where sch1dllrs and sch1ratio are the school 1 dollars spent per student and student-teacher ratio, respectively; sch3dllrs and sch3ratio are the school 3 dollars spent per student and student-teacher ratio, respectively; 0 is the grand mean, when all school 3 variables are held constant; 1…5 are the fixed effects of the student-level predictors; γ01 and γ02 are the fixed effects of the school 1 variables; 01and 02 are the fixed effects of the school 3 variables; b00j and c00k are the residual random effect of school 1 and school 3, respectively, on the overall grand-mean achievement score after taking into account the school 1 and school 3 predictors; and b10j is the residual random effect of school 1 on the grand-mean centered performance assessment domain score. international electronic journal of elementary education vol.5, issue 2, 125-142, 2013 135 mathematical thinking and language and literacy domain scores were grand-mean centered (i.e., centered on the school performance assessment domain score means) to create a meaningful reference point for the predictors in the model. that is, by centering on the school performance domain score means, a score of zero can be interpreted as a student receiving the average performance assessment domain score; whereas if the performance assessment domain scores were not centered, a score of zero would be interpreted as a student receiving a performance assessment domain score of zero. given the range of the performance assessment domain scores—12 to 36 for language and literacy and 14 to 42 for mathematical thinking—a student receiving a performance assessment domain score of zero is impossible. in addition, standardized coefficients were computed as  y x s s (snijders & bosker, 1999) for the multilevel results to provide a scale for effects independent of the measurement units. hence, results are reported in terms of standard deviations. results correlations between performance assessment domain scores and achievement test scores as presented in table 3, positive significant associations were found between language and literacy domain scores and english language arts scale scores (r = .37, p < .001) and between mathematical-thinking domain scores and mathematics scale scores (r = .30, p < .001). the magnitude of the associations can be described as medium. based on the partial correlation coefficients, approximately 14% of the variability in language and literacy domain scores overlapped with the variability in english language arts achievement scale scores, whereas approximately 9% of the variability in mathematical thinking domain scores overlapped with the variability in mathematics achievement scale scores. table 3. correlations among first grade performance assessment domain scores and third grade achievement scale scores, controlling for student demographic variables (n = 4287) first-grade performance domains third-grade achievement scales variable language & literacy mathematical thinking english language arts mathematics language & literacy — .75* .37* .29* mathematical thinking — .32* .30* english language arts — .62* mathematics — note. partial correlation coefficients were computed for 4,287 records that included all demographic variables. *p < .001, two-tailed. cross-classified multilevel analyses for mathematics and english language arts achievement mathematics achievement. estimates for the fixedand random-effects for the unconditional and conditional models for predicting student mathematics achievement are reported in table 4. for the unconditional model, the mean mathematics achievement score was 308.26, with a total variance of 145.31. of the total variance, approximately 2% of the variance was between scores of students who attended the same first-grade school but a different thirdgrade school and approximately 12% of the variance was between scores of students who using first-grade teacher ratings to predict third-grade english language arts and mathematics achievement on a high-stakes statewide assessment / gallant 136 attended the same third-grade school but a different first-grade school. thus, differences in mathematics achievement were attributed more to the third-grade school attended than to the first-grade school attended. additionally, approximately 86% of the total variance was within cross-classification of the firstand third-grade school attended (i.e., within students who attended the same firstand third-grade school). in the conditional model, controlling for student demographic variables and school demographic variables, mathematical thinking domain scores were statistically significant predictors of mathematics achievement (1 = .69, p < .001). thus, each additional standard deviation on the mathematical thinking domain score led, on average, to an increase of a .31 standard deviation in third-grade mathematics achievement. furthermore, the inclusion of student and school variables in the model decreased the estimated student-level variance from 124.74 to 93.94 (approximately a 25% reduction) and the estimated third-grade school attended variance from 18.05 to 13.88 (approximately a 23% reduction). table 4. estimation of fixed and random effects for based on cross-classification models for mathematics achievement unconditional model conditional model fixed effects coefficient se t ratio coefficient se t ratio intercept, 0 000 308.26** 0.49 625.95 313.66** 5.00 62.71 school 1 dollars γ01 -0.00 0.00 -0.09 school 1 ratio γ02 0.09 0.10 0.86 school 3 ratio 01 -0.07 0.15 -0.47 school 3 dollars 02 0.00 0.00 0.02 domain score, 1 100 0.69** 0.04 17.17 gender, 2 200 -0.25 0.30 -0.83 iep, 3 300 -5.24** 0.49 -10.71 race, 4 400 -4.32** 0.38 -11.35 lunch, 5 500 -3.63** 0.37 -9.75 random effects estimate estimate school 1, b00j b00 2.52** domain score, b10j b10 0.04** school 3, c00k c00 18.05** 13.88** students, eijk 2 124.74 93.94 model deviance 33091.96 31889.28 model df 4 13 note. ratio = student-teacher ratio. dollars = dollars spent per student. domain score = first-grade mathematical thinking domain score. school 1 = first-grade school attended. school 3 = third-grade school attended. for the unconditional model, df = 4,291 for fixed effects; df = 130 for school 1 random effect; and df = 136 for school 3 random effect. for the conditional model, df = 4,282 for fixed effects; df = 126 for mathematical thinking domain score random effect; and df = 132 for school 3 random effect. **p < .001. *p < .05. english language arts achievement. estimates for the fixedand random-effects for the unconditional and conditional models for english language arts achievement are presented in table 5. the mean english language arts achievement was 310.77 and the total variance was international electronic journal of elementary education vol.5, issue 2, 125-142, 2013 137 206.04 for the unconditional model. of the total variance, approximately 11% was between scores of students who attended the same first-grade school but a different third-grade school and approximately 2% was between scores of students who attended the same thirdgrade school but a different first-grade school. unlike mathematics achievement findings, differences in english language arts achievement could be attributed more so to the firstgrade school attended than to the third-grade school attended. similar to the findings for mathematics achievement, approximately 87% of the total variance was due to the within cross-classification of firstand third-grade school attended. for the conditional model, controlling for student and school demographic variables, language and literacy domain scores were found to be statistically significant predictors of third-grade english language arts achievement (θ1 = 1.12, p < .001). hence, we can expect, on average, a .37 standard deviation increase in third-grade english language arts achievement for each additional standard deviation increase in the language and literacy domain score. the inclusion of student and school demographic variables in the model decreased the studentlevel variance from 179.79 to 122.59 (approximately a 32% reduction) and decreased the firstgrade school attended variance from 22.91 to 6.57 (approximately a 71% reduction) but increased the third-grade school attended variance from 3.34 to 7.69 (approximately a 130% increase). table 5. estimation of fixed and random effects for based on cross-classification models for english language arts achievement unconditional model conditional model fixed effects coefficient se t ratio coefficient se t ratio intercept, 0 000 310.77** 0.57 547.03 311.69** 5.29 58.97 school 1 dollars γ01 0.00 0.00 0.55 school 1 ratio γ02 0.22 0.13 1.71 school 3 ratio 01 -0.09 0.15 -0.62 school 3 dollars 02 -0.00 0.00 -0.03 domain score, 1 100 1.12** 0.05 21.19 gender, 2 200 2.67** 0.35 7.69 iep, 3 300 -5.11** 0.56 -9.07 race, 4 400 -5.10** 0.43 -11.76 lunch, 5 500 -4.32** 0.43 -10.12 using first-grade teacher ratings to predict third-grade english language arts and mathematics achievement on a high-stakes statewide assessment / gallant 138 table 5 (continue). estimation of fixed and random effects for based on cross-classification models for english language arts achievement random effects estimate estimate school 1, b00j b00 22.91** 6.57** domain score, b10j b10 0.08** school 3, c00k c00 3.34* 7.69** students, eijk 2 179.79 122.59 model deviance 34648.67 33029.83 model df 4 15 note. ratio = student-teacher ratio. dollars = dollars spent per student. domain score = first-grade mathematical thinking domain score. school 1 = first-grade school attended. school 3 = third-grade school attended. for the unconditional model, df = 4291 for fixed effects; df = 130 for school 1 random effect; and df = 136 for school 3 random effect. for the conditional model, df = 4282 for fixed effects; df = 112 for school 1 random effect; df = 116 for language and literacy domain score random effect; and df = 132 for school 3 random effect. chi-square statistics were based on 117 of 131 first-grade schools that had sufficient data for computation. **p < .001. *p < .05. discussion this study applied a cross-classified random effects multilevel approach to determine the extent to which first-grade teacher ratings, as indicated by performance assessment domain scores, predicted third-grade student mathematics and english language arts achievement on a high-stakes, standards-based, criterion-referenced test. correlations between performance assessment domain scores and achievement subscale scores were also computed to address the research questions. results from this study provide support for the use of teacher ratings to predict students’ later achievement, within the context of a high-stakes accountability system. overall, positive medium associations were observed between first-grade performance assessment domain scores and third-grade achievement scale scores, controlling for students’ gender, iep status, race, and eligibility for subsidized meals. the associations found in this study presents about a 14% overlap in the variability among language and literacy and english language arts scores and about a 9% overlap in the variability among mathematical thinking and mathematics scores across a two-year period. these results are slightly smaller than those found by gallant (2009) and much smaller than those found by meisels et al. (1995). gallant (2009) found approximately an 18% overlap in the variability among the english language arts scores and approximately an 11% overlap in the mathematics scores over two years without controlling for student demographic variables. whereas, meisels et al. (1995) found within a one-year period at least a 56% overlap in the variability among kindergarteners’ fall work sampling system (wss) checklists (i.e., language and literacy and concept and number) scores and the spring kindergarten achievement battery of the international electronic journal of elementary education vol.5, issue 2, 125-142, 2013 139 woodcock-johnson psychoeducational battery-revised (wj-r) total score without controlling for student demographic variables. not controlling for student demographic variables or other extraneous variables may have contributed to the larger percentages of overlap in the gallant and meisels et al. studies. furthermore, the correlation coefficients obtained in this study are not atypical given the time span of the scores used. kim and suen (2003) suggests that a one-year prediction using ratings of early childhood cognitive ability is expected to have the highest predictive validity coefficients in predicting later achievement. determining predictive validity with time intervals longer than one year between early assessments and later achievement has the potential for poorer predictability compared to shorter time intervals. hence, the covariance estimates found in this study are reasonable considering the two-year time period between first-grade teacher observations and third-grade student achievement. the presence of overlap in variance among the instruments suggests some alignment of curriculum content standards across grade levels, as was the case for this state, and that similar content are being measured across years. moreover, controlling for student and school variables, performance assessment domain scores were statistically significant predictors of third-grade achievement. on average, the mean achievement scores are expected to increase about one-third of a standard deviation for each one-point increase in domain score. this finding was slightly lower than that found by gallant (2009). the author found about a .34 standard deviation increase in mathematics achievement and about a .43 standard deviation increase in english language arts achievement for each one-point increase in the performance domain score. the smaller standard deviations at the school level are possibly attributed to similarities in student scores across schools due to the cross-classification of students and schools and the fact that about 77% of students attended the same school both years. conclusions the findings from this study provide support for the use of a curriculum-embedded performance assessment, based on the work sampling system, within the context of a state accountability system. the potential long-term benefits to students in schools that use developmentally appropriate curriculum and assessments cannot be disregarded. for example, students exposed to a curriculum-embedded performance assessment for at least three years were found to display gains in reading and mathematics achievement from third to fourth grade, as measured by a norm-referenced test, compared to students who were not exposed to a curriculum-embedded performance assessment (meisels et al., 2003). hence, the alignment of early childhood practices, curriculum content, and assessment is essential to meeting the overall developmental needs of all children (e.g., naeyc & naecs/sde, 1991; shephard, kagan, & wurtz, 1998) as well as ensuring that students achieve later academic successes. it is important to remember that within early childhood and elementary school settings, principals and parents often rely on teachers to make decisions regarding the academic performance of students. the use of developmentally appropriate practices—such as using continuous observations instead of group-administered tests to assess students’ academic, social, emotional, and physical development in early childhood programs serving children from birth to age 8—has been advocated by early childhood professionals and professional organizations for decades (e.g., national association for the education of young children [naeyc], 1988, 1997, 2009; naeyc & national association of early childhood specialists in state departments of education [naecs/sde], 2003; national association of school using first-grade teacher ratings to predict third-grade english language arts and mathematics achievement on a high-stakes statewide assessment / gallant 140 psychologists, 2005). the use of continuous assessments in the early grades recognizes the continuum on which young children develop. information obtained from teacher assessments can be used to ensure that students are receiving not only the necessary academic services but also the necessary stimuli to enhance their overall development. using teachers as evaluators of students’ academic achievement is crucial because of teachers’ awareness and knowledge of curriculum content standards at the next grade levels (mashburn & henry, 2004). although the findings in this study were positive, the study was not without limitations. first, the use of domain score ranges as a criterion for inclusion in the study reduced the number of students per school in the study and potentially reduced the withinand betweenschool variances. future studies should consider using the mean ratings instead of domain scores to reduce the number of eliminated records. second, the retrospective nature of this study and the use of extant data made it impossible to determine what interventions, if any, were provided to students in the second grade based on first-grade ratings on the performance assessment. it was also unknown if schools had adopted any special reading or mathematics programs for third-grade students during the academic year and prior to the spring administration of the achievement tests. hence, the presence of special programs that schools may have implemented was not controlled for in the current study. . . . acknowledgements the author would like to thank theresa siskind, shiqi hao, and christine schneider for their assistance in providing the data used in this study, and james l. moore iii and mark allen for comments provided on an earlier version of the manuscript. dorinda j. gallant received the ph.d. degree in educational psychology and research (research track) from university of south carolina in 2005. she is an associate professor in the department of educational studies, quantitative research, evaluation and measurement since 2005. her research interests include applied measurement in elementary, secondary, and postsecondary education within the context of program, product, or personnel evaluation. references alpern, g., boll, t., shearer, m. (1986). developmental profile ii manual. los angeles: western psychological services. beretvas, s.n. (2008). cross-classified random effects models. in a. a. o’connell & d. b. mccoach (eds.), multilevel modeling of educational data (pp. 161-197). charlotte, nc: information age publishing, inc. cohen, j. (1992). a power primer. psychological bulletin, 112(1), 155-159. dichtelmiller, m.l., jablon, j.r., meisels, s.j., marsden, d.b., & dorfman, a.b. (1998). using work sampling guidelines and checklists: an observational assessment. ann arbor, mi: rebus. international electronic journal of elementary education vol.5, issue 2, 125-142, 2013 141 education accountability act of 1998, chapter 18 (1998). gallant, d.j. (2005). predictive nature of a curriculum-embedded performance assessment for young children (doctoral dissertation). university of south carolina, columbia, sc. gallant, d.j. (2009). predictive validity evidence for an assessment program based on the work sampling system in mathematics and language and literacy. early childhood research quarterly, 24, 133-141. doi: 10.1016/j.ecresq.2009.03.003 gresham, f.m., & elliott, s.n. (1990). social skills rating system. circle pines, mn: american guidance service. hecht, s.a., & greenfield, d.b. (2001). comparing the predictive validity of the first grade teacher ratings and reading-related tests on third grade levels of reading skills in young children exposed to poverty. school psychology review, 30, 50-69. hoge, r.d., & coladarci, t. (1989). teacher-based judgments of academic achievement: a review of literature. review of educational research, 59, 297-313. huynh, h., meyer, j.p., iii, & barton, k. (2000). technical documentation for the 1999 palmetto achievement challenge tests of english language arts and mathematics, grades three through eight. location: state department of education. huynh, h., prior, s.v., & gallant-taylor, d.j. (2002). technical documentation for the [state] readiness assessment of kindergarten and grade one. location: state department of education. jastak, j.f., bijou, s.w., & jastak, s.r. (1965). wide range achievement test. wilmington, de: guidance association. kim, j., & suen, h.k. (2003). predicting children’s academic achievement from early assessment scores: a validity generalization study. early childhood research quarterly, 18, 547-566. mashburn, a.j., & henry, g.t. (2004). assessing school readiness: validity and bias in preschool and kindergarten teachers’ ratings. educational measurement: issues and practices, 23(4), 16-30. meisels, s.j., atkins-burnett, s., xue, y., nicholson, j., bickel, d.d., & son, s-h. (2003). creating a system of accountability: the impact of instructional assessment on elementary children’s achievement test scores. education policy analysis archives, 11(9), retrieved on january 19, 2006, from http://epaa.asu.edu/epaa/v11n9/. meisels, s.j., bickel, d.d., nicholson, j., xue, y., & atkins-burnett, s. (2001). trusting teachers’ judgments: a validity study of a curriculum-embedded performance assessment in kindergarten to grade 3. american educational research journal, 38, 73-95. meisels, s.j., jablon, j., marsden, d.b., dichtelmiller, m.l., & dorfman, a.b. (1994). the work sampling system: an overview (3rd ed.). ann arbor, mi: rebus inc. meisels, s.j., jablon, j., marsden, d.b., dichtelmiller, m.l., & dorfman, a.b. (2001). the work sampling system: an overview (4th ed.). ann arbor, mi: rebus inc. meisels, s.j., liaw, f., dorfman, a., & nelson, r.f. (1995). the work sampling system: reliability and validity of performance assessment for young children. early childhood research quarterly, 10, 277296. national association for the education of young children. (1997). developmentally appropriate practice in early childhood programs serving children from birth through age 8 [on-line]. washington, dc: authors. retrieved may 2, 2007 from www.naeyc.org/about/positions.asp. national association for the education of young children. (1988). position statement on standardized testing for young children 3 through 8 years of age. young children, 43(3), 42-47. national association for the education of young children. (2009). developmentally appropriate practice in early childhood programs serving children from birth through age 8 [on-line]. washington, dc: authors. retrieved may 1, 2009, from http://www.naeyc.org/about/positions.asp. using first-grade teacher ratings to predict third-grade english language arts and mathematics achievement on a high-stakes statewide assessment / gallant 142 national association for the education of young children and national association of early childhood specialists in state departments of education. (1991). guidelines for appropriate curriculum content and assessment in programs serving children ages 3 through 8. young children, 46(3), 21-38. national association for the education of young children and national association of early childhood specialists in state departments of education. (2003). early childhood curriculum, assessment, and program evaluation: building an effective, accountable system in programs for children birth through age 8 [on-line]. washington, dc: authors. retrieved january 19, 2006, from http://www.naeyc.org/about/positions.asp. national association of school psychologists. (2005). position statement on early childhood assessment [on-line]. bethesda, md: authors. retrieved may 2, 2007, from http://www.nasponline.org/about_nasp/pospaper_eca.aspx. quay, l.c. & steele, d.c. (1998). predicting children’s achievement from teacher judgements: an alternative to standardized testing. early education & development, 9(3), 207-218. raudenbush, s.w., & bryk, a.s. (2002). hierarchical linear models: applications and data analysis methods (2nd ed.). thousand oaks, ca: sage publications, inc. raudenbush, s.w., bryk, a.s., & congdon, r. (2004). hlm 6.0: hierarchical linear and nonlinear modeling [computer software]. chicago, il: scientific software international. shephard, l., kagan, s., & wurtz, e. (1998). principles and recommendations for early childhood assessments [on-line]. washington, dc: national education goals panel. retrieved january 19, 2006, from http://govinfo.library.unt.edu/negp/reports/prinrec.pdf. snijders, t.a.b., & bosker, r.j. (1999). multilevel analysis: an introduction to basic and advanced multilevel modeling. thousand oaks, ca: sage publications inc. stevenson, h.w., parker, t., wilkinson, a., hegion, a., & fish, e. (1976). predictive value of teachers’ ratings of young children. journal of educational psychology, 68(5), 507-517. woodcock, r.w., & johnson, m.b. (1989). woodcock-johnson psychoeducational battery-revised. allen, tx: dlm teaching resources. international electronic journal of elementary education vol.5, issue 2, 125-142, 2013 143 www.iejee.com this page is intentionally left blank http://www.iejee.com/ microsoft word 4_iejee_4_1_pae_sevcik international electronic journal of elementary education, 2011, 4(1), 47-65. issn:1307-9298 copyright © iejee www.iejee.com the role of verbal working memory in second language reading fluency and comprehension: a comparison of english and korean hye k. pae ∗∗∗∗ university of cincinnati, united states rose a. sevcik georgia state university, united states abstract this study examined the respective contribution of verbal working memory, which was operationalized as immediate digit and sentence recall, to bilingual children’s reading fluency and comprehension in the first language (l1) and second language (l2). fifty children from two international sites took part in this study: one group was english-korean bilinguals in the u.s., while the other was korean-english bilinguals in korea. the manifestation of the prediction model varied across the learning contexts or learner groups. l1 forward and backward digit spans accounted for the significant variances in l2 reading fluency and comprehension for the english-speaking children in the u.s., whereas l1 forward digit span was more predictive of l2 reading fluency and comprehension than backward digit span and sentence recall for the korean-speaking counterparts in korea. the results were interpreted with respect to the orthographic depth, linguistic differences, and cognitive demands. implications and future directions are discussed. keywords: reading fluency and comprehension; english-korean bilinguals; korean-english bilinguals; verbal memory introduction since baddeley and hitch’s publication (1974) on the construct of working memory, researchers have investigated not only the nature, structure, and function of working memory but also its relation to children’s language and reading acquisition (daneman & carpenter, 1980; gathercole & baddeley, 1993; windfuhr & snowling, 2001). working memory refers to the transitory storage capacity and operations that manipulate verbal or written input while processing incoming information and retrieving relevant phonological information from the long-term lexicon (miller & kupfermann, 2009). it works as a processing ∗ hye k. pae, school of education, university of cincinnati, cincinnati, oh 45221, united states. phone: 513556-7112, e-mail: hye.pae@uc.edu international electronic journal of elementary education 48 source for the active maintenance of task-relevant information while simultaneously processing the same or other information activated along with task operations (swanson, zheng, & jerman, 2009). under the working memory model (baddeley, 1986), verbal shortterm memory or the phonological loop captures a subset of working memory performance. this subsystem is usually measured using immediate serial recall tasks in which the examinee is verbally presented with the sequence of isolated digits or words (gupta, 1996). learning relies on an individual’s ability to conceptualize and categorize new information and to make associations with other information housed in mental storage. given the importance of mental storage, working memory has received abundant attention over time. limited auditory memory span is one of the sources of deficiencies in language processing because a reduced memory span can inhibit the efficiency of working memory, which is necessary for processing and comprehending extended verbal narrations (gathercole & baddeley, 1993; miller & kupfermann, 2009). there have been conflicting views on the definitions of working memory and short-term memory. some researchers (colom, flores-mendoza, quiroga, & privado, 2005; gathercole, 1998; unsworth & engle, 2007) have differentiated working memory from short-term memory, while others (mcdougall, hulme, elllis, & monk, 1994) have used the two terms interchangeably. the former’s rationale is based on the claim that working memory relies on the central executive system with a heavy demand of information manipulation. in other words, short-term memory works to retrieve a sequence of items in the order in which information is stored without manipulations, while working memory requires recalling information that is transformed from its initial encoding in order to perform task-relevant operations through manipulations (swanson, zheng, & jerman, 2009). the latter upholds a claim that working memory and short-term memory share some commonalities on the learner’s part with regard to the speech-based phonological input memory bank and extratransformational processes. regardless of the position on the relationship between working memory and short-term memory, working memory and short-term memory are forms of transient memory. since this study was not designed to test memory models, verbal memory was operationalized as transitory immediate digit and sentence recall or memory in this paper. the purpose of this study was to investigate the manifestation of cognitive demands necessary for first language (l1) and second language (l2) reading. relationships between phonological and verbal memory and reading given the way that phonological coding contributes to the retention of information in working memory and vice versa, studies have demonstrated a significant association between verbal memory span and word reading (jeffries & everatt, 2004; johnston, rugg, & scott, 1987; jorm, share, maclean, & matthews, 1984). deficits in verbal working memory and slow or imprecise word recognition impede higher-order processes, such as semantic retrieval and syntactic judgment, due to a lack of residual cognitive space. as a result, it leads to significantly reduced reading comprehension (katz, shankweiler, & liberman, 1981; stanovich, siegel, & gottardo, 1997). because neighborhood candidates (e.g., phonology, spelling patterns, syntax, and semantic properties) are activated upon the stimulus of the text in the face of interference (swanson, zheng, & jerman, 2009), effective activation and inhibition processes through the maintenance of pertinent memory traces are crucial while reading. therefore, the efficiency of the informational filtering system through memory traces facilitates reading comprehension. due to its phonological processing and the maintenance of task-relevant information in an active state, verbal memory is robustly related to the performance of complex cognitive tasks, such as vocabulary acquisition and reading (baddeley, 1986). research has the role of verbal working memory / pae & sevcik 49 demonstrated that verbal short-term memory abilities and reading acquisition have a shared set of processes and utilize a common cognitive system (adams & gathercole, 1996; gupta, 1996; gupta & macwhinney, feldman, & sacco, 2003; henry & maclean, 2003). a reliable relationship has been observed between digit span recall and word knowledge even when other variables, such as age and iq scores, were controlled for (gernsbacher, 1990; gupta, 1996). gupta (1996) has attempted to explain the relation between immediate serial recall and word learning using a computational model in which a general sequencing mechanism provides immediate memory for the sequence of word forms. the model consists of three vital levels, including a phoneme layer, a phonological-chunk layer, and a semantic/context layer. the phoneme layer relates to output phonology at which phonemes are represented. the phonological-chunk layer entails the representations for word forms which are shared by input and output phonology. the last level, the semantic/context layer, represents semantic and contextual information about word forms (see gupta, 1996, for details). according to gupta’s (1996) model which takes a new processing-oriented approach to examining word learning, the significant relationship between immediate serial recall and word learning lies in the common dependence of these two capabilities on core phonological and semantic processing mechanisms. as a whole, verbal memory is indispensable with regard to reading in that textual inputs go through multiple processes, including encoding and retrieval of phonological, orthographic, and semantic referents stored in the mental lexicon (gupta, 1996; swanson, saez, & gerber, 2006). through these processes, activated information about words, phrases or sentences should be sustained for a short period of time in order for the contents to be integrated into a context. daneman and carpenter (1980) developed a reading-span test (rst) to investigate the contribution of working memory to reading comprehension. in the rst, participants read a few sentences aloud and were asked to remember the last word of each sentence. due to the limited capacity of working memory, the mental resources to read a sentence and store the last word of each sentence were limited. since participants needed to allocate mental resources efficiently during the task, the rst measured storing and processing capabilities concurrently upon reading. there was a significant correlation found between the rst score and reading comprehension performance (daneman & carpenter, 1980). a meta-analysis of studies using working memory-span tasks conducted by daneman and merikle (1996) also showed that working-memory-span capacity was correlated significantly with reading skills. there are individual differences in working memory capacity and these differences are related, in part, to language comprehension (daneman & carpenter, 1983; just & carpenter, 1992). daneman and carpenter (1983) have suggested that the semantic processing of sentence comprehension is attributable to individual differences in memory capacity. students who show efficient working memory retrieval (i.e., higher scores on the rst) are more likely to make use of fewer resources for the semantic processing of sentences, and, as a result, have sufficient resources to retain the words. in contrast, participants who show inefficient working memory (i.e., lower scores on the rst) have difficulty retaining target words due to the insufficient working memory capacity during reading. similarly, from a capacity-oriented perspective, skilled readers make more text-based inferences because the multiple sources of processing, such as reading a sentence and storing the last word of each sentence at the same time, are available in working memory. hence, high functioning students with greater working memory capacity are able to sustain more information international electronic journal of elementary education 50 necessary to complete the given task, such as reading, because they successfully utilize the efficient activation of semantic and syntactic information (budd, whitney, & turley, 1995). l1 and l2 interdependence and the korean language a myriad of studies have demonstrated cross-language links in the acquisition of language and reading skills, and been explained by the cross-linguistic interdependence hypothesis (cummins, 1994). phonological processing skills, for example, can transfer from one language to another (gottardo, yan, siegel, & wade-woolley, 2001; pae, sevcik, & morris, 2010). if one has robust l11 oral skills and reading proficiency, the likelihood for the child to gain success in l2 oral performance and reading is greater than those who lack those skills. within the roman alphabetic languages, children are able to make use of commonalities between l1 and l2, when they learn l2. although universality exists in the acquisition of languages, language-specific variations are observed, because learning to read accommodates how a writing system maps phonological properties in the language under consideration (pae, 2011; perfetti & liu, 2005). therefore, it is possible that working memory and l1 ability may influence the learning of some second languages more than others, particularly if those languages use different forms of script or orthography. the ability to control attention for the active maintenance of given information and the inhibition of irrelevant information is viewed as a domain-general construct (kane et al., 2004; payne, kalibatseva, & jungers, 2009). with robust evidence that verbal memory skills contribute to l1 reading comprehension (daneman & carpenter, 1980, 1983; kane et al., 2004; swanson & berninger, 1995), the significant role of working memory has been expanded to the prediction of l2 language processing and reading comprehension. for instance, independent contributions of working memory for hebrew-speaking high-school students were found in their learning english as l2 (abu-rabia, share, & mansour, 2003). a study of japanese-speaking students evidenced a significant correlation in working memory between l1 and l2, as well as a mediator effect of l2 working memory on the relationship between l1 working memory and l2 syntactic comprehension (miyake & friedman, 1998). significant relationships among variables of working memory, l1 reading comprehension, and l2 reading comprehension were also found in english-speaking college students’ spanish comprehension (payne, kalibatseva, & jungers, 2009). a salient contribution of verbal working memory, measured using memory span and tongue twister, to text comprehension by chinese children has also been reported (leong, tse, loh, & hau, 2008). the significant role of phonological memory has been expanded to l2 speech production for english-speaking adults who were learning spanish as l2. o’brien, segalowitz, collentine, and freed (2006) found that phonological memory (serial nonword recognition) explained significant amounts of the variance in l2 narrative oral skills for both less proficient and more proficient adults, after controlling for speech output. in cross-language research, a study of english and korean offers an excellent opportunity to examine between-language interdependence because the two languages share the alphabetic principle (pae, 2011; ziegler & goswami, 2005), but exhibit profound differences in their visual lexical form (linearity vs. block layout) as well as orthographic, phonological, and linguistic features. korean is noticeably different from other languages in its origin; the 1 since l1 is to be processed automatically, the automaticity can be a way to determine the child’s l1. in this study, a dominant language and l1 are operationalized as the language in which a child knows instantly, intuitively, and effortlessly, when he/she processes an utterance in a language, regardless of the first language to which the child was exposed. hence, l1 was used throughout the paper for the sake of consistency with previous studies in order to refer to a dominant language. the role of verbal working memory / pae & sevcik 51 korean writing system was invented and promulgated by king sejong in 1443, rather than being evolved over time. the key differences between english and korean include lexicality and orthographic depth, writing systems, phonological differences, syntactic word order in sentences, and subject and/or object omissions in sentences. english is considered to be a deep orthography (katz & frost, 1992), while korean is a shallow orthography in which any words, even if unfamiliar words or nonwords, can be sounded out quickly and accurately. the korean writing system, an alphabetic syllabary (pae, 2011; taylor, 1980, 1997), is unlike those of most other roman alphabetic languages. the graphemes are not written one after another in a horizontal line, but form a square-like syllable block consisting essentially of a phonetic syllable (e.g., rather than ). the number of korean syllables currently in use in korea is 2,457 (see pae, 2011 for details), which is larger than chinese which contains about 400 syllables or 1,300 syllables with tones (taylor, 1997) and japanese that includes fewer than 113 syllables (taylor, 1997). however, the number of syllables is much smaller than that of english which includes about 8,000 syllables (katz & frost, 1992). korean phonemes do not include the labio-dentals, interdentals, sibilants, or retroflexes that are found in english. korean falls into the ural-altaic family of language, with an agglutinative structure of grammar (i.e., nouns and verbs are formed by attaching derivational and/or inflectional prefixes and suffixes to the root). the basic syntactic structure of korean, like japanese, is a predicate-final structure with the basic word order of subject-object-predicate (verb or adjective). in other words, korean is a verb/predicate-final language (i.e., a language in which the verb or predicate always comes at the end of the sentence; sohn, 1999). the korean language is a context-rich language in that most of the sentences do not carry subjects, and, often times, objects are also omitted. the listener is forced to rely on the context in which the utterance takes place in order to comprehend the true meaning of the sentence. for example, “i love you” in english can be said “love” in korean. with the omissions of the subject and the object, both the speaker and the listener are able to decipher the meaning of the utterance without problems in korean by relying on the context in which communication occurs. the present study despite the comparatively wide knowledge base on literacy acquisition in linearly arranged alphabetic languages, little is known about literacy development in writing systems with non-linear symbol arrangements. korean is an outstanding example of alphasyllabic languages, as briefly noted earlier, because it entails the alphabetic principle (i.e., a grapheme maps onto a phoneme and strings of graphemes form a syllable) as well as a distinct visual syllabic structure with an adhesive rule of a consonant to a vowel. no consonant strings are allowed, except five doublets ( ) and legal consonantconsonant forms at the bottom of the top-down syllable (see pae, 2011, for more details). given that reading is a high-order cognitive function, the aim of this study was to investigate the independent contributions of three different types of verbal working memory to l2 reading fluency and comprehension via a comparison of english and korean. two research questions were posed as follows: 1. what is the individual contribution of phonological working memory, verbal digit working memory, and sentence verbal working memory to l1 and l2 reading for english-speaking bilinguals in the u.s. who learn korean as l22? 2 again, for the sake of consistency, l1 and l2 were used in this study, although the u.s. children might have exposed to korean first, regardless the dominance of language skills in english and korean. the u.s. participants’ international electronic journal of elementary education 52 2. what is the individual contribution of phonological working memory, verbal digit working memory, and verbal sentence working memory to l1 and l2 reading for korean-speaking bilinguals in korea who learn english as l2? on the basis of the findings of previous research, it was assumed that phonological working memory, measured using forward digit spans, has more of a working memory load than the sentence task for children, while verbal digit working memory has an extra transformational processing element and has the highest demand for working memory processes. verbal sentence working memory includes a rich set of contextual and semantic information, allowing for long-term memory traces that should free up working memory to some extent. this task may have the least load on working memory process. therefore, it was hypothesized that verbal digit working memory was more predictive for comprehension than phonological working memory and sentence recall skills. although the literature indicates a significant relationship between l1 verbal memory and l2 reading comprehension in roman alphabetic languages, there has been little exploration of the role of verbal working memory in accounting for reading comprehension in an alphasyllabic language. we hypothesized that verbal memory capacity in l1 would be predictive of reading achievement in both l1 and l2, but the extent to which verbal memory played a role in english and korean performance would be different when learning the language as l2. because korean has a consistent grapheme-phoneme correspondence, korean involves direct lexical access in text processing with a little phonological mediation3. given the grapheme-phoneme consistency in korean, it was also hypothesized that the amount of variance explained by verbal memory would be smaller in korean than that in english when learning the language as l2. we also examined whether differences exist between english-speaking bilinguals and their korean-speaking counterparts in terms of the strength of relationship between l1 and l2. this analysis would lead to a better understanding of l2 acquisition in learning contexts: one in an english-as-a-second-language (esl)4 context by english-korean bilinguals and the other in an english-as-a-foreign-language (efl) context by korean-english bilinguals. if the language system had an influence on reading performance, verbal memory would play a role in a different way across sites where the l1 is different from each other. we expected a different pattern of relationship between verbal working memory and reading (i.e., different amount of the variance explained in the prediction model) across the two sites. methods participants fifty firstand second-grade children took part in this study. the participants consisted of two groups. the first group comprised 29 korean immigrant children residing in the u.s. who home language was predominantly korean, although the children mostly communicate in english at school and at home in which korean was spoken by the participant’s parent. 3 the dual-route hypothesis, including the lexical route (a.k.a., addressed, word-detector, or lexical look-up route) and the sublexical route (a.k.a., nonlexical, assembly, rule-based, or phonologically mediated procedure), offers an explanation of mapping print into sound. since the word processing route was beyond the scope of this study, the two routes were not considered in this study. 4 although it can be questionable as to whether the korean-immigrant children in the u.s. were actually learning korean as an l2-equivalent language, the questionnaire by participant’s parent and a brief interview with the child pointed to language automaticity established in english. see results for further information on the language dominance. the role of verbal working memory / pae & sevcik 53 were learning korean as l2 (english-korean bilinguals) within a korean educational program (i.e., a saturday school) in a metropolitan area in the southeastern united states. the second group was composed of 21 korean natives who lived in korea and learned english as l2 (korean-english bilinguals) in the formal education setting in busan, south korea. in order to match the socioeconomic status across the sites, the participants in south korea were restricted to students who were attending a private elementary school. parental informed consent and child assent were obtained for all the participants. the english-korean learners’ mean age was 91.82 months (sd = 8.62; 15 males, 14 females). the korean-english counterparts’ mean age was 87.66 months (sd = 5.92; 13 males, 8 females). the parent questionnaire and the child oral report indicated that all the participants were not exposed to other languages beyond the two languages under consideration. it should be noted that the english-korean bilingual children in the u.s. might have been exposed to korean first because their parents spoke korean at home. the questionnaires completed by the parent and brief interviews with the children indicated that the parent typically spoke to the child in korean, and the child responded to his/her parent in english. hence, the strength of their language skills was shown in english, and their oral production in korean was laborious, which was a typical indication of l2. because of the wide range of variability in both home-language use and language dominance for the english-korean speaking bilinguals, the u.s. participants’ dominant language was validated using object picture naming latency in l1 and l2. this validation procedure was derived from the premise that object naming in l1 is typically faster than l2, due to the additional processing demand to resolve a competition from l1 candidates (kroll & stewart, 1994). the results showed that the participants took significantly longer naming objects in l2 than l1 [u.s. participants: t(1,28) = 7.70, p < .00], validating the participants’ and parents’ self-reports. since the korean native students spoke korean as their l1, the validation procedure was not necessary. procedure a test battery, which included measures of forward and backward digit recall, sentence recall, and reading skills, was individually administered at the korean schools or the participant’s home by a bilingual examiner. before the test administration, a brief interview with the child was performed, in addition to child assent. measures based on the protocols of english measures which were u.s. norm-referenced, the korean measures were experimentally designed to achieve comparability with the english version. word frequency, face validity, inter-item consistency, and test-retest reliability were taken into account to overcome potential limitations and to maximize the cross-cultural comparability. for the experimental korean measures, internal consistency and test-retest reliability coefficients ranged from .79 to .91. predictor measures phonological working memory. the forward digit span subtest of the wechsler intelligence scale for children-third edition (wisc-iii; wechsler, 1991) was used as a measure of phonological working memory. the forward digit span task required an immediate recall of auditory number strings forwards at a rate of one digit per second and asked the child to simply repeat the list of digits as heard. this task involves a phonological memory reservoir that stores in-coming information passively for a few seconds, and then produces articulatory output under constraints. international electronic journal of elementary education 54 verbal digit working memory. the backward digit span subtest of the wisc-iii (wechsler, 1991) was used as a measure of verbal digit memory. this test asked the child to say the list of numbers in a reversed order for an instant recall. since it was designed to measure the participant’s ability to store and manipulate digits that were in temporary short-term memory, this task entailed an extra-transformational processing element. it required the child not only to recite the presented information, but also to make some transformation. the additional processing on top of memory makes this task quite powerful and consistent predictor of a wide array of high order processes related to fluid intelligence, such as reading comprehension. for the forward and backward digit span tests, a set of two lists with the same length was provided, and if at least one of the two lists was correctly produced, the subsequent list was increased by one digit until the child reached the ceiling. the internal consistency values and test-retest reliability coefficients for age groups of 6 to 9 were high, ranging from .89 to .94. the same protocol was used for the korean measure, but the digits were presented in korean by the examiner and the participants responded in korean as well. since the articulation of the number in a language involves its phonological coding, using the same stimuli in the two languages maintains a unitary construct. verbal sentence short-term memory. the sentence repetition subtest of a developmental neuropsychological assessment (korkman, kirk, & kemp, 1998) assessed the ability to recall sentences of increasing complexity and length. the examinee was asked to repeat verbatim the sentence presented by the examiner. the generalizability coefficients for children of 6 to 9 years of age ranged from .72 to .85. the stability coefficient was .91 for that age group. the korean measure of sentence repetition was constructed on the basis of the english version, by attending to the word and sentence lengths and the semantic complexities of the given sentences. the internal consistency coefficient was .81 for the korean sentence repetition test. outcome measures reading fluency. the fluency subtest of the gray oral reading test-4 (gort-4; wiederholt & bryant, 2001) was employed as a reading fluency assessment tool. the gort-4 is composed of 4 subtests, including reading rate, accuracy, fluency, and comprehension; the fluency score is a composite score of rate and accuracy scores. the gort-4 was designed to determine the particular reading strengths and weaknesses that individual students possess and to serve as a measurement device in assessing reading ability (wiederholt & bryant, 2001). reading comprehension. the comprehension subtest of the gray oral reading test-45 (gort4; wiederholt & bryant, 2001) was employed as a reading comprehension assessment tool. the gort-4 evidences a high degree of reliability, which was consistently high across all three types of reliability: content sampling ranged from .91 to .97; test-retest, .85 .95; and scorer differences, .94 .99. for the korean version of the reading fluency and comprehension measures, form b stories of the gort-4 were translated into korean so that the participants could receive an equivalent fluency and comprehension measures in the two languages. 5 there have been concerns about content validity and concurrent validity of the gort-4 (keenan & betjemann, 2006). however, keenan & betjemann (2006) administered the gort passages to college students to examine the extent to which gort questions were passage-independent items. since our participants’ ages fell in the range of the gort-4’ normative sample which was 6-18 years of age, we followed the gort manual. the role of verbal working memory / pae & sevcik 55 results descriptive statistics and correlations a descriptive analysis of all data was initially conducted, including an examination of variable means and standard deviations, outlier checks, and the distribution of scores. since the focal point centered on reading fluency and comprehension in relation to verbal memory, word and nonword identification skills were not included as variables in this study. the distribution of the raw scores on the reading tests was slightly positively and negatively skewed according to the measures, but no outstanding differences were found for the planned inferential statistical analyses. due to cross-cultural measurement issues given that no measures utilized were normed in korea, raw scores were used for analyses. as expected, the participants consistently demonstrated higher performance in their l1 than l2, which was a typical pattern found in previous research (pae, sevcik, & morris, 2004, 2010). the two groups did not differ in age as assessed by a t-test corrected for unequal variances, t = 1.91, p > .05. although the u.s. children slightly outperformed the korean counterparts, the children’s performance on the measures was not significantly different for the u.s. and korean groups, except for l1 sentence repetition and l1 backward digit span (f = 8.17, p < .01; f = 9.90, p < .01, respectively). table 1 shows the means and standard deviations for l1 and l2 measures. the reading measures, including fluency and comprehension, did not differ between the two groups. table1. descriptive statistics english-korean bilinguals korean-english bilinguals mean sd range mean sd range l1 forward digit span 7.69 1.83 5-11 7.19 1.57 5-10 l2 forward digit span 5.90 1.49 4-9 6.05 1.32 4-8 l1 backward digit span 5.55 1.84 2-9 4.67 1.15 3-7 l2 backward digit span 3.83 1.23 2-6 3.71 1.27 2-6 l1 sentence repetition 16.72 3.99 9-26 13.57 3.64 8-20 l2 sentence repetition 10.10 4.04 3-19 8.81 2.44 6-14 l1 reading fluency 40.69 17.39 1-71 40 15.93 16-84 l2 reading fluency 8.10 7.52 0-25 11.90 7.65 0-35 l1 reading comprehension 20.28 11.09 0-42 16.10 8.65 5-40 l2 reading comprehension 6.21 3.89 0-13 6.81 4.02 0-18 since each participant was tested on l1 and l2 skills across the two sites, and provided scores for each permutation of the variables, anova was performed with two foci. the first focus was placed on the difference between the learner groups (i.e., esl u.s. vs. efl korea) to examine main effects and interactions with site (u.s. vs. korea) as the between-subject factor and language (l1 vs. l2) as the within-subject factor. the results revealed a main effect of language (f(1, 48) = 42.21, p = .000, partial η2 = .32), indicating that there are differences in the means of reading fluency skills between the two learner groups when ignoring other factors. there was an interaction effect, indicating that the children’s performance was not the same at the two sites [f(1,48) = 5.27, p = .026, partial η2 = .10]. main effects for sentence repetition, forward digit span, and backward digit span were significant. there were interaction effects for forward and backward digit span [f(1, 48) = 4.45, p = .04, partial η2 = .09; f(1, 48) = 4.38, p = .04, partial η2 = .08, respectively], suggesting that the verbal memory skills in digit span were not uniform. the two-way interaction between the two sites for the sentence repetition task did not reach significance [f(1, 48) = 3.52, p = .07, partial η2 = .07]. international electronic journal of elementary education 56 for the reading measures, there were significant main effects for reading fluency and reading comprehension performance. a markedly significant interaction implied that this pattern of variance was different for the two participant groups. the second focus was placed on the language command (i.e., l1 vs. l2). since the language command was a within-subject variable, the main effect and interaction effect were not reported. the l1 backward digit span and l1 sentence repetition were significantly different across language commands (f(1,48) = 9.90, p = .003, f(1,48) = 8.17, p = .006, respectively). zero-order correlation coefficients among the variables were obtained. table 2 reports the bivariate correlation coefficients in the lower-left triangle and the partial correlation coefficients controlling for age in the upper-right triangle for the english-speaking participants. the partial correlation coefficients were computed because verbal memory is age-sensitive. for the u.s. students, there were moderate to high significant correlations among the variables under consideration after controlling for age (r ranges: .41 .85, p < .05). interestingly, the sentence repetition performance showed the lowest correlations with the other variables. for the korean counterparts, the pattern of the significant correlations among the variables was different from that of the english-speaking participants (see table 3). korean (l1) sentence recall proficiency was significantly correlated with korean (l1) and english (l2) backward digit span (r = .52 and .57, respectively) after controlling for age. overall, the korean natives’ scores showed comparatively lower correlations among the variables under consideration than those of the u.s. children. table 2. correlations among the variables below the diagonal and partial correlations controlling for age above the diagonal for the english-speaking children in the u.s. 1 2 3 4 5 6 7 8 9 10 1. l1 forward digit span 1 .85*** .51** .43* .62*** .38* .65*** .47* .28 .49** 2. l2 forward digit span .84*** 1 .45* .34 .48* .49** .59** .43* .23 .56** 3. l1 backward digit span .51** .46* 1 .59** .50** .32 .64*** .56** .62*** .54** 4. l2 backward digit span .44* .36 .61** 1 .44* .23 .45* .41* .51** .44* 5. l1 sentence repetition .61*** .47* .45* .41* 1 .54 .63*** .36 .36 .15 6. l2 sentence repetition .35 .45* .23 .17 .54** 1 .49** .43* .37 .36 7. l1 reading fluency .65*** .59** .64*** .46* .62** .44* 1 .71*** .70*** .64*** 8. l2 reading fluency .47* .42* .52** .40* .37 .42* .70*** 1 .70*** .79*** 9. l1 reading comprehension .25 .19 .48** .42* .36 .41* .63*** .68*** 1 .64*** 10. l2 reading comprehension .49** .57** .55** .45* .14 .32 .64*** .78*** .58** 1 *p < .05; ** p < .01; *** p < .001 the role of verbal working memory / pae & sevcik 57 table 3. correlations among the variables below the diagonal and partial correlations controlling for age above the diagonal for the korean-speaking children in korea 1 2 3 4 5 6 7 8 9 10 1. l1 forward digit span 1 .48* .31 .54* .37 .58** -.03 .58** -.06 .50* 2. l2 forward digit span .57** 1 .48* .70** .40 .36 -.02 .30 .18 .36 3. l1 backward digit span .39 .57** 1 .63** .52* .66** -.13 .23 -.27 .07 4. l2 backward digit span .61** .75*** .68** 1 .57** .50* .16 .44 .12 .42 5. l1 sentence repetition .47* .56** .60* .64** 1 .54* .06 .22 -.15 .09 6. l2 sentence repetition .64** .50* .70*** .58** .63** 1 -.04 .59** -.17 .39 7. l1 reading fluency .08 .15 -.02 .25 .18 .08 1 .16 .84*** .29 8. l2 reading fluency .64** .45* .33 .52* .36 .65** .25 1 .03 .87*** 9. l1 reading comprehension -.03 .19 -.23 .14 -.10 -.13 .83*** .05 1 .29 10. l2 reading comprehension .51* .36 .10 .43 .13 .40 .31 .84*** .30 1 *p < .05; ** p < .01; *** p < .001 hierarchical regression analyses for reading fluency and comprehension in order to explore the respective contribution of the phonological and verbal memory in l1 and l2 reading skills, a series of hierarchically nested regressions were performed separately for each learner group. after controlling for age, forward digit span capacity was entered first, backward digit span second, and then sentence repetition was entered last into a model as predictor variables. reading fluency and comprehension skills served as dependent variables. the within-language analyses of l1-l1 and l2-l2 as well as the between-language analysis of l1-l2 were performed. the results of hierarchical regression analyses for the english-speaking children are shown in table 4. the english forward digit span accounted for 41% of the unique variance in english reading fluency (f change (1,26) = 19.14, p = .000), and the english backward digit span explained an additional 13% of the variance in english reading fluency (f change (1,25) = 7.04, p = .014). the english backward digit span was predictive of 29% of the unique variance of english reading comprehension (f change (1,25) = 12.53, p = .002). the l2 korean forward digit span was also a significant predictor of l2 korean fluency and comprehension skills (17% and 31% of the unique variance, respectively). when it came to cross-language prediction, the english forward and backward digit span tasks explained the unique variances in both korean reading fluency and reading comprehension. the english (l1) forward digit span task explained 23% and the english backward digit span measure accounted for 13% of the additional unique variance in korean (l2) reading fluency (f change (1,26) = 7.54, p = .010; f change (1,25) = 5.35, p = .029, respectively). the english (l1) forward span task explained 23% of the unique variance and the backward digit span test accounted for an additional 11% of the unique variance in korean (l2) reading comprehension (f change (1,26) = 8.26, p = .008; f change (1,25) = 4.31, p = .048, respectively). international electronic journal of elementary education 58 table 4. hierarchical regression models for english-korean bilinguals in the u.s. dependent variables predictors r2 △r2 β t within language (english-english) l1 fluency step 1. age .02 .02 ns ns step 2. l1 forward digits .43*** .41*** .44 2.81* step 3. l1 backward digits .56*** .13* .44 2.65* step 4. l1 sentences .60*** .04 ns ns l1 comprehension step 1. age .07 .07 .46 2.73* step 2. l1 forward digits .14 .07 ns ns step 3. l1 backward digits .43** .29** .63 3.21** step 4. l1 sentences .44** .01 ns ns within language (korean-korean) l2 fluency step 1. age .00 .00 ns ns step 2. l2 forward digits .18 .18* ns ns step 3. l2 backward digits .26 .08 ns ns step 4. l2 sentences .32 .06 ns ns l2 comprehension step 1. age .02 .02 ns ns step 2. l2 forward digits .33** .31** .42 2.22* step 3. l2 backward digits .39** .06 ns ns step 4. l2 sentences .40* .01 ns ns between languages (english-korean) l2 fluency step 1. age .00 .00 ns ns step 2. l1 forward digits .23* .23* .48 7.54* step 3. l1 backward digits .36* .13* .46 2.20* step 4. l1 sentences .36* .00 ns ns l2 comprehension step 1. age .02 .02 ns ns step 2. l1 forward digits .25* .23** .48 2.36* step 3. l1 backward digits .36** .11* .51 2.64* step 4. l1 sentences .45** .09 ns ns note * p < .05; ** p < .01; *** p < .001 ns = not significant as seen in the correlation matrix, a difference of the prediction was also found in the regression analysis for the korean-speaking children from the english-speaking counterparts (see table 5). as hypothesized, the predictive power of the phonological and verbal memory skills in reading fluency and comprehension diminished for the korean-speaking children. in the korean (l1) within-language relationship, no predictors accounted for the significant variance in korean reading fluency and comprehension. the english (l2) sentence repetition task explained 15% of the unique variance in english reading fluency (f change (1,16) = 4.59, p = .048). the korean (l1) forward digit span explained a unique variance of 28% in english (l2) reading fluency and an additional 24% of the variance in l2 reading comprehension. the role of verbal working memory / pae & sevcik 59 table 5. hierarchical regression models for korean-english bilinguals in korea dependent variables predictors r2 △r2 β t within language (korean-korean) l1 fluency step 1. age .08 .08 ns ns step 2. l1 forward digits .08 .00 ns ns step 3. l1 backward digits .09 .01 ns ns step 4. l1 sentences .11 .02 ns ns l1 comprehension step 1. age .01 .01 ns ns step 2. l1 forward digits .01 .00 ns ns step 3. l1 backward digits .08 .07 ns ns step 4. l1 sentences .08 .00 ns ns hierarchical regression models for korean-english bilinguals in korea within language (english-english) l2 fluency step 1. age .15 .15 step 2. l2 forward digits .23 .08 step 3. l2 backward digits .32 .09 step 4. l2 sentences .47* .15* .49 2.14* l2 comprehension step 1. age .02 .02 step 2. l2 forward digits .14 .12 step 3. l2 backward digits .19 .05 step 4. l2 sentences .24 .04 between languages (korean-english) l2 fluency step 1. age .15 .15 step 2. l1 forward digits .43** .28** .56 2.55* step 3. l1 backward digits .44* .01 step 4. l1 sentences .44* .00 l2 comprehension step 1. age .02 .02 step 2. l1 forward digits .26 .24* .58 2.34* step 3. l1 backward digits .27 .01 step 4. l1 sentences .28 .01 discussion the aim of this study was to investigate the extent to which phonological and verbal memory capacity, as indicated by forward and backward digit spans and sentence immediate recall tasks, affected reading outcomes in english and korean by english-korean and korean-english bilinguals. the general pattern of l1 and l2 performance by the participants in the two international sites was consistent with the findings of previous research, indicating that the students demonstrated a greater strength in l1 than l2 (pae, sevcik, & morris, 2004, 2010). the pattern of the bilinguals’ performance on the given measures was different across the two sites. the difference in reading outcomes might result from the l2 learning context; one with an esl context and the other with an efl context. international electronic journal of elementary education 60 obviously, the efl students in korea, in which the korean language is ubiquitously spoken, have limited l2 exposure, input, and use, compared to their esl counterparts in the u.s. whose dual languages are spoken on a daily basis. the english-speaking children in the u.s. appeared to perform better in phonological and verbal memory tasks. since phonological and verbal memory is closely linked to the phonological representations of the language, the richer phonological component of english (due to the number of legal sequences in speech sounds) than korean might have influenced the english-speaking children’s verbal memory performance. as stated earlier, the number of syllables in english and korean is drastically different (2,000 vs. 8,000). this finding suggests that the availability of a wide range of phonological information may facilitate the maintenance and manipulation of input information and eventually help the production of the stored phonological information. however, further research is warranted for the explanation of the role of phonological and verbal memory in reading across different languages. the partial correlations, controlling for age, showed significant correlations between l1 and l2. this result is consistent with the notion of cross-language interdependence (cummins, 1994). there were significant correlations between phonological memory and verbal working memory and between these skills in l1 and l2, suggesting that the two indicators reflect similar constructs. interestingly, the sentence verbal working memory demonstrated low correlations with other variables for the english-speaking children in the u.s. these low correlations indicate that the sentence repetition task measured the same qualities as other variables to some extent, but the overlapping is not conspicuous, suggesting that the sentence repetition task might measure a unique element of verbal memory skills. no significant correlations were found for the korean-speaking participants residing in korea in between-language reading fluency and comprehension. this can be explained as the comparatively limited proficiency of l2 fluency and comprehension skills, compared to the acquired optimal level of l1 skills. the magnitude of the correlation coefficients among the variables was larger in the english l1 group in the u.s. one possible explanation for the weaker correlations in the korean l1 children may be the fact that effects stemming from digit and sentence recall can mask the reading outcomes of l1 and l2 due to fewer demands imposed on phonological coding resulting from the shallow orthographic nature of korean (pae, 2011). as gupta (1996) notes, phonological mapping appears to be related to the interaction between verbal memory and reading-related activities. the individual contribution of phonological and verbal memory to reading fluency and comprehension for the english-speaking children (research question 1) for the english-speaking children who were learning korean as l2 in the u.s., phonological memory capacity, measured using the forward digit span task, played a salient role in english and korean within-language reading fluency and comprehension. it also played a significant role in the prediction of cross-language reading fluency and comprehension. digital verbal working memory, as indicated by the backward digit span task, played a significant role in english within-language reading fluency and comprehension as well as cross-language reading fluency and comprehension. the pattern of findings can be explained in four ways. first, the weak relationship between verbal memory and reading comprehension in korean can be attributable to the shallow orthography in which the grapheme-phoneme correspondence is transparent and regular. since learners can easily decode words without problems, reading korean may not necessitate a significant memory load for fluent reading. second, reading english may be more challenging because english has a wider phonological repertoire than korean. the role of verbal working memory / pae & sevcik 61 specifically, the orthographically deep characteristic affects english reading at the phonemic level, as a single grapheme can be pronounced differently according to neighboring graphemes. third, english words have richer semantic representations than korean, and are multidimensional in that many words carry multiple meanings. for example, english words have many homographs (e.g., bat, a baseball vs. bat, an animal) and homophones (e.g., hi vs. high; whole vs. hole, and heteronyms (e.g., the wind is too strong vs. wind the toy, and it will move) (pae, greenberg, & williams, 2011). hence, the ability to activate task-relevant information and to inhibit incongruent information with the given syntactic and semantic coherence may be directly related to the ability to recall orally presented stimuli. this capability may also aid the child to build extended meanings on the basis of the root word and strengthen his/her word repertoire, because word knowledge can be broadened through a facilitation of effective verbal memory capability and a core phonological and semantic processing mechanism (gupta, 1996). lastly, the relationship between crosslanguage verbal working memory and reading seems to be more challenging than that in l1 and l2 within languages. specifically, both l1 phonological and verbal working memory played a significant role in l2 reading fluency and comprehension for the english-speaking children. since l2 has weaker connections to the mental lexicon and storage (kroll & stewart, 1994), higher cognitive loads and demands may be required for l2 than l1. the individual contribution of phonological and verbal memory to reading fluency and comprehension for the korean-speaking children (research question 2) the results of hierarchical regression analyses showed different predictive patterns across sites. in general, the capability of recalling the immediate serial numbers dominantly served as a significant predictor for the english-speaking children. however, the magnitude of association between verbal memory capacity and reading fluency and comprehension diminished for the korean-speaking counterparts. in korean within-language reading, the predictor variables did not account for a significant variance in reading fluency and comprehension. for english within-language reading, phonological working memory explained only english reading fluency. the ability to form phonological representations of serially presented number stimuli seems to be important to read fluently in english. this may be because skills in phonological coding and quick retrieval facilitate word reading (gathercole & baddeley, 1989). in a similar vein to the finding with the english-speaking children, cross-language reading seems to be more challenging for the korean-speaking children. the l1 phonological working memory skills played a significant part in both l2 reading fluency and comprehension. this result indicates that reading fluency and comprehension development in l2 english is contingent upon children’s ability to recall and efficiently retrieve phonological input. since the representations of words stored in an individuals’ memory are multifaceted, the ability to make use of the connections between phonological information and the given sentence may vary across learners. reading english and korean variability in the ability to retain and manipulate serially presented digits seems to reflect the likelihood of success in reading fluency and comprehension because reading requires interrelated process skills, such as accessing stored information, selecting a relevant meaning on the basis of contextual information, and evaluating the appropriateness of the chosen meaning. since the pronounceability of a new word is vital in word learning, phonological working memory seems to play a dominant role in l2 reading skills for korean native children. international electronic journal of elementary education 62 one possible explanation about the different dimension across the learner groups may be the linguistic features of the two languages that the learner groups speak on a daily basis. it is possible that the demand of phonological coding depends on the extent to which english requires in the process of grapheme-phoneme mapping. attempts have been made in the literature to examine the features of lexical structures influencing visual processing or phonological mediation (coltheart, laxon, keating, & pool, 1986; rastle & coltheart, 1999). a shallow orthography, such as korean, involves automatic and direct access to stored orthographic and phonological representations, while a deep orthography, such as english, goes through a phonologically mediated procedure (stone & van orden, 1993). as such, the results of this study can be aligned with the explanation of direct access or phonologically mediated access to the mental storage in that reading english involves stronger phonological mediation due to the nature of a deep orthography than reading korean (see coltheart, laxon, keating, & pool, 1986; rastle & coltheart, 1999 for details). besides, verbal memory permits learners to store input sounds in a manner that allows for easy storage and retrieval from long-term memory depending on the linguistic characteristics in which children learn. to summarize, developing l2 fluency and comprehension is a long-term process and a multifaceted construct, which involves phonological coding, encoding, appropriate verbal working memory processing, and semantic decoding. due to the interplay of the variables, the function of verbal working memory and reading fluency and comprehension cannot be disregarded in l1 and l2 reading. limitations and future directions the present study was unique with respect to a contrast between a roman alphabetic language and the alphasyllabic language as well as an employment of two learning contexts (i.e., esl and efl contexts), while investigating the role of verbal memory in reading fluency and comprehension in l1 and l2. however, the following limitations are worthwhile to mention. first, the small sample size limited the statistical power of the inferential analyses. second, the lack of standardized instruments across sites deter the generalization of this study's findings. it is necessary to have culturally and linguistically appropriate instruments in cross-language studies. third, cultural variations in the two sites were not taken into account in the analyses and interpretation. fourth, the potentially confounding effects of phonological complexity in english and korean on the task were not examined. as the phonological and phonotactic structure of words is language-specific, the articulatory difficulty at the boundaries of the number of words pronounced in sequence needs to be taken into account. the phonological complexity of the numerals might have influenced the children’s performance on the forward and backward digit span tasks. lastly, there are still unanswered questions regarding the locus, nature, causality, and function of verbal memory in the development of l1 and l2 fluency and comprehension. a more systematic large-scale study would allow for an investigation to answer remaining questions. future studies examining the following are recommended. first, studies with a larger sample size will provide higher statistical power that will validate the findings of this study. second, cross-cultural measurement development is needed to address complexities of two cultures, including cultural, contextual, linguistic differences, and to have valid psychometric properties to achieve measurement equivalence across the two groups. third, since this study is a cross-sectional study, it cannot explain a developmental trajectory of the bilingual children in the two sites. a longitudinal study would offer an examination of children's dual-language learning and developmental trajectories in a systematic way. the role of verbal working memory / pae & sevcik 63 . . . acknowledgement partial support for this study was provided to hye pae by the university research council of the university of cincinnati. we thank the families of our participants for their cooperation and support. we also thank the anonymous reviewers for the valuable comments on the earlier version of this paper. hye k. pae, ph.d., is assistant professor of literacy and second language studies at the university of cincinnati. her research interests include psycholinguistics, second language acquisition, and assessment challenges across cultures. rose a. sevcik, ph.d., is professor of psychology at georgia state university, atlanta. her research has focused on the language and reading acquisition of children and youth at risk for or with learning disabilities. references adams, a. m., & gathercole, s. e. (1996). phonological working memory and spoken language development in young children. the quarterly journal of experimental psychology, 49a, 216233. abu-rabia, s., share, d., & mansour, m. s. (2003). word recognition and basic cognitive processes among reading-disabled and normal readers in arabic. reading and writing: an interdisciplinary journal, 16, 423-442. baddeley, a. d. (1986). working memory. new york: oxford university press. baddley, a. d., & hitch, g. (1974). working memory. in g. a. bower (ed.). the psychology of learning and motivation (vo. 8, pp. 47-89). new york: academic. budd, d., whitney, p., & turley, k. j. (1995). individual differences in working memory strategies for reading expository text. memory & cognition, 23, 735-748. coltheart, v., laxon, v. j., keating, g. c., & pool, m. m. (1986). direct access and phonological encoding processes in children’s reading: effects of word characteristics. british journal of educational psychology, 56, 255-270. colom, r., flores-mendoza, c., quiroga, m. a., & privado, j. (2005). working memory and general intelligence: the role of short-term storage. personality & individual differences, 39, 1005-1014. cummins, j. (1994). the acquisition of english as a second language. in r. pritchard & k. spangenberg-urbschat (eds.) kids come in all languages. (pp. 36-62). newark, de: international reading association. daneman, m. & carpenter, p. a. (1980). individual differences in working memory and reading. journal of verbal learning & verbal behavior, 19, 450-466. daneman, m., & carpenter, p. a. (1983). individual differences in integrating information between and within sentences. journal of experimental psychology: learning, memory and cognition, 9, 561583. daneman, m. & merikle, p. m. (1996). working memory and language comprehension: a meta-analysis. psychonomic bulletin & review, 3, 422-433. gathercole, s. e. (1998). the development of memory. journal of child psychology and psychiatry, 39, 327. gathercole, s. e., & baddeley, a. d. (1989). evaluation of the role of phonological stm in the development of vocabulary in children: a longitudinal study. journal of memory and language, 28, 200-213. gathercole, s. e., & baddeley, a. d. (1993). working memory and language. hillsdale, nj: lawrence erlbaum. international electronic journal of elementary education 64 gernsbacher, m. a. (1990). less skilled readers have less efficient suppression mechanisms. psychological science, 4, 294-298. gottardo, a., yan, b., siegel, l. s., & wade-woolley, l. (2001). factors related to english reading performance in children with chinese as a first language: more evidence of cross-language transfer of phonological processing. journal of educational psychology, 93 (3), 530-542. gupta, p. (1996). word learning and verbal short-term memory: a computational account. proceedings of the 18th annual conference of the cognitive science society, 189-194. gupta, p., macwhinney, b., feldman, h. m., and sacco, k. (2003). phonological memory and vocabulary learning in children with focal lesions. brain and language, 87, 241-252. henry, l. & maclean, m. (2003). relationships between working memory, expressive vocabulary and arithmetical reasoning in children with and without intellectual disabilities. educational and child psychology, 20(3), 51-63. jeffries, s., & everatt, j. (2004). working memory: its role in dyslexia and other specific learning difficulties. dyslexia: an international journal of research and practice, 10, 196-214. johnston, r. s., rugg, m. d., & scott, t. (1987). phonological similarity effects, memory span and developmental reading disorders: the nature of the relationship. the british journal of psychology, 78, 205-211. jorm, a. f., share, d. l., maclean, r., & matthews, r. (1984). phonological confusability in short-term memory for sentences as a predictor of reading ability. the british journal of psychology, 75, 393-400. just, m. a., & carpernter, p. a. (1992). a capacity theory of comprehension: individual differences in working memory. psychological review, 99, 122-49. kane, m. j., hambrick, d. z., tuholski, s. w., wilhelm, o., payne, t. w., & engle, r. w. (2004). the generality of working memory capacity: a latent variable approach to verbal and visuospatial memory span and reasoning. journal of experimental psychology: general, 133, 189-217. katz, l., & frost, r. (1992). the reading process is different for different orthographies: the orthographic depth hypothesis. in r. frost & l. katz (eds.), orthography, phonology, morphology, and meaning (pp. 67-84). amsterdam: north holland. katz, r. b., shankweiler, d., & liberman, i. y. (1981). memory for item order and phonetic recoding in the beginning reader. journal of experimental child psychology, 32, 474-484. keenan, j. m., & betjemann, r. s. (2006). comprehending the gray oral reading test without reading it: why comprehension tests should not include passage-independent items. scientific studies of reading, 10 (4), 363-380. korkman, m., kirk, u., & kemp, s. (1998). a developmental neuropsychological assessment. san antonio, tx: the psychological corporation. kroll, j., & stewart, e. (1994). category interference in translation and picture naming: evidence for asymmetric connections between bilingual memory representations. journal of memory & language, 33, 149-174. leong, c. k., tse, s. k., loh, k. y., & hau, k. t. (2008). text comprehension in chinese children: relative contribution of verbal working memory, pseudoword reading, rapid automatized naming, and onset-rime phonological segmentation. journal of educational psychology, 100, 135-149. mcdougall, s., hulme, c., ellis, a., & monk, a. (1994). the role of short-term memory and phonological skills in reading. journal of experimental child psychology, 58, 112-133. miller, p., & kupfermann, a. (2009). the role of visual and phonological representations in the processing of written words by readers with diagnosed dyslexia: evidence from a working memory task. annals of dyslexia, 59, 12-33. miyake, a., & friedman, n.p. (1998). individual differences in second language proficiency: working memory as language aptitude. in a.f. healy & l.e. bourne (eds.), foreign language learning: psycholinguistic studies on training and retention (pp. 339-364). mahwah, nj: lawrence erlbaum associates. o’brien, i., segalowitz, n., collentine, j., freed, b. (2006). phonological memory and lexical, narrative, and grammatical skills in second language oral production by adult learners. applied psycholinguistics, 27, 377-402. pae, h. k. (2011). is korean a syllabic alphabet or an alphabetic syllabary? writing systems research, doi: 10.1093/wsr/wsr002. the role of verbal working memory / pae & sevcik 65 pae, h. k., sevcik, r., & morris, r. d. (2004) cross-language links between english and korean in the second-language reading acquisition, journal of multilingual communication disorders, 2 (1), 68-79. pae, h. k., sevcik, r. a., & morris, r. d. (2010). cross-language correlates in phonological process and naming speed: evidence from deep and shallow orthographies. the journal of research in reading, 33(4), 335-436. pae, h. k., greenberg, d., & williams, r. s. (2011). an analysis of differential error patterns on the peabody picture vocabulary test-iii in children and struggling african-american adults readers. reading and writing: an interdisciplinary journal. doi: 10.1007/s11145-011-9315-x. payne, t. w., kalibatseva, z., & jungers, m. k. (2009). does domain experience compensate for working memory capacity in second language reading comprehension? learning and individual differences, 19, 119-123. perfetti, c. a. and liu, y. (2005). orthography to phonology and meaning: comparisons across and within writing systems. reading and writing: an interdisciplinary journal, 18, 193-210. rastle, k. and coltheart, m. (1999). serial and strategic effects in reading aloud. journal of experimental psychology: human perception & performance, 25, 482-503. sohn, h-m. (1999). the korean language. new york: cambridge university press. stanovich, k. e., siegel, l. s., & gottardo, a. (1997). converging evidence for phonological and surface subtypes of reading disability. journal of educational psychology, 89 (1), 114-127. stone, g., & van orden, g. c. (1993). strategic control of processing in word recognition. journal of experimental psychology: human perception and performance, 19, 744-774. swanson, h. l., & berninger, v. (1995). the role of working memory in skilled and less skilled readers’ comprehension. intelligence, 21, 83-108. swanson, h. l., saez, l., & gerber, m. (2006). growth in literacy and cognition in bilingual children at risk or not at risk for reading disabilities. journal of educational psychology, 98, 247-264. swanson, h. l., zheng, x., & jerman, o. (2009). working memory, short-term memory, and reading disabilities: a selective meta-analysis of the literature. journal of learning disabilities, 43 (3), 260-287. taylor, i. (1980). the korean writing system: an alphabet? a syllabary? a logography? in p. a. kolers, m. e. wrostad & h. bouma (eds.). processing of visible language 2. new york: plenum press. taylor, i. (1997). psycholinguistic reasons for keeping chinese characters in korean and japanese. in h. c. chen (ed.), cognitive processing of chinese and related asian languages (pp. 299-319). hong kong: chinese university press. unsworth, n., & engle, r. w. (2007). the nature of individual differences in working memory capacity: active maintenance in primary memory and controlled search from secondary memory. psychological review, 114, 104-132. wechsler, d. (1991). the wechsler intelligence scale for children-iii. new york: psychological corp. wiederhold, j. l., & bryant, b. r. (2001). gray oral reading tests-4. austin, tx: pro-ed. windfuhr, k., & snowling, m. j. (2001). the relationship between paired associate learning and phonological skills in normally developing children. journal of experimental child psychology, 80, 160-173. ziegler, j. c., & goswami, u. (2005). reading acquisition, developmental dyslexia, and skilled reading across languages: a psycholinguistic grain size theory. psychological bulletin, 131, 3-29. microsoft word 3_kunzman_3_1 edits international electronic journal of elementary education vol. 3, issue 1, october, 2010. issn:1307-9298 copyright © iejee www.iejee.com homeschooling and religious fundamentalism robert kunzman ∗∗∗∗ indiana university, united states abstract this article considers the relationship between homeschooling and religious fundamentalism by focusing on their intersection in the philosophies and practices of conservative christian homeschoolers in the united states. homeschooling provides an ideal educational setting to support several core fundamentalist principles: resistance to contemporary culture; suspicion of institutional authority and professional expertise; parental control and centrality of the family; and interweaving of faith and academics. it is important to recognize, however, that fundamentalism exists on a continuum; conservative religious homeschoolers resist liberal democratic values to varying degrees, and efforts to foster dialogue and accommodation with religious homeschoolers can ultimately help strengthen the broader civic fabric. keywords: home education, homeschooling, christian fundamentalism. introduction seventy years before the rise of modern homeschooling, british poet and critic edmund gosse published a memoir titled father and son, in which he reflected on the stifling isolation of his childhood, educated at home by his religiously conservative plymouth brethren father. gosse described himself as “a bird fluttering in the net-work of my father’s will, and incapable of the smallest independent action” (1907, p. 232). images of religiously-inspired oppression such as this have dominated public perceptions of homeschooling until only recently, when it became apparent that homeschoolers are an ∗ correspondence: robert kunzman, associate professor, indiana university school of education, 201 n. rose ave. #3288, bloomington, in 47405-1006. phone: (812) 856-8122. e-mail: rkunzman@indiana.edu international electronic journal of elementary education vol.3, issue 1, october,2010 18 increasingly diverse group, running the gamut of pedagogical philosophy and methods. nevertheless, religious conservatives likely remain the largest subset of homeschoolers, at least in the united states. the relative freedom and flexibility of homeschooling allows parents to craft an educational environment that reflects their values and priorities, and religious conservatives find such an option particularly appealing. this essay explores the relationship between homeschooling and religious fundamentalism, and suggests four ways in which homeschooling provides an educational setting especially conducive to reinforcing core fundamentalist principles. the point of such a comparison, however, is not to imply that homeschooling inherently fosters religious fundamentalism. the structural flexibility of homeschooling, and the space it provides both literally and ideologically, lends itself to countercultural movements of all kinds. for instance, it also supports a socially progressive critique, which is where the modern homeschooling movement gained early inspiration, in the writings of john holt (gaither, 2008). nor do i mean to imply that all religious conservative homeschooling echoes these four themes in full. in fact, as my research suggests, the internal diversity of religious conservative homeschoolers may provide important opportunities for civic conversation across ideological differences (kunzman, 2009). understanding―and appreciating, even while perhaps disagreeing with―the motivations and perspectives of homeschoolers is a vital step in respectful civic engagement in a pluralistic society. the origins and nature of fundamentalism while fundamentalism holds a variety of meanings today, it has a very specific historical origin. the term fundamentalist first emerged from early twentieth century america, when conservative christians published a 12volume series of books entitled the fundamentals, which sought to provide a wide-ranging assertion of orthodox christian belief against liberal protestantism and an increasingly modernist culture. one particular target of the fundamentalists was darwinian evolution and its teaching in public schools. this struggle peaked in 1925, with the scopes “monkey” trial dealing a powerful public relations blow to anti-evolutionists. fundamentalists became marginalized outsiders, retreating to their own local communities and largely beyond the national consciousness (marsden, 1980; smith, 1998). until the 1940s, the terms evangelical and fundamentalist were largely synonymous in american society. disagreements began to arise among conservative christians, however, over what kind of relationship they should have with the broader culture. as nancy ammerman (1991) explains, fundamentalists emphasized separation over engagement and issued a “resounding condemnation of compromise,” asserting that “getting homeschooling and religious fundamentalism / kunzman 19 along was no virtue and that active opposition to liberalism, secularism, and communism was to be pursued” (pp. 14, 4). fundamentalists had become cultural outsiders, but the social upheaval of the 1960s and 1970s sparked the beginnings of a new movement, a growing disenchantment and sense of activism that culminated in the formation of the moral majority in 1979. fundamentalists had made effective use of popular media since the days of the printing press and the advent of radio, and now turned enthusiastically to television as well, both to attract followers and communicate their sociopolitical agenda to the outside world (almond, sivan, & appleby, 1991; ammerman, 1991). conservative christians, as the original fundamentalists, have certainly made their presence felt in american culture, but fundamentalism obviously exists and thrives in many religions today across the globe.1 at its core, religious fundamentalism pushes back against what it perceives as an increasingly secular culture, striving to remain separate from that culture while seeking to restore religion to its rightful, central place in society. intersections of homeschooling and fundamentalism quantitative research on homeschooling offers a patchwork of estimates and small-scale studies. in the united states, data collection is limited by widely varying state regulations; nearly a quarter of states don’t even know how many homeschoolers there are within their borders. because of these uncertainties about total numbers, details about demographics subsets, including religion, are largely guesswork. the best estimates we have about homeschoolers across the united states come from the national center for education statistics, which conducts a survey every four years on american households.2 in the 2007 nces survey, respondents who homeschooled their children were asked about their reasons for doing so, and 83% pointed to providing “religious or moral instruction” (planty et al., 2008, p. 135). certainly not all of these respondents were christians, much less fundamentalist in orientation, but it helps lend credence to the generally accepted notion that conservative christians comprise the largest subset of homeschoolers in the united states. what is beyond dispute, however, is the disproportionate influence that conservative christians have had on public policy and public perceptions of homeschooling. this is due in large part to the activity and influence of the home school legal defense association (hslda), which 1 nor is fundamentalism limited to religious perspectives. as almond, sivan, & appleby (1991) point out, “religion is not the only matrix out of which fundamentalism like movements emerge. race, language, and culture may also serve as the bases of revivalism and militance” (p. 404). 2 given their general reluctance to provide information to government agencies, it seems likely that homeschoolers are underrepresented in this study (see, for example, kaseman & kaseman, 1991). international electronic journal of elementary education vol.3, issue 1, october,2010 20 identifies itself as a christian organization and whose 85,000 members (and a quarter of a million children) form the largest homeschool advocacy group in the world. hslda has been particularly effective at monitoring the regulatory climate surrounding homeschooling (both in the united states and abroad) and advocating for reduced requirements and state oversight. while still dwarfed by conservative christians in absolute numbers, muslim-americans claim to be among the fastest growing subsets of homeschoolers in the united states. other religious homeschoolers include mormons, jews, catholics, and seventh day adventists, to name just a few. most of the research on religious homeschooling, including my own, doesn’t attempt to distinguish fundamentalists from the broader category of religious conservatives, in part because such distinctions are imprecise, even within single christian denominations. it seems more useful to conceive of fundamentalism as existing on a continuum of sorts, with varying levels of rejection, resistance, compromise, and accommodation toward the broader culture, depending upon the particular issue. with this “family resemblance” approach to description in mind, i suggest four ways in which the uniqueness of the homeschooling educational context can offer support for core commitments of religious fundamentalism. some of these intersections, such as parental authority over education, are core commitments of homeschooling as well. other aspects, such as the interweaving of academics and religion, are simply options that homeschooling’s flexibility allows. resistance to contemporary culture religious fundamentalists, write marty and appleby (1991), “no longer perceive themselves as reeling under the corrosive effects of secular life. on the contrary, they perceive themselves as fighting back, and doing so rather successfully” (p. ix). homeschooling their children is a potent way that fundamentalists can resist secular culture; the philosophical and pedagogical flexibility of homeschooling provides the opportunity to instill values and beliefs while reducing the power and presence of the broader culture around them. in his study of american evangelical culture, sociologist christian smith (1998) suggests that, rather than being weakened by modern pluralistic society, religious conservatives actually gain strength through their ongoing conflicts with this wide diversity of ethical values. homeschooling not only provides a private realm for parents to instill their vision of the good life, but the very act of homeschooling serves as an assertion of a conservative religious identity (liao, 2006). one element of cultural resistance common to religious fundamentalists is their endorsement of stratified gender roles. on the surface, at least, conservative christian homeschooling promotes traditional gender roles as well (kunzman, 2009; stevens, 2001), prompting criticism and concern from many observers (joyce, 2009; nemer, 2004; yuracko, homeschooling and religious fundamentalism / kunzman 21 2008). but other scholars have suggested that, in practice, women as homeschool mothers wield significant influence. while these mothers would certainly point to the tremendous influence they have on the growth of their children as their most important contribution, women have also been prime movers in political organizing and advocacy in regards to homeschooling (gaither, 2009; stevens, 2001). the issue of homeschoolers’ political and civic engagement remains a central concern of some observers (balmer, 2007; reich, 2002; lubienski, 2000). contrary to the typical view of homeschoolers as isolated from the public square, however, research suggests that religious school and homeschool families (at least in the united states) are consistently more involved in civic activities than families with children in public schools (smith & sikkink, 1999).3 significantly, many citizens learn the skills and practices of engaged citizenship through their religious institutions and affiliations (weithman, 2006). but the heart of the argument isn’t simply about civic participation, but rather what kind of participation. i spent several years following a civic education program run by hslda, called generation joshua. with its mission to “take back america for god,” genj offers a rich array of educational opportunities for christian adolescent youth, ranging from online discussions and formal curricula to spearheading voter registration drives and direct campaigning for politically conservative candidates. based on my ten years of teaching public high school english and social studies, it was clear that these students were far more informed and involved in the civic realm than the average public school student. but the adversarial nature of their approach, and the way it seemed to inhibit respectful engagement with opposing perspectives, raises challenging questions about what it means to educate for civic virtue―and who gets to decide what qualifies as virtuous (kunzman, 2009). suspicion of institutional authority and professional expertise as part of their resistance to the broader surrounding culture, religious fundamentalists are especially wary of government institutions and the notion of professional expertise. conservative christians in early twentieth century america saw themselves as cultural insiders and the state as a means by which to bring transgressors against cultural norms back into line. but when these norms began to shift in the 1960s, religious conservatives began to see the government, its courts, and its schools as the enemy, one committed to the evils of secular humanism (ammerman, 1991; gaither, 2008).4 3 this held true even controlling for differences in education, income, age, race, family structure, region, and the number of hours per week that parents work. 4 homeschoolers’ view toward government appears to be more varied in europe. beck (2006), for example, suggests the religious homeschoolers in norway generally do not share this inherent suspicion of government and its institutions. certainly the general populace international electronic journal of elementary education vol.3, issue 1, october,2010 22 this negative attitude toward authority and expertise resonates with most homeschoolers as well, liberal or conservative (apple, 2005; moss, 1995). public schools are frequently portrayed as sites of indoctrination, somewhere between an inadvertent fostering of unreflective consumerism and a full-blown satanic conspiracy. many homeschoolers use the term “government schools” rather than “public schools,” to emphasize control by the state instead of we-the-public. another form of fundamentalist and homeschooler resistance to authority is directed toward “experts,” particularly those who work with children: teachers, administrators, health care providers, and social workers especially. this isn’t necessarily an outright rejection of expertise; most of the homeschoolers i speak with, for example, recognize the daunting task that institutional schoolteachers face in classrooms of twenty or more students. but they are unwilling to concede that someone else would be better qualified to teach their own children. “i’m not a teacher in the sense of being prepared to teach large groups of strangers,” one homeschool mother told me. “but there is nobody who teach my kids better than i can” (kunzman, 2009, p. 210). fundamentalist suspicion of centralized human authority has a theological side to it as well, with fundamentalists often claiming that “the only biblical form of church organization is the local body” (ammerman, 1991, p. 30). the growth of “house churches” (small groups of families meeting in someone’s home) among both fundamentalists and homeschoolers serves as a parallel to their commitment to bringing formal schooling back to the home as well (gaither, 2008; mcdannell, 1995). parental control and centrality of the family while both homeschoolers and fundamentalists cast a suspicious eye toward experts and their institutions, the ultimate site of resistance to contemporary culture is the family. at the heart of the christian fundamentalist surge into american politics in the late 1970s was the idea that the “traditional” family, the most basic unit of society, must be protected against the onslaught of modern culture. this emphasis resonates with the central conviction of most homeschoolers as well―that parents should have authority over the education of their children, with little or no state regulation (carper & ray, 2002; kunzman, 2009; martinez, 2009; moss, 1995; van galen, 1987). interestingly, this core homeschooling conviction appears to be one of the in europe holds less of a libertarian, anti-state perspective than is evident in the united states. in the swedish context, for instance, villalba (2009) describes the widely affirmed concept of insyn, whereby the state ensures that all citizens receive a roughly similar education. the recent case of a german homeschool family who sought and received political asylum in the united states provides a comparative example; the u.s. immigration judge who granted their request reportedly described the german prohibition on homeschooling as “repellent to everything we believe as americans” (robertson, p. a12). homeschooling and religious fundamentalism / kunzman 23 few instances―in american politics, at least―where the far left and far right find common ground. it’s an uncomfortable alliance, to be sure, with the broader political activity of hslda frequently infuriating liberal homeschoolers, but they are willing to join forces when they perceive the state is attempting to usurp their educational authority as parents.5 for conservative christian homeschoolers, the education of their children is a god-given right and responsibility, and one they can delegate only at great moral and spiritual peril. homeschooling is a shaping not only of intellect but―even more crucially―of character. this means more than just moral choices of right and wrong; character is developed through the inculcation of an overarching christian worldview that guides those moral choices. while all good parents strive to protect their children in a variety of ways, conservative christians see much of contemporary culture as an assault on the values they seek to impart to their children (apple, 2005; mcdannell, 1995; stevens, 2001). mothers, in particular, strive to be deeply engaged in the day-to-day lives of their children (lois, 2009). these parents share a fierce determination to instill christian character in their children, a process that entails protecting them from the corrupting influences of broader society. as one homeschool mother remarked, “why would you want to send your child away for the majority of the day and let someone else’s ideas and personality be placed in your child every day? i’m her parent. god gave her to me to form and to raise, so i feel that’s my responsibility” (kunzman, 2009, p.180). the family serves as the defensive bulwark and sanctuary wherein children are prepared for eventual engagement with the world (valle, 1998; van galen, 1987). interweaving of academic and religious one obvious way in which religious parents use homeschooling to shape their children’s character is by using curricula infused with their faith convictions. for religious conservatives, the intellectual life only finds meaning when it aligns with religious truth. in the eyes of fundamentalists, the sanctity of sacred scriptures trumps all human sources of knowledge and understanding. for conservative christian homeschoolers, this means that “if it doesn’t line up with the word, throw it out” (cizek, 1994; kunzman, 2009; mcdannell 1995; valle, 1998). the curricula that are generally most popular with conservative christian homeschoolers―bob jones, a beka, sonlight, alpha omega, classical christian, to name a few―seek to integrate faith and intellect, whether by detailing the congruence between scientific research and 5 ironically, hslda’s focus on protecting parents’ rights also generates perhaps the greatest friction between liberal and conservative homeschoolers. hslda believes that anything threatening to the “traditional family”—gay marriage, for instance—ultimately endangers the parents’ traditional right to control the education of their children, and thus actively oppose legislation legalizing gay marriage. international electronic journal of elementary education vol.3, issue 1, october,2010 24 religious doctrine, using literature to illustrate scriptural truths, or simply providing illustrations and examples with religious content. as a beka notes on its website, “the most original source is always the word of god, the only foundation for true scholarship in any area of human endeavor.” one of its history textbooks affirms, “students will learn to recognize the hand of god in history and to appreciate the influence of christianity in government, economics, and society.” another publishing company’s science textbook includes “evolution stumpers,” which question evidence such as the fossil record and examples of random mutation. for most of these conservative christian homeschooling texts, it is not a simple case of adding or subtracting information from what a public school student might encounter. instead, it is a reframing of the entire subject; given this radical departure from mainstream publishers, it is perhaps not surprising that estimates put the homeschool curriculum market as approaching one billion dollars a year in sales in the united states alone. at the heart of religious fundamentalists’ desire to select and direct curricular experiences is the conviction that absolute moral and religious truths do exist, and are accessible to the faithful. this is not to say, however, that fundamentalists are of one accord about the content of those truths. the diversity of belief even among religious fundamentalists of the same tradition, and the resulting arguments and church splits, populate the historical record (stevens, 2001). the more stridently beliefs are held, it seems, the more likely that internal divisions will arise as both sides are certain they are correct.6 sustaining the civic conversation homeschooling offers religious fundamentalists a potent means to resist the encroachment of secular culture on their families. the philosophical and pedagogical flexibility of homeschooling provides an educational environment especially suitable for parents to cultivate a particular set of values and commitments in their children. in writing about the american fundamentalist context, ammerman (1991) underscores what is at stake: “fundamentalists are convinced that america must have a pro-religion culture, one in which they have a stronger voice in shaping the values and images that guide society. theirs is an ideological battle for control of the way america will view its past and its future” (p. 47). education is a primary site of this struggle to determine who passes on what messages about what should matter to us and why. 6 in her thoughtful exploration of the quiverfull movement (populated by christian fundamentalist homeschoolers), kathryn joyce (2009) expresses alarm at reconstructionist theology, which advocates a return to old testament law, including stoning to death of homosexuals and habitually disobedient children. not surprisingly, she is not alone in her dismay. but reconstructionists have already suffered numerous splits and dissolutions, and there is little reason to think this pattern won’t continue. homeschooling and religious fundamentalism / kunzman 25 to many of those outside such religious traditions, however, such a dynamic appears problematic, even threatening. on one hand, if the ammerman quote were instead about liberal progressives striving for a stronger voice in shaping the values and images that guide society, it might be seen as just part of the agonistic democratic political process. on the other hand, when citizens—religious fundamentalists or anyone else— approach the political process with a self-righteous zeal that abides neither compromise nor accommodation, then it’s easy to see why others view them as a threat to liberal democracy. when fundamentalist resistance includes an unwillingness to distinguish between the infallible certainty of their religious world and the ethical pluralism of the public square, when their ultimate goal is to transform that public square into a mirror image of their belief system, then conversation may be next to impossible. but this hardly describes all religious conservatives, or even all fundamentalists. the label of religious fundamentalist—like most labels— often obscures more than it reveals. fundamentalism exists on a continuum, and conservative religious homeschoolers resist liberal democratic values to varying degrees. with this in mind, then, liberal democratic states should avoid viewing conservative religious homeschooling as simply an enemy to be resisted or silenced. seeking possibilities for compromise and accommodation, providing space and opportunity for religious conservatives to enter into dialogue without demanding they leave behind their deeply held identities—these are the difficult steps that may help strengthen the broader civic fabric. furthermore, when we raise concerns about the lack of ethical pluralism in the fundamentalist homeschooling context, it’s worth considering how much ethical diversity our public schools provide. do they encourage the questioning of received wisdom, interrogation of popular culture, and room for diversity of beliefs―even those of religious conservatives? granted, there are limits to what ideas and expression are tolerable in a liberal democracy (blacker, 1998), but if religious conservatives perceive public schools as so inhospitable toward their values, priorities, and beliefs as to push them into homeschooling, we have missed a key opportunity for dialogue across difference. even encouraging homeschool students to enroll part time in selected classes can get them involved in the life of the school and broader community in important ways. some fundamentalist homeschoolers charge that such an approach is an attempt to defuse their resistance and liberalize their children. i would see it instead as an attempt to include religious conservatives in a civic conversation about how to live together despite our ethical differences.7 7 much of this concluding discussion has focused on what is best for liberal democracy as a whole. but it’s also worth considering that for some children of religious fundamentalists, having the option to homeschool might even be better than the alternative. in 2008, the international electronic journal of elementary education vol.3, issue 1, october,2010 26 homeschooling is typically conceived of as a distinct educational practice that takes place in the home or private associations. but as we move into a new decade, the increasing hybridization of schooling and the blurring of lines between public and private are becoming prominent themes (gaither, 2009). what exactly is public―beyond the funding source―when a student spends her days at home in front of the computer, but enrolled in a public virtual high school? the role of technology and distance education will almost surely re-shape the nature of schooling, and indeed the process is already underway. one possible civic benefit, for homeschoolers and everyone else, may be the opportunity for increased communication between members of society with diverse perspectives and beliefs. but distance learning and online communication can also foster greater connection between like-minded citizens as well, for better or worse. the example of the generation joshua described earlier illustrates this. as one participant remarked, “i’ve found a place where people agree with my political views, and they are willing to stand up for them! it’s great!” but there’s an inevitable tension between cultivating a powerful group identity while still preserving room for ideological diversity, or at least the room to question dogma and consider alternative perspectives. on the whole, generation joshua seems a good example of what political theorist cass sunstein (2007) describes as ‘ideological amplification”: like-minded group members pushing one another toward more extreme versions of their already-held beliefs. stereotypes of religious homeschooling often involve parents creating brainwashed automatons, unable to think for themselves and either sequestered from society or determined to impose their worldview on others. but consider what it means to homeschool, whether religiously motivated or not, in a society where at least 95 percent of the population does otherwise (and far more in europe). by virtue of their freedom to shape their child’s education in almost any way they choose, homeschoolers are pushed to grapple with several vital and profound questions: what are the central purposes of education? what kind of person do i want my child to become? how can i make their learning experience the best it can be? one might argue that the rest of us, and the schools we support and send our children to, neglect such fundamental questions far too much. a persistent conversation about the values that inform such schooling, and an ongoing consideration of how it can invite and involve as much of the public as possible, remains a vital civic task. • • • new york times profiled the growing muslim homeschooler population in a rural california town. certainly, the girls of these south asian immigrant families experience significant social isolation while being homeschooled. but before homeschooling became a legitimate option for them, the girls were instead shipped back to their south asian villages to be married when they reached adolescence (macfarquhar). homeschooling and religious fundamentalism / kunzman 27 robert kunzman is an associate professor in the indiana university school of education. his research focuses on the intersection of religion, citizenship, and education. his most recent book is write these laws on your children: inside the world of conservative christian homeschooling, and he maintains a website devoted to homeschooling research and scholarship: www.indiana.edu/~homeeduc. references almond, g. a., sivan, e., & appleby, r. s. (1991). fundamentalism: genus and species. in m. e. marty & r. s. appleby (eds.), fundamentalisms comprehended (pp. 399424). chicago: university of chicago press. ammerman, n. t. (1991). north american protestant fundamentalism. in m. e. marty & r. s. appleby (eds.), fundamentalisms observed (pp. 1-65). chicago: university of chicago press. apple, m. (2005). away with all the teachers: the cultural politics of home schooling. in b. s. cooper (ed.), homeschooling in full view: a reader (pp. 75-95). greenwich, ct: information age publishing. balmer, r. (2007). homeschooling endangers democracy. in h. williams (ed.), homeschooling (pp. 16-24). detroit, mi: greenhaven press. beck, c. w. (2006). home education: motives, numbers, and social integration: a mirror of educational politics? norsk pedagogisk tidsskrift, 3, 191-204. blacker, d. (1998). fanaticism and schooling in the democratic state. american journal of education, 106, 241-272. carper, j. c. and ray, b. d. (2002). religion, schooling, and home education: past and present. in e. l. blumhofer (ed.), religion, education, and the american experience (pp. 223-242). tuscaloosa, al: university of alabama press. cizek, g. j. (1994). religious education in home schools: goals/outcomes mismatch? religious education, 89(1), 43-51. gaither, m. (2009). homeschooling in the usa: past, present and future. theory and research in education, 7(3), 331-346. gaither, m. (2008). homeschool: an american history. new york: palgrave macmillan. joyce, k. (2009). quiverfull: inside the christian patriarchy movement. boston: beacon press. kaseman, l., and kaseman, s. (1991). “does homeschooling research help homeschooling?” home education magazine, january-february. kunzman, r. (2009). write these laws on your children: inside the world of conservative christian homeschooling. boston, ma: beacon press. liao, m. s. (2006). keeping home: home schooling and the practice of conservative protestant identity. (doctoral dissertation, vanderbilt university, 2006) umi microform, umi 3292673. lois, j. (2009). emotionally layered accounts: homeschoolers' justifications for maternal deviance. deviant behavior, 30(2), 201-34. lubienski, c. (2000). whither the common good? a critique of home schooling. peabody journal of education, 17(1&2), 167-178. macfarquhar, n. (2008, march 26). many muslims turn to home schooling. the new york times. available at www.nytimes.com/2008/03/26/us/26muslim.html. international electronic journal of elementary education vol.3, issue 1, october,2010 28 marsden, g. m. (1980). fundamentalism and american culture: the shaping of twentiethcentury evangelicalism, 1870-1925. new york: oxford university press. martinez, p. (2009). school is where home is. islamic horizons, 38(1), 46-51. marty, m. e. and appleby, r. s. (1991). introduction. in m. e. marty & r. s. appleby (eds.), fundamentalisms observed (pp. vii-xiii). chicago: university of chicago press. mcdannell, c. (1995). home schooling in contemporary america. in d. chidester & e. t. linenthal (eds.), american sacred space (pp. 187-219). bloomington, in: indiana university press. moss, p. a. (1995). benedictines without monasteries: homeschoolers and the contradictions of community. (doctoral dissertation, cornell university, 1995) umi microform, umi 9509446. nemer, k. m. (2004). schooling alone: homeschoolers, individualism, and the public schools. (doctoral dissertation, university of california, los angeles, 2004) umi microform, umi 3142549. planty, m., hussar, w., snyder, t., kena, g., kewalramani, a., kemp, j., bianco, k., dinkes, r. (2009). the condition of education 2009 (nces 2009-081). national center for education statistics, institute of education sciences, u.s. department of education. washington, dc. reich, r. (2002). the civic perils of homeschooling. educational leadership, 59(7), 56-59. robertson, c. (2010, 28 february). judge grants asylum to german home schoolers. new york times, p. a12. smith, c. (1998). american evangelicalism: embattled and thriving. chicago: university of chicago press. smith, c. and sikkink, d. (1999, april). is private school privatizing? first things, 92, 1620. stevens, m. l. (2001). kingdom of children: culture and controversy in the homeschooling movement. princeton, nj: princeton university press. sunstein, c. r. (2007). ideological amplification. constellations, 14(2), 273-279. valle, j. p. (1998). their choice: how homeschooling makes sense to fundamentally christian parents. (doctoral dissertation, pennsylvania state university, 1998). umi microform, umi 9836783. van galen, j. a. (1987). explaining home education: parents’ accounts of their decisions to teach their own children. the urban review, 19(3), 161-177. villalba, c. m. (2009). home-based education in sweden: local variations in forms of regulation. theory and research in education, 7(3), 277-296. weithman, p. j. (2006). religion and the obligations of citizenship. new york: cambridge university press. yuracko, k. (2008). education off the grid: constitutional constraints on homeschooling. california law review, 96, 123-184. 291 copyright © www.iejee.com issn: 1307-9298 © 2022 published by kura education & publishing. this is an open access article under the cc by ncnd license. (https://creativecommons.org/ licenses/by/4.0/) international electronic journal of elementary education january 2023, volume 15, issue 3, 291-306 supervision strategies for treatment fidelity and job satisfaction in applied behavior analysis services melissa s. saunders* abstract introduction supervising behavior technicians in the implementation of services derived from the science of applied behavior analysis with children diagnosed with autism spectrum disorder (asd) is a common practice for behavior analysts. however, there is limited training available on supervision strategies for those applying this model of services. lack of training can lead to ineffective and inconsistent supervisory practices, resulting in low rates of job satisfaction for supervisees and variable fidelity in treatment implementation. the quality of supervision can improve job satisfaction and treatment fidelity. this study was conducted using a nonconcurrent multiple baseline design across participants to evaluate the effects of an evidence-based approached to supervision on treatment fidelity and job satisfactory for three behavior technicians providing services for a child with asd. each of the three technicians demonstrated improved levels of treatment fidelity and increased job satisfaction across several facets of their job during the intervention and maintenance phase. implications of these findings, limitations of this study, and suggestions for future research are included. oftentimes behavioral treatment derived from applied behavior analysis (aba) is delivered in a tiered model (council of autism service providers [casp], 2020). this tiered model typically consists of a behavior analyst who is licensed (association of professional behavior analysts [apba], 2022) and/or board certified (behavior analyst certification board [bacb], 2022), supervising the direct services of a behavior technician. in this model, the supervising behavior analyst is expected to provide the technician with training on treatment protocols specific to those receiving the services as well as ongoing supervision to ensure treatment recommendations are delivered with fidelity. for individuals receiving aba-based services, a customized treatment plan is developed by the supervising behavior analyst to include skill acquisition as well as a behavior increase and reduction protocol when appropriate. the technicians are expected to implement the detailed protocol as outlined in keywords: supervision, job satisfaction, applied behavior analysis, behavior technician, treatment fidelity received : 17 december 2022 revised : 8 february 2023 accepted : 20 march 2023 doi : 10.26822/iejee.2023.301 * correspondance details: melissa s. saunders, creative interventions, usa. e-mail: melissa@creativeinterventions.net orcid: https://orcid.org/0000-0001-5671-6583 292 january 2023, volume 15, issue 3, 291-306 each treatment plan to assist the individual receiving treatment in achieving their individualized goals. the protocols include, but are not limited to, conducting teaching procedures, recording accurate data on client behaviors, delivering prompting strategies as outlined, and using potent reinforcers effectively (cooper et al., 2019). within the field of aba there are high rates of technician turnover (sundberg, 2016). turnover can inadvertently effect treatment fidelity as well as the progress consumers make (mandell et al., 2013; wine et al., 2020). retaining trained technicians is a priority because of the specialized training and oversight invested by the supervising behavior analyst when working with individuals with autism spectrum disorder (asd), not only for the employer funding the training but also for those receiving the aba-based treatment. job satisfaction and intention to leave have been linked to the quality of supervision received (e.g., training received, communication, recognition received by the supervisor) and level of competency to do the job (kazemi et al., 2015; wilson, 2015). in traditional work environments a supervisor commonly refers to the person whom the employee reports and may complete employment reviews and determine promotions. in aba-based services, the term supervisor is most often used to refer to the supervisor of a trainee (i.e., individual accruing fieldwork experience and seeking certification) or that of a supervisee (i.e., individual implementing behavioral services [bacb, 2020]). for the purposes of this paper, the term clinical supervision/supervisor will be used for the supervision of a supervisee delivering aba-based services for an individual. in the tiered model of services found in aba, the clinical supervisor will often have the most direct and frequent exposure to the technician. there is a tremendous investment required of the clinical supervisor in training behavior technicians on their teams; each program is personalized for the individual receiving treatment, requiring specific training on protocols and ongoing oversight provided to a behavior technician by a board certified behavior analyst (bcba; bacb, 2014; odom et al., 2010; volkmar et al., 2014). having a tool that supports job satisfaction and retention of trained behavior technicians who can implement protocols with fidelity would provide meaningful change for organizations that offer ababased services for individuals with autism. employees who are provided with quality supervision and training are more satisfied with their job (collins et al., 2008; eisenberger et al., 2002; mor barak et al., 2009; parsons et al., 2003; reid et al., 2011). studies show that there is a relationship between the satisfaction level of an employee and employee turnover (kuo et al., 2014; sageer et al., 2012). research has indicated that improved treatment implementation can be achieved by providing specific goals, training to mastery, and giving direct feedback to implementers (digennaro-reed et al., 2011; miles & wilder, 2009; parsons et al., 2012; sarokoff & sturmey, 2004) and that job satisfaction can be improved as competency is built (wilson, 2015). importance of treatment fidelity a substantial challenge to intervention is behavioral drift on the part of the implementor in the application of intervention, resulting in protocols being implemented incorrectly or not to fidelity (allen & warzak, 2000). even when the most effective procedure is being recommended by a clinical supervisor, if that intervention is not being implemented to fidelity it could be compared to an individual taking half of their prescribed medication (miller & rollnick, 2014). the importance of measuring treatment fidelity was illustrated by rodriguez et al. (2009) whose findings showed a decrease in problem behavior for three children enrolled in school programs when their teachers scored higher on treatment fidelity measures. teachers with lower fidelity scores showed minimal decrease in problem behavior. although more research is needed on the effects of treatment fidelity, high quality implementation can impact results of an intervention (thijssen et al., 2017). many times, a behavior technician that has not mastered a skill can practice mistakes leading to problems with treatment outcomes (digennaro-reed et al., 2011). a key component of aba-based services is delivering prompts and reinforcement at the rate defined by the clinical supervisor. the timing of delivery is important because the implementer risks missing an opportunity to prompt or reinforce the behavior of interest, inadvertently reinforcing undesirable behaviors (cooper et al., 2019; mayer et al., 2012). for example, when considering the target behavior of a child vocally stating the word bird at the presentation of the image of a bird on a picture card, the behavior technician will present the card and should provide a short delay (i.e., 2-3 s) to allow a response. if the child responds within the specified time with the vocal utterance bird, a prescribed reinforcer is delivered; if there is no utterance, a prescribed prompt can be delivered. if, in this example, the technician waits 10 s for a response and during the 10 s delay the child looks at their mother and then says the vocal utterance bird, receiving reinforcement for that vocal utterance, the behavior technician may have reinforced labeling the client’s mother with the vocal utterance bird. in this example, latency in the delivery of reinforcement is critical to the fidelity of the intervention. 293 supervision strategies for treatment fidelity and job satisfaction in applied behavior analysis services / saunders importance of job satisfaction interventions derived from the science of aba have been highly successful in helping to improve the developmental delays of children with asd and studies show evidence of the significant benefits for improving quality of life (ben-itzchak & zachor, 2007; eldevik et al., 2012; leaf et al., 2011; lovaas, 1987; rogers & vismara, 2008). however, grindle et al. (2009) discussed the limited research on parents’ perceptions of the effects of aba-based services for their children. grindle et al. explored parent perceptions by interviewing parents (i.e., 32 mothers and 21 fathers) who received 2 years of aba-based services about their experience in home-based services. grindle et al. presented findings indicating most parents recognized the benefits of 2 years of aba-based services for their child and their child achieved growth in developing new skills and reduced challenging behaviors. grindle et al. (2009) went further to offer a summary of frequently occurring concerns shared by parents. difficulties with direct support staff were expressed by 91% of mothers and 100% of fathers. the difficulties included challenges with recruiting new therapists and high turnover rates, causing disruption to their program. parents felt that the challenges related to recruitment and retention slowed progress for their child. in addition, parents of children receiving ababased services feel that maintaining a strong rapport with their intervention team allows for more successful outcomes (grindle et al., 2009). leach (2005) surmised that well-established positive rapport has a direct effect on outcomes of treatment. when a behavior technician with whom a family has a strong bond resigns, the family’s trust could weaken for the entire organizations and its aba-based program. the behavior technician’s role is oftentimes entry level; this means that many behavior technicians have little to no experience or training prior to starting in their first position. although offering treatment in a tiered model helps to keep costs low for those funding treatments, it places a heavy burden on the employer to train those individuals (wood et al., 2007). it is important to consider that the population receiving services can be vulnerable and requires skilled and carefully planned treatment implementation and oversight by the supervising behavior analyst. any level of turnover results in high costs for organizations. sundberg (2016) asserted that the average cost of turnover for a behavior technician to the organization can be as high as $5,000 per person. in addition to the expected costs associated with turnover, such as advertising fees, recruitment, and orientation (abbasi & hollman, 2000), there are also hidden costs such as decreased morale, decreased consumer relations, and damaged organizational reputation. when a behavior technician resigns, there are direct consequences to the consumers, sometimes resulting in the decision to terminate services, submit complaints or find another provider to work with. employees who are satisfied are more loyal (sageer et al., 2012) and more likely to be retained. collins et al. (2008) found that employee retention is a catalyst to consumer satisfaction and relations. these findings are supportive of geyer’s (2005) argument that there is a need for improved training, through evidence-based supervisory interventions, to improve employee satisfaction. collins et al.’s (2008) results showed that more satisfied employees do a better job and make their patients happier. businesses should focus on preserving highly trained employees that they have within their organization by enhancing job satisfaction through the supervision and training programs offered to employees (abbasi & hollman, 2000). ganster et al. (2011) stated, “how individuals are rewarded at work is perhaps one of the most salient features of the work environment and can serve as a source of satisfaction, challenge and fulfillment or a source of uncertainty, mistrust and perceived inequity” (p. 224). it is essential that a model of reinforcement and reward be incorporated into supervision as part of maintaining satisfaction and staff retention for ababased service providers if those are desired behaviors to increase. connecting supervision, job satisfaction, and treatment fidelity job satisfaction is a vital aspect of consideration for supervisors (kazemi et al, 2015; reid et al., 2021). the measurement of job satisfaction for human service providers directly correlates to employee effort, interactions with consumers, and even absenteeism (reid et al., 2021). there are many factors that can contribute to employee satisfaction and retention, but a key controllable variable is the supervisor’s efforts towards fostering a motivating work environment. regrettably, opportunities for supervisors to provide immediate reinforcement (e.g., praise, support, training) to behavior technicians is commonly limited to supervision sessions. additionally, many behavior analysts are not trained or fluent in supervisory practices and consequently those practices can be applied inconsistently. as such, behavior technicians may leave their position because of lack of supervisordirected reinforcement, lack of effective training on interventions, or lack of motivation to keep them in their role. guidelines for effective supervision strategies are emerging (e.g., bacb, 2018; reid et al., 2011; sellers et al., 2016; turner et al., 2016). when a supervisor does not have a structured model, there is room for variability and possibly inadequate supervision that could lead 294 january 2023, volume 15, issue 3, 291-306 to dissatisfied employees (green et al., 2002). when an employee feels valued by the supervisor, there is a decrease in rates of turnover (eisenberger et al., 2002). a positive relationship between employees and their supervisors is a strong indicator of employee productivity and retention (kuvaas & dysvik, 2010). as previously discussed, behavior analysts who are supervising the direct services of technicians are required to provide hands-on training of interventions. kazemi et al. (2015) examined turnover of technicianlevel interventionists working with individuals with asd through an extensive survey. kazemi et al. evaluated several aspects of turnover in aba-based services and found that 38% of the 96 respondents were likely to leave their position because they were not satisfied with their supervision, training, and pay. notably, kazemi et al. expressed that pay level was not indicative of satisfaction, meaning the amount the technician was paid varied and satisfaction with the amount was subjective. simply providing generic supervision to technicians is not enough to ensure they are supported and equipped to not only perform, but also to sustain their role. the bacb (2018) recently released an updated supervision curriculum. this is a great step and provides necessary guidance; however, there continues to be a need to build access to trainings and improve supervisory practices. to improve behavior technicians training, reid et al. (2017) suggested that clinical supervisors not only supervise the direct services of assigned clients, but also have a collaborative approach with all aspects of the behavior technician’s employment from the start of hire. furthermore, gibson et al. (2009) surmised that perceived supervisor support was connected to technicians strong or weak feelings of accomplishment and emotional exhaustion. when a technician perceives they are supported by the supervisor, there is less emotional exhaustion and a greater feeling of achievement in their work. the supervising behavior analysts must not only focus their time on the case supervision (i.e., ensuring programing is individualized and appropriate for the consumer), but also the relationship they have with their supervisees. it is a difficult challenge for behavior analysts, especially when considering all the duties involved in their role. achieving treatment fidelity can be a challenge; therefore, strategies to ensure that there be consistent supervision that incorporate opportunities to evaluate and maintain skills are essential (carr et al., 2013). the findings in the literature suggest that supervisory methods are linked to improved treatment fidelity and job satisfaction. essentially when an employee is doing a good job, they are happier at that job. when employees are provided with quality supervision, they are more loyal, less likely to quit, more productive in their work, and are better service providers. moreover, job satisfaction and retention lead to consumer satisfaction and lower turnover costs for the employing organization. much of the research in employee satisfaction and retention has mainly been derived by employee and supervisor reports via survey, such as that used by collins et al. (2008). the purpose of the present study was to evaluate the effects of the supervisor training curriculum: evidence-based ways to promote work quality and enjoyment among support staff (reid et al., 2011) when applied to clinical supervision with behavior technicians as it relates to treatment fidelity and job satisfaction. the supervision training used was interpreted as an intervention package that included modifying the level, quality, and expectations related to the performance of a clinical supervisor who provided oversight to three behavior technicians. method participants three behavior technicians participated in this study. all potential candidates volunteering to participate must had worked at the research site for a minimum of 6 months to be eligible. participants were offered an opportunity to volunteer and were selected at random by an administrative assistant. any behavior technicians who volunteered but were not assigned to shared cases with the selected a selected clinical supervisor were excluded from participation. this exclusionary criterion was established because the study required that the clinical supervisor who was selected to implement the intervention have prior supervisory experience with the technician and the analyst and technicians had a shared case assignment at the time of the study. the three technicians were selected at random from those who volunteered and were not excluded. participants were all females with an age range of 27-52 years, were employed at the site for a range of 8 months to 11 years, two participants identified as white, and one participant identified as black. educational backgrounds varied, with participants holding bachelor level degrees in psychology and social work, and a master’s degree in behavior analysis. each technician was trained in the implementation of protocols written by a behavior analyst and using prompting and reinforcement delivery systems. each technician met minimum competencies as required of their job, requiring a minimum score of 80% before beginning direct service with clients. the competency checklist was composed of 17 subject areas that required formal training related to the role of a behavior technician. setting this study was conducted at an agency in the northeast providing home, school, and clinic-based intervention for individuals between birth and 21 295 supervision strategies for treatment fidelity and job satisfaction in applied behavior analysis services / saunders years of age. the intervention was conducted with three behavior technicians delivering aba services for a child diagnosed with asd, aged 2.5 years, in the child’s home setting under the clinical supervision of a certified behavior analyst. dependent variables one variable evaluated was treatment fidelity, the measure in which the behavior technicians implemented protocols as designed. for the purposes of this study, specific components of the treatment protocol were identified as the targets. the identified targets were the latency between the delivering a discriminative stimulus and the delivery of a prompt, the latency between the delivery of reinforcement between the prompted or independent behavior, and missed opportunities to deliver a prompt or reinforcement. another variable that was evaluated during the study was job satisfaction. this study used the job descriptive index-job in general ([jdi-jig] bowling green state university, 2009) to measure the level of job satisfaction the behavior technicians had before and after the intervention. the jdi-jig requires the employees to consider specific components of their job as well as their job in general and rate their level of satisfaction across each. the jdi-jig components evaluated by the participants in this study were their level of satisfaction in their (a) work on present job, (b) pay, (c) opportunities for promotion, (d) supervision, (e) people on present job, and (f) job in general. for the section related to work on present job and supervision, the technicians were asked to think specifically about case that they were to be illustrating in the study and the supervision they are receiving from the assigned clinical supervisor on that case. all other responses related to pay, opportunities for promotion, people on present job, and job in general were to be reflection on their attitude in each area across all aspects of their job. this served not only as a tool for measuring job satisfaction, but also as a social validity measure of the intervention itself. data collection for the three phases in this study (i.e., baseline, intervention, and maintenance), trained observers recorded video samples of the latency of delivering prompts and reinforcement by the technicians across multiple. data were recorded using a paper data sheet and the time stamp located on each video sample. the use of video samples allowed the observer to pause and rewind to increase the likelihood of accuracy in the data. the data sheet was designed to allow the observer to document the recorded latency or an absence of delivering a prompt or reinforcement by the technician. delivering prompts was defined as the technician providing the client with one of the following teaching prompts during a teaching trial: full or partial physical prompt, full or partial verbal prompt, gestural or a model prompt. delivering reinforcement was defined as the technician delivering an item to the client that had been established as reinforcing through a preference assessment conducted at the onset of each session. latency was defined as the amount of time recorded between the delivery of a discriminative stimulus by the technician and the delivery of a prompt or a reinforcer. during each video sample, there were several opportunities to deliver reinforcement or prompts. each trial conducted by the technician offered an opportunity for delivery of prompts or reinforcement; an absence of delivering either was recorded as such. during the baseline and maintenance phase of the study, the jdi-jig was used to provide a measure of job satisfaction in numerical form with a top score of 54 for each facet. the percentage of job satisfaction was determined by measuring the actual score and dividing it by the total possible score to provide a percentage that was applied as level of job satisfaction for the specified facet across two phases of the intervention. the scores indicated the level of job satisfaction the participant had prior to receiving the intervention (i.e., during baseline) and after receiving the intervention (i.e., during maintenance). general procedures the clinical supervisor completed 14 hr of group training across two days and was required to display mastery for each module. attendees to the training received an individual trainee guide and handouts to keep as reference tools. all trainees attending were required to engage in role-play activities and be active in discussions related to each of the 11 modules. to confirm the attendees were able to correctly implement the evidence-based methods in the supervision intervention package, there was a required on-the-job competency assessment that evaluated generalization. a generalization competency assessment was used to evaluate the clinical supervisor implementing all relevant strategies with a supervisee in the natural setting, requiring a minimum of 90% accuracy to meet competency. a nonconcurrent multiple baseline design across subjects was used to measure the effectiveness of the supervisor training intervention (reid et al., 2011) on the behavior technicians’ treatment fidelity. there were three phases (i.e., baseline, intervention, and maintenance) in this study across three participants. during the baseline phase, a measure of current performance was recorded for each participant prior to receiving the intervention. the intervention phase was staggered across all three participants to ensure 296 january 2023, volume 15, issue 3, 291-306 that the onset of the intervention phase occurred across different times. the maintenance phase began for participants once it was determined that mastery was achieved. during the maintenance phase, the clinical supervisor discontinued providing supervision for the established protocol, and fidelity continued for a minimum of four sessions across several weeks. the supervising behavior analyst provided a job duties checklist detailing expectations and reviewed progress and errors using diagnostic performance feedback during supervision sessions as well as formal and informal performance monitoring. the supervisor continued to train using the strategies outlined in the supervision training package until the technicians were accurately implementing the protocol as designed. baseline during baseline, video samples were recorded of the behavior technician implementing a protocol designed to teach gross motor imitation skills and a protocol designed to teach matching skills with their client. the client and protocols were the same across all three participants. both protocols called for a continuous schedule of reinforcement for all prompted and independent behaviors. the behavior technicians were expected to avoid missed opportunities and to prompt and reinforce responses. the video samples were reviewed, and a baseline measure was determined for each participant prior to the onset of the intervention measuring the latency between the delivery of a discriminative stimulus and the delivery of a reinforcer or the latency of the delivery of the prompt and reinforcer. the latency of the delivery of prompting and reinforcement was determined as the time in which it took for the technician to deliver the choice response (i.e., prompt or reinforcement) during a teaching procedure. missed opportunities for the delivery of prompts or reinforcement were also measured using frequency of occurrence per video sample. during each session, a video sample was recorded to allow for more accurate measurement, opportunities to attain interobserver agreement, and were used during the feedback sessions in which the supervising behavior analyst was not present at the live session. during the baseline phase, each participant was required to complete a survey that measured their level of job satisfaction, the jdi-jig. the survey results were not shared with the supervising behavior analyst, and results did not have any bearing on the employment status of the participant. the purpose was to establish a baseline level of job satisfaction prior to the intervention and to measure job satisfaction again post intervention to compare results and evaluate any changes. intervention the intervention phase was staggered across all three participants. although the researcher preferred to have a minimum of three points of reference prior to termination of a baseline phase, it was determined by the supervising behavior analyst to begin intervention or participant 1 after only having two data points for reference in baseline. participant 1 was scheduled to implement sessions with the client a frequency of one time per week, whereas other participants had several sessions scheduled per week with the same client. the limited frequency that participant 1 had sessions with the client was a potential challenge, as it was important to the behavior analysts that the protocol be implemented to fidelity as soon as possible. during baseline, behavior challenges were identified, and it was hypothesized that they could have been related to the lack of fidelity with the targets being measured. it was determined that, for the well-being of the client, participant 1 begin the intervention phase no later than by the third session. the latency of the choice response (i.e., prompt or reinforcement) and missed opportunities for delivering prompts and reinforcement during video sample was recorded in the same way they were recorded at baseline. during the intervention phase, the supervising behavior analyst began by providing each technician with a job duties checklist that detailed expectations of the delivery of prompts and reinforcement, reviewed the job duties checklist, provided modeling of the expectation, observed implementation, and provided diagnostic performance feedback during direct supervision sessions (reid et al., 2011). diagnostic feedback included the written and verbal description of expectations on delivering prompts or reinforcement with immediacy and as designed. the clinical supervisor modeled the expected behaviors by implementing the protocols (i.e., gross motor imitation and matching using discrete trial training) and delivering reinforcement within the required 1-3 s or prompts within the required 2-5 s during sessions with zero missed opportunities. the technician was then observed in person and in a video sample implementing discrete trial training procedures with several opportunities for the delivery of reinforcement or prompts. the technician received ongoing feedback throughout supervision that included praise for correct action and a detailed description of incorrect action. to ensure reliability, the same clinical supervisor provided the supervision across all three participants with the same client using the same teaching protocols. each video sample was reviewed by two separate observers who were provided with the definition of the target behaviors. the evaluators collected data on the latency of the delivery of 297 supervision strategies for treatment fidelity and job satisfaction in applied behavior analysis services / saunders prompts or reinforcement and missed opportunities for each videotaped session. the information was used to determine readiness for phase changes, behavior changes, treatment fidelity, and to attain a measure of interobserver agreement. maintenance and generalization to determine if the technicians maintained the skills acquired during the intervention phase, maintenance probes were conducted across a minimum of four and a maximum of nine sessions, where the technicians returned to baseline conditions and data were collected in the same manner as phase 1 and phase 2. although it would have been ideal to have also included opportunities to probe generalization, the participants were unable to attain consent to collect video samples during sessions with clients other than the shared case assignment identified for the study. interobserver agreement to ensure reliability, data were recorded of the participants implementing procedures across numerous videotaped samples during each phase. two trained observers viewed the video recordings and collected data on percentage of sessions. a total agreement-recording formula was used to calculate the percentage of agreement between observers (gast, 2010). the mean percentage of agreement for latency to prompt across participants was 92% (range, 90-93%) at baseline, 97% (range, 92.4-100%) during the intervention phase, and 94% (range, 83-100%) during the maintenance phase. the mean percentage of agreement across participants for latency to reinforcement was 97% (range, 94-100%) at baseline, 97% (range, 91.2-100%) during the intervention phase, and 99% (range, 95.5-100%) during maintenance. the mean percentage of agreement across participants for missed opportunities was 98% (range, 93-100%) at baseline, and 100% agreement across both the intervention and maintenance phases. results data analysis of treatment fidelity each participant showed variability in the latency of the delivery of prompts or reinforcement at baseline. in addition, each participant showed missed opportunities beyond the expected threshold of three during baseline. results for the latency of the delivery of prompts are presented in figure 1. during the baseline condition, participant 1 had an average latency of 24.07 s for the delivery of prompts, participant 2 had an average latency of 38.02 s for the delivery of prompts, and participant 3 had an average latency of 4.1 s for the delivery of prompts, indicating within criterion mastery at baseline. figure 1 latency to prompt note: this graph depicts results for three behavior technicians and latency of the delivery of prompts. data are reported as the average latency for each participant across session samples during three phases: baseline, intervention and maintenance. participant 1 reached criterion for the delivery of prompts after three intervention sessions and continued to receive intervention for an additional two sessions before returning to baseline conditions at session eight; the average latency during the intervention phase for the delivery of prompts reduced to 4.05 s. participant 2 reached criterion for the delivery of prompts after two intervention sessions and continued to receive intervention for an additional five sessions before returning to baseline conditions at session 10; the average latency during the intervention phase for the delivery of prompts reduced to 2.7 s. participant 3 had already established achievement of mastery criterion for the delivery of prompts at baseline; however, to ensure consistency across all targets being measured, intervention was implemented for six sessions. the latency of delivery of prompts did show a slight reduction to an average of 3.3 s before returning to baseline conditions at session 12. participant 1 had eight maintenance sessions over several weeks. the average latency for delivery of prompts maintained at or below the established criteria for all maintenance sessions. participant 2 had six maintenance sessions. the average latency 298 january 2023, volume 15, issue 3, 291-306 for delivery of prompts maintained at or below the established criteria for all maintenance sessions. participant 3 had four maintenance sessions. the average latency for delivery of prompts maintained at or below the established criteria for all maintenance sessions. results for the latency of the delivery of reinforcement are presented in figure 2. during baseline, participant 1 had an average latency of 14.46 s for the delivery of reinforcement, participant 2 had average latency of 8.5 s for the delivery of reinforcement, and participant 3 had an average latency of 5.87 s for the delivery of reinforcement. figure 2 latency to reinforcement note: this graph depicts results for three behavior technicians and latency of the delivery of reinforcement. data are reported as the average latency for each participant across session samples during three phases: baseline, intervention and maintenance. participant 1 reached criteria for the delivery of reinforcement after two intervention sessions before returning to baseline conditions at session eight. the average latency for delivery of reinforcement reduced to 3.31 s from the 14.46 s recorded at baseline. participant 2 reached criterion for the delivery of reinforcement after one intervention session before returning to baseline conditions at session 10. the average latency for delivery of reinforcement reduced to 1.85 s from the 8.5 s at baseline. participant 3 reached criterion for the delivery of reinforcement after one intervention session before returning to baseline conditions at session 12. the average latency for delivery of reinforcement reduced to 3.46 s from the 5.87 s recorded at baseline. the average latency of the delivery of reinforcement for all participants maintained at or below the established criteria of no more than a 3 s latency between the required response and the delivery of an established reinforcer for all maintenance sessions. results for missed opportunities to deliver a prompt or reinforcement are presented in figure 3. during baseline, participant 1 missed opportunities to provide reinforcement or prompts on an average of seven opportunities. participant 2 missed opportunities to deliver prompts or reinforcement on an average of 11 opportunities. participant 3 missed opportunities to deliver reinforcement or prompts on an average of five opportunities. during intervention, the average number of missed opportunities reduced to three per session sample for participant 1, zero missed opportunities per session sample for participant 2, and the average number of missed opportunities reduced to three per session sample for participant 3. all participants were exposed to probes across several weeks. participant 1 had eight maintenance probes, participant 2 had six probes, and participant 3 have four maintenance probes. the average number of missed opportunities maintained at or below the established criteria of no more than three per session sample for all maintenance sessions. figure 3 missed opportunities note: this graph depicts results for three behavior technicians and missed opportunities to deliver a prompt and/or deliver reinforcement. data are reported as the number of missed opportunities for each participant across session samples during three phases: baseline, intervention and maintenance. 299 supervision strategies for treatment fidelity and job satisfaction in applied behavior analysis services / saunders data analysis of employee satisfaction results for participant 1 on the jdi-jig are presented in table 1. results indicated a baseline level of 55% for work on present job, which increased to 78% during the maintenance phase. the percentage of job satisfaction for pay was 59% and increased to 70% during the maintenance phase. in the facet of opportunities for promotion, participant 1 had a job satisfaction score of 48% that increased to 70% when measured in maintenance. the supervision facet score was 61% at baseline and increased to 93% when measured during the maintenance phase. the score under people on your present job for participant 1 was 80% at baseline and increased to 87% during the maintenance phase. the general score under job in general was 87% for participant 1 at baseline, and increased to 91% after the intervention in the maintenance phase. data are reported as percentages of job satisfaction across six facets. table 1 results by percentage for participant 1 item baseline maintenance work on present job 55 78 pay 59 70 opportunities for promotion 48 70 supervision 61 93 people on your present job 80 87 job in general 87 91 results for participant 2 on the jdi-jig are presented in table 2. results indicated a baseline level of 80% for work on present job which increased to 89% during the maintenance phase. the percentage of job satisfaction for pay was 67% and increased to 70% during the maintenance phase. in the facet of opportunities for promotion, participant 2 had a job satisfaction measure of 78% that remained stable, scoring the same 78% when measured in maintenance. the supervision facet score was 63% at baseline and increased to 96% when measured during the maintenance phase. table 2 results by percentage for participant 2 item baseline maintenance work on present job 80 89 pay 67 70 opportunities for promotion 78 78 supervision 63 96 people on your present job 85 96 job in general 87 96 the score for people on your present job for participant 2 was 85% at baseline and increased to 96% during the maintenance phase. the general score of job in general was 87% for participant 2 at baseline, and it increased to 96% after the intervention in the maintenance phase. the jdi-jig scale provided a measure of job satisfaction in numerical form with a top score of 54 for each facet. the percentage in this table was determined by measuring the actual score and dividing it by the total possible score to provide a percentage that was applied as level of job satisfaction for the specified facet across two phases of the intervention. the scores indicated the level of job satisfaction the participant had prior to receiving the intervention (i.e., baseline) and after receiving the intervention (i.e., maintenance). results for participant 3 on the jdi-jig are presented in table 3. results indicated a baseline level of 55% for work on present job, and the level increased to 77% during the maintenance phase. the percentage of job satisfaction for pay was 77% and increased to 85% during the maintenance phase. in the facet of opportunities for promotion, participant 3 had a job satisfaction measure of 48% and increased to 77% when measured in maintenance. the supervision facet score was 62% at baseline and increased to 92% when measured during the maintenance phase. table 3 results by percentage for participant 3 item baseline maintenance work on present job 55 77 pay 77 85 opportunities for promotion 48 77 supervision 62 92 people on your present job 85 87 job in general 87 90 the score under people on your present job for participant 3 was 85% at baseline and increased to 87% during the maintenance phased. the general score of job in general was 87% for participant 3 at baseline, and it increased to 90% after the intervention in the maintenance phase. the jdi-jig scale provided a measure of job satisfaction in numerical form with a top score of 54 for each facet. the percentage in this table was determined by measuring the actual score and dividing it by the total possible score to provide a percentage that was applied as level of job satisfaction for the specified facet across two phases of the intervention. the scores indicated the level of job satisfaction the participant had prior to receiving the intervention (i.e., baseline) and after receiving the intervention (i.e., maintenance). 300 january 2023, volume 15, issue 3, 291-306 summary of results the results provided evidence to support the use of the supervisor training curriculum: evidence-based ways to prompt work quality and enjoyment among support staff (reid et al., 2011) as an intervention package for evidence-based supervision strategies to improve treatment fidelity and job satisfaction for behavior technicians. participants all reached the established criterion within 1 to 4 sessions and maintained levels at or below criterion across several weeks postintervention. the results of the jdi-jig showed that all three participants had an increase in job satisfaction from baseline conditions across multiple facets of their job. discussion the purpose of this study was to evaluate the effects of the supervisor training curriculum: evidencebased ways to promote work quality and enjoyment among support staff (reid et al., 2011) as a model to be used by a behavior analyst when supervising behavior technicians providing aba-based services for a child with asd. the supervision training curriculum developed by reid et al. (2011) outlined 11 areas of evidence-based practice in supervision. the curriculum provided a structured protocol that can be applied to behavior analysts who are supervising technicians. the curriculum provides a model that offers behavior analysts a way to incorporate evidence-based practices into every supervisory session. the strategies outlined are intended to support the growth and development of the technicians while also promoting enjoyment in their work. although behavior analysts are often familiar with evidencebased strategies that can be applied within their supervision sessions, the application of strategies can be inconsistent and lack structure. by incorporating a more structured approach, such as that provided in the supervisor training curriculum, behavior analysts can ensure they do not miss important components to supervision. typically, the structure, quality, and outcomes of clinical supervision are dependent on the decisions and experience of the supervising behavior analyst. although they are required to receive training in supervision, this training often focuses on supervision of a trainee with less emphasis on supervision of a supervisee. consequently, it is common for clinical supervisory skills to be learned on the job, by trial and error, or through self-initiated, but not required, post certificate training. the supervision training curriculum used in this study provided an overview of evidencebased practice in supervision, and competency testing was used as a tool to determine whether a behavior analyst, acting as a clinical supervisor who attended the training, met minimum levels of competency to provide supervision. this tool could be used as a general guide for any behavior analyst who will be acting as a supervisor and may provide a strong foundation for clinical supervisors to use. the clinical supervisor in this study generalized supervisory practices learned in a 2-day training to his practice. a generalization observation was scheduled following training, and a final competency checklist was completed to assess their implementation of the strategies in a real-world setting. the generalization session showed a 100% accuracy on the competency checklist completed 4 weeks after training. prior to receiving the 2-day training on supervision, the clinical supervisor reported that they had at least a basic understanding of all areas that were covered in the modules; however, they did not have a strategy to pull each of them together to structure his supervision practices. after receiving training, the supervisor changed their approach to supervision to include evidence-based supervisory strategies across all direct supervision sessions. the three technicians had an inconsistent latency of the delivery of reinforcement and prompts, as well as exceeded the expected number of missed opportunities to provide a reinforcer or prompt at baseline. the three participants did not meet fidelity at baseline when treatment fidelity was measured. the degree of variability was mixed across all three participants. treatment fidelity can be measured by analyzing how closely a treatment is implemented to the way it was designed. the treatment protocols outlined that the latencyto-delivery prompts should be 2-5 s, reinforcement should be delivered within 3 s of the choice response, and there should be no more than three missed opportunities per session. the baseline level for participant 1 showed that, across two consecutive sessions, the average latency was 24.07 s to deliver a prompt, 14.46 s to deliver reinforcement, and the participant missed an average of seven opportunities. consequently, participant 1 had three target areas that required intervention to achieve a higher level of fidelity. participant 2 had an average latency to prompt of 38.2 s, 8.5 s to reinforce, and 11 missed opportunities at baseline; all three target areas were well beyond achieving fidelity. participant 3 had an average of 4.1 s, 5.87 s to reinforce and missed opportunities an average of five times at baselined. participant 3 was close to fidelity prior to the intervention across all three target areas. upon receiving a more structured approach to supervision, each of the behavior technicians achieved and maintained improvements in the level of treatment fidelity. participant 1 made dramatic improvements in reducing the latency for the delivery of prompts and reinforcement as well as reducing missed opportunities within session samples significantly. with the change 301 supervision strategies for treatment fidelity and job satisfaction in applied behavior analysis services / saunders to supervision strategies, participant 1 reduced the average latency to well within that required to meet fidelity benchmarks. the results for participant 2 were no different, having been previously exposed to the implementation of the teaching protocols; however, there were significantly longer latencies than designed across both prompts and reinforcement at baseline, as well as concerns with missed opportunities. after receiving the intervention, treatment fidelity improved across all three target areas, reducing the latency to well within those outlined in the treatment protocols. interestingly, participant 3 was close to meeting fidelity across all three target areas at baseline. having more experience, 11 years as a behavior technician, implementing similar protocols may have afforded her an advantage. participant 3 did show improvements and reduced the average latency for both the delivery of prompts and reinforcement as well as missed opportunities to an average of zero before returning to baseline conditions and maintaining that level across at least three session samples. these improvements, although seemingly minor, may make a difference in treatment outcomes. as expected, there were improvements in treatment fidelity with the implementation of an evidence-based structure supervision model. it was clear, from the findings, that after receiving more structured supervision that consistency incorporated evidence-based strategies there was a positive effect on the level of treatment fidelity. the level of treatment fidelity for all participants showed a range from significant to minor across all participants. although the researcher assumed there would be some difference between the latency and some missed opportunities, during baseline sessions, participant 1 and participant 2 showed a latency of between 24 seconds to as much as 59 seconds before delivering prompts, with a protocol designed to deliver the prompt within 2 to 5 seconds after the instruction. not reported in the data, but appearing in the session samples, were repeated deliveries of the sd before ultimately prompting the choice response. this deviation from the protocol as designed could be potentially detrimental to the overall success of the client. this unexpected finding reinforced the need for an intervention targeted on improving supervision strategies that included focusing on treatment fidelity. the findings in this study indicated that the quality and type of supervision influenced the level of job satisfaction for technicians. the tool used to analyze job satisfaction; the jdi-jig is designed to assess the level of job satisfaction across several components of an individual’s job. from the research, job satisfaction is highly correlated with an individual’s perception of their relationship with their supervision (eisenberger et al., 2002). the researcher anticipated that, with an improved quality of supervision that incorporated empirically validated strategies, there would be improvements in the satisfaction level. as expected, job satisfaction, as it relates to the clinical supervision, improved across participants. results indicated a significant change in this area of job satisfaction; at baseline, all three participants’ supervision satisfaction levels were around 60%, and, after the intervention, they improved to over 90% satisfaction. the researcher also expected that there may be improvements in the job satisfaction area of work at the participants’ present job. the reason for this expectation was because the participants were asked to consider the specific case assignment when responding to this prompt in the survey. the researcher expected that the participants would be more satisfied with their work on that case assignment if they were implementing procedures with more integrity/fidelity. research has indicated that, when employees are more competent in their job, they have a high level of satisfaction (arifin, 2014). therefore, receiving supervision focused on improving competency was likely to improve satisfaction. the supervision intervention had a strong emphasis on building skills to competency. the technicians had all been exposed to the teaching procedures prior to the intervention; however, previous supervision was not focused on building competency in the same way and reportedly was variable and inconsistent. the participants baseline level of satisfaction for work on present job ranged from 55% to 80% and increased to a range between 77% and 89%. the technicians receiving the structured and improved quality of supervision not only had improvements to levels of treatment fidelity, but also marked improvements across all facets of job satisfaction assessed. the researcher anticipated that employee satisfaction for the research participants would be high prior to intervention, with the concern that an employee who was willing to volunteer to participate in a study without additional incentives would have a proactive work personality, which is linked to high levels of job satisfaction (li et al., 2017). employees with proactive personalities are those individuals who taken initiative, engage in a wide range of activities, and are actively engaged in activities in the workplace (li et al., 2017). however, the findings were inconsistent with those of li et al. (2017), as all participants, showing some aspects of proactive personalities and highly engaged workers, showed areas for improvement in job satisfaction. not surprisingly, participant 3, who had been employed with the organization the longest, had the highest baseline and postbaseline results. this supports the previous studies that have indicated job satisfaction correlates with an increase in retention (yarbrough et al., 2017). 302 january 2023, volume 15, issue 3, 291-306 in this study, the participants did not have a change in clinical supervision strategies across any of their other case assignments. therefore, they were receiving the same manner of clinical supervision as they received in the past in all other areas of their work outside of the specific case assignment for the study. one would expect that all other areas may reflect no change in satisfaction level. however, the findings indicated that, by simply changing the supervision methods for one case assignment, there may be improvements across other facets of job satisfaction. all three technicians had been trained on and implementing the teaching protocol for at least 30 days prior to the baseline phase. it was determined at baseline that they were not implementing the protocols with fidelity. the supervisory strategies the clinical supervisor was using prior to the intervention were producing low levels of fidelity. at the start of the intervention, high rates of fidelity were reached quickly, meeting established criterion within one to four supervision sessions, with results maintaining for several weeks without direct intervention. the clinical supervisors who are supervising in a similar manner may find the results applicable when they are overseeing technicians who have low fidelity in implementation. children receiving services may have the potential for improved outcomes from technicians who are receiving structured quality supervision, such as the one used for this study. during the baseline phase, challenging behaviors were exhibited by the client receiving the treatment. although it was not the purpose of this study, upon the implementation of the intervention, the rates of challenging behavior decreased as the delays in the latency of the delivery of prompts and reinforcement decreased. with the reduction in challenging behavior, there were increased teaching opportunities across session samples. although speculative, low levels of fidelity may have been contributing to the behaviors displayed by the child during sessions, as it appeared problem behavior decreased over time in the recorded video samples. higher levels of treatment fidelity have previously been associated with decreases in challenging behaviors (pinkelman & horner, 2017). the results of this study may be valuable to those who provide similar tiered model services such as nurses, teachers, and occupational therapists. the study provides a practical and cost-effective training package as well a strategy to assess job satisfaction across several facets. in the field of aba, there are high rates of turnover (novack & dixon, 2019). the skills gained by the behavior analyst participant in this study proved influential in contributing to better treatment fidelity and job satisfaction. the improvements across all facets of the job satisfaction could indicate that, by making a minor change to supervision quality, job satisfaction can potentially improve globally. using instruments, such as the jdi-jig, may help organizations gain a better understanding, monitor, and take actions as they relate to job satisfaction. limitations prior to the start of the study, the researcher was aware of the risks related to exposure to extended baseline conditions. due to this limitation, the researcher observed baseline closely across participants to avoid prolonged exposure. it was determined that participant 1 was a potential risk for this challenge; therefore, baseline was shortened to only two sessions. decreasing the baseline for participant 1 could make the results less reliable, as a stable baseline is best achieved by evaluating behaviors from at least three data points and, in this case, ideally three sessions (kazdin, 2010). another potential limitation to the current design was the issue of generality. in the study, participants had similar pre-baseline demographics. the results of this study cannot be assumed as generalizable outside of the specific setting and beyond the specific participants or even beyond what was achieved on the specific case assignment. as with all singlesubject research, generalization to a larger population can only be shown with future replication studies, where manipulations of the subjects and settings can be made. another limitation to study was related to irreversibility. the newly acquired skills of supervision could not be removed from the supervising behavior analyst who was instructed to stagger the intervention across the three technicians; therefore, it is not possible to know if the clinical supervisor inadvertently began some portions of the intervention with one or more of the participants prior to the onset of the intervention. in addition, the researcher focused on only a few areas associated with treatment fidelity (i.e., latency of delivery of prompts and reinforcement). this is a limitation to the findings of the study; further studies should investigate additional areas of treatment fidelity to examine if similar results are established. although the results of this study are promising, there are limitations related to generalization. the results are specific to the participants and their learning histories, as well as the specific case assignment and the target behaviors evaluated. it is not clear if the same results would be found with other participants, across other target behaviors, with a difficult case assignment. additionally, these results can only be applied to the specific service model and practices of the aba agency used as the research site. it is not possible to know if these results could be generalized across other aba service provider agencies without future research. furthermore, while there was an increase in job satisfaction across participants practice effects 303 supervision strategies for treatment fidelity and job satisfaction in applied behavior analysis services / saunders could occur because of the participants previous exposure the jdi-jig. future research directions turnover is a problem in the field of aba. research has shown job satisfaction as an important factor in reducing turnover (abbasi & hollman, 2000; ezeh & olawale, 2017; kuo et al., 2014). the results related to the employee job satisfaction were favorable in this study, and future research directions could focus on longterm gains of job satisfaction, or the factors related to turnover of behavior technicians. future research could expand on the number of supervisors using the supervision curriculum (reid et al., 2011) across a diverse pool of technicians. additionally, an extension to this study could focus on broadening the setting of the intervention assessing whether the intervention could be generalized across other settings and other types of providers. references abbasi, s. m., & hollman, k. w. (2000). turnover: the real bottom line. public personnel management, 29(3), 333-342. https://doi. org/10.1177/009102600002900303 allen, k. d., & warzak, w. j. (2000). the problem of parental nonadherence in clinical behavior analysis: effective treatment is not enough. journal of applied behavior analysis, 33(3), 373391. https://doi.org/10.1901/jaba.2000.33-373 apba. (2022 may) overview of state laws to regulate practitioners of applied behavior analysis. apba home. https://cdn.ymaws.com/www. apbahome.net/resource/resmgr/pdf/state_ regulation_of_ba_may20.pdf arifin, h. m. (2014). the influence of competence, motivation, and organisational culture to high school teacher job satisfaction and performance. international education studies, 8(1). https://doi.org/10.5539/ies.v8n1p38 behavior analyst certification board. (2014). applied behavior analysis treatment of autism spectrum disorder: practice guidelines for healthcare funders and managers (2nd ed.). littleton, co: author. behavior analyst certification board. (2018). supervision training curriculum outline (2.0) https://www. b a c b.c o m / w p c o nte nt/u p l o a d s / 202 2 /01 / supervision_training_curriculum_220318.pdf behavior analyst certification board. (2020). ethics code for behavior analysts. https://bacb. com/wp-content/ethics-code-for-behavioranalysts/ behavior analyst certification board. (2022). supervision, assessment, training, and oversight. https://www.bacb.com/supervisionand-training/ ben-itzchak, e., & zachor, d. a. (2007). the effects of intellectual functioning and autism severity on outcome of early behavioral intervention for children with autism. research in developmental disabilities, 28(3), 287-303. doi. org/10.1016/j.ridd.2006.03.002 bowling green state university. (2009). job descriptive index. bowling green, oh: author. carr, j. e., wilder, d. a., majdalany, l., mathisen, d., & strain, l. a. (2013). an assessment-based solution to a human-service employee performance problem: an initial evaluation of the performance diagnostic checklist–human services. behavior analysis in practice, 6(1), 1632. https://doi.org/10.1007/bf03391789 council of autism service providers. (2020). applied behavior analysis treatment of autism spectrum disorder: practice guidelines for healthcare funders and managers (2nd edition). casp providers. https://casproviders. org/asd-guidelines/ collins, k., collins, s., mckinnies, r., & jensen, s. (2008). employee satisfaction and employee retention: catalysts to patient satisfaction. health care manager, 27(3), 245-251. https://doi. org/10.1097/01.hcm.0000318755.24332.4b cooper, j. o., heron, t. e., & heward, w. l. (2019). applied behavior analysis (3rd edition). hoboken, nj: pearson education. digennaro-reed, f. d., reed, d. d., baez, c. n., & maguire, h. (2011). a parametric analysis of errors of commission during discrete-trial training. journal of applied behavior analysis, 44(3), 611615. https://doi.org/10.1901/jaba.2011.44-611 eisenberger, r., stinglhamber, f., vandenberghe, c., sucharski, i. l., & rhoades, l. (2002). perceived supervisor support: contributions to perceived organizational support and employee retention. journal of applied psychology, 87(3), 565-573. https://doi.org/10.1037/0021-9010.87.3.565 eldevik, s., hastings, r. p., jahr, e., & hughes, j. c. (2012). outcomes of behavioral intervention for children with autism in mainstream pre-school settings. journal of autism and developmental disorders, 42(2), 210-220. https://doi.org/10.1007/ s10803-011.1234-9 304 january 2023, volume 15, issue 3, 291-306 ezeh, l. n., & olawale, k. (2017). pay satisfaction, job satisfaction and gender as correlates of turnover intention among federal civil servants in awka metropolis of anambra state, southeast, nigeria. international journal of advanced multidisciplinary research reports, 2(1), 12-20. ganster, d. c., kiersch, c. e., marsh, r. e., & bowen, a. (2011). performance-based rewards and work stress. journal of organizational behavior management, 31(4), 221-235. https://doi.org/10.10 80/01608061.2011.619388 gast, d. l. (2010). single-subject research methodology in behavior sciences. new york, ny: routledge. geyer, s. (2005). hand in hand: patient and employee satisfaction. trustee, 589(6), 1219. gibson, j. a., grey, i. m., & hastings, r. p. (2009). supervisor support as a predictor of burnout and therapeutic self-efficacy in therapists working in aba schools. journal of autism and developmental disorders, 39(7), 1024-1030. https://doi.org/10.1007/s10803-009-0709-4 green, c. w., rollyson, j. h., & passante, s. c. (2002). maintaining proficient supervisor performance with direct support personnel: an analysis of two management approaches. journal of applied behavior analysis, 35(2), 205-208. https:// doi.org/10.1901/jaba.2002.35-205 grindle, c. f., kovshoff, h., hastings, r. p., & remington, b. (2009). parents’ experiences of home-based applied behavior analysis programs for young children with autism. journal of autism and developmental disorders, 39, 42-56. https://doi. org/10.1007/s10803-008-0597-z kazdin, a. e. (2010). single-case research designs: methods for clinical and applied settings (2nd ed.). new york, ny: oxford university press. kazemi, e., shapiro, m., & kavner, a. (2015). predictors of intention to turnover in behavior technicians working with individuals working with autism spectrum disorder. research in autism spectrum disorders, 17, 106-115. https://doi.org/10.1016/j. rasd.2015.06.012 kuo, h., lin, k., & li, i. (2014). the mediating effects of job satisfaction on turnover intention for longterm care nurses in taiwan. journal of nursing management, 22(2), 225-233. https://doi. org/10.1111/jonm.12044 kuvaas, b., & dysvik, a. (2010). exploring alternative relationships between perceived investment in employee development, perceived supervisor support and employee outcomes. human resource management journal, 20(2), 138-156. https://doi.org/10.1111/j.1748-8583.2009.00120.x leach, m. j. (2005). rapport: a key to treatment success. complementary therapies in clinical practice, 11(4), 262-265. https://doi.org/10.1016/j. ctcp.2005.05.005 leaf, j. b., leaf, r. b., mceachin, j. j., taubman, m. t., & tsuji, k. h. (2011). a program description of a community-based intensive behavioral intervention program for individuals with autism spectrum disorders. education and treatment of children, 34(2), 259-285. https://doi.org/10.1353/ etc.2011.0012 li, m., wang, z., gao, j., & you, x. (2017). proactive personality and job satisfaction: the mediating effects of self-efficacy and work engagement in teachers. current psychology, 36(1), 48-55. https://doi.org/10.1007/s12144-015-9383-1 lovaas, o. i. (1987). behavioral treatment and normal educational and intellectual functioning in young autistic children. journal of consulting and clinical psychology, 55(1), 3-9. https://doi. org/10.1037/0022-006x.55.1.3 mandell, d. s., stahmer, a. c., shin, s., xie, m., reisinger, e., & marcus, s. c. (2013). the role of treatment fidelity on outcomes during a randomized field trial of an autism intervention. autism, 17(3), 281295. https://doi.org/10.1177/1362361312473666 mayer, g. r., sulzer-azaroff, b., s., & wallace, m. (2012). behavioral analysis for lasting change (2nd ed.). cornwell-on-hudson, ny: sloan. miles, n. i., & wilder, d. a. (2009). the effects of behavioral skills training on caregiver implementation of guided compliance. journal of applied behavior analysis, 42(2), 405-410. https://doi. org/10.1901/jaba.2009.42-405 miller, w. r., & rollnick, s. (2014). the effectiveness and ineffectiveness of complex behavioral interventions: impact of treatment fidelity. contemp clin trials, 37(2), 234-241. https://doi. org/10.1016/j.cct.2014.01.005 mor barak, m., travis, d., pyun, h., & xie, b. (2009). the impact of supervision on worker outcomes: a meta-analysis. social service review, 83(1), 3-32. https://doi.org/10.1086/599028 305 supervision strategies for treatment fidelity and job satisfaction in applied behavior analysis services / saunders novack, m. n., & dixon, d. r. (2019). predictors of burnout, job satisfaction, and turnover in behavior technicians working with individuals with autism spectrum disorder. review journal of autism and developmental disorders, 22(1), 1-9. https://doi.org/10.1007/s40489-019-00171-0 odom, s. l., collet-klingenberg, l., rogers, s. j., & hatton, d. d. (2010). evidencebased practices in interventions for children and youth with autism spectrum disorders. preventing school failure, 54(4), 275-282. https://doi. org/10.1080/1045.9881.003785506 parsons, m. b., reid, d. h., & crow, r. e. (2003). best and worst ways to motivate staff in community agencies: a brief survey of supervisors. mental retardation, 41(2), 96-102. https://doi. org/10.1352/0047-6765(2003)041<0096:bawwt m>2.0.co;2 parsons, m. b., rollyson, j. h., & reid, d. h. (2012). evidence-based staff training: a guide for practitioners. behavior analysis in practice, 5(2), 2-11. https://doi.org/10.1007/bf03391819 pinkelman, s. e., & horner, r. h. (2017). improving implementation of function-based interventions: self-monitoring, data collection, and data review. journal of positive behavior interventions, 19(4), 228-238. https://doi. org/10.1177/1098300716683634 reid, d. h., parsons, m., b., & green, c. w. (2011). the developmental disabilities supervisor training curriculum: evidence-based ways to promote work quality and enjoyment among support staff. washington, cd: american association on intellectual and developmental disabilities. reid, d. h., parsons, m. b., & green, c. w. (2021). the supervisor's guidebook: evidence-based strategies for promoting work quality and enjoyment among human service staff. charles c thomas publisher. reid, d. h., parsons, m. b., & jensen, j. m. (2017). maintaining staff performance following a training intervention: suggestions from a 30-year case example. behavior analysis in practice, 10(1), 12-21. https://doi.org/10.1007/ s40617-015-0101-0 rodriguez, b. j., loman, s. l., & horner, r. h. (2009). a preliminary analysis of the effects of coaching feedback on teacher implementation fidelity of first step to success. behavior analysis in practice, 2(2), 11-21. https://doi.org/10.1007/ bf03391744 rogers. s. j., & vismara, l. a. (2008). evidencebased comprehensive treatments for early autism. journal of clinical child and adolescent psychology, 37(1), 8-38. https://doi. org/10.1080/15374410701817808 sageer, a., rafat, s., & agarwal, p. (2012). identification of variables affecting employee satisfaction and their impact on the organization. iosr journal of business and management, 5(1), 3239. https://doi.org/10.9790/487x-0513239 sarokoff, r. a., & sturmey, p. (2004). the effects of behavioral skills training on staff implementation of discrete-trial teaching. journal of applied behavior analysis, 37(4), 535-538. https://doi. org/10.1901/jaba.2004.37-535 sellers, t. p., leblanc, l. a., & valentino, a. l. (2016). recommendations for detecting and addressing barriers to successful supervision. behavior analysis in practice, 9, 309-319. sundberg, d. p. (2016). why people quit your company and how behavior analysis can slow the revolving door at aba service providers. pompano beach, fl: behavioral science in the 21st century. thijssen, j., albrecht, g., muris, p., & de ruiter, c. (2017). treatment fidelity during therapist initial training is related to subsequent effectiveness of parent management training: oregon model. journal of child and family studies, 26(7) 1991-1999. https://doi.org/10.1007/s10826-017-0706-8 turner l.b., fischer a.j., and luiselli, j.k. (2016). towards a competency-based, ethical, and socially valid approach to the supervision of applied behavior analytic trainees. behav anal pract. 2016 mar 28;9(4):287-298. https://doi.org/10.1007/ s40617-016-0121-4. volkmar, f., siegel, m., woodbury-smith, m., king, b., mccracken, j., & state, m. (2014). practice parameter for the assessment and treatment of children and adolescents with autism spectrum disorder. journal of the american academy of child and adolescent psychiatry, 53(2), 237-257. https://doi.org/10.1016/j.jaac.2013.10.013 wilson, n. a. (2015). factors that affect job satisfaction and intention to leave of allied health professionals in a metropolitan hospital. australian health review, 39(3), 290-294. https:// doi.org/10.1071/ah14198 306 january 2023, volume 15, issue 3, 291-306 wine, b., osborne, m.r., newcomb, e.t. (2020). on turnover in human services. behavior analysis & practice. 2020 jan 6;13(2):492-501. https://doi. org/10.1007/s40617-019-00399-6. wood, a. l., luiselli, j. k., & harchik, a. e. (2007). training instructional skills with paraprofessional service providers at a community-based habilitation setting. behavior modification, 31(6), 847-855. https://doi.org/10.1177/0145445507302893 yarbrough, s., martin, p., alfred, d., & mcneill, c. (2017). professional values, job satisfaction, career development, and intent to stay. nursing ethics, 24(6), 675685. https://doi. org/10.1177/0969733015623098 microsoft word 6_iejee_4_1_oakhill_yuill_garnham international electronic journal of elementary education, 2011, 4(1), 83-106. issn:1307-9298 copyright © iejee www.iejee.com the differential relations between verbal, numerical and spatial working memory abilities and children's reading comprehension jane oakhill ∗∗∗∗ school of psychology university of sussex, brighton, united kingdom nicola yuill school of psychology university of sussex, brighton, united kingdom alan garnham school of psychology university of sussex, brighton, united kingdom abstract working memory predicts children's reading comprehension but it is not clear whether this relation is due to a modality-specific or general working memory. this study, which investigated the relations between children's reading skills and working memory (wm) abilities in 3 modalities, extends previous work by including measures of both reading comprehension and reading accuracy. tests of word reading accuracy and reading comprehension, and working memory tests in three different modalities (verbal, numerical and spatial), were given to 197 6to 11-year old children. the results support the view that working memory tasks that require the processing and recall of symbolic information (words and numbers) are better predictors of reading comprehension than tasks that require visuo-spatial storage and processing. the different measures of verbal and numerical working memory were not equally good predictors of reading comprehension, but their predictive power depended on neither the word vs. numerical contrast nor the complexity of the processing component. in general, performance on the verbal and numerical working memory tasks predicted reading comprehension, but not reading accuracy, and spatial wm did not predict either. the patterns of relations between the measures of working memory and reading comprehension ability were relatively constant across the age group tested. keywords: reading comprehension, reading accuracy, working memory, information processing introduction the concept of working memory has a role in most theories of text comprehension, and in attempts to explain individual differences in text comprehension (see, e.g. just and carpenter, 1992). daneman and carpenter (1980) suggested that the crucial difference ∗ jane oakhill, school of psychology, pevensey 1 building, university of sussex, falmer, bn1 9qh, uk. e-mail: j.oakhill@sussex.ac.uk international electronic journal of elementary education 84 between tests of working memory and those of short-term memory (such as digit span and word span) which are not, or are only weakly, related to comprehension skill, is that shortterm memory tests only require the use of a passive storage buffer. daneman and carpenter went on to argue that both storage and processing of information in memory is important in comprehension, and suggested that the concept of working memory (e.g. baddeley & hitch, 1974) better accounts for the sharing of resources between the processing and storage demands of a particular task. in order to measure this functional capacity, daneman and carpenter developed the reading span task. in contrast to digit span and related tasks, performance on both reading and listening versions of daneman and carpenter's working memory span tasks predicted performance on comprehension tests. daneman and carpenter's (1980; 1983) reading span test is now a frequently used measure of working memory in reading research. in this test, participants either read or listen to a set of unrelated sentences (processing requirement) and have to retain the final word of each sentence (storage requirement) for recall after all the sentences have been read. participants also have to answer simple comprehension questions about the sentences to ensure that they have processed the text for meaning. studies of college students have shown that scores on this test correlate highly with many measures of reading comprehension such as remembering facts, detecting and recovering from semantic inconsistencies, and resolving pronouns, especially those with distant antecedents (see, e.g., cantor, engle, & hamilton, 1991; dixon, lefevre, & twilley, 1988; engle, nations, & cantor, 1990). the correlations of span with performance on various comprehension tests ranged from .7 to .9 in the original samples, and a meta-analysis by daneman and merikle (1996), shows an average correlation of .41 between reading span and global reading comprehension in adults. the link between working memory and reading comprehension probably holds because a major component of skilled comprehension is the ability to compute the semantic and syntactic relations among successive words, phrases and sentences, in order to construct a coherent overall representation of the text. in all current models of text comprehension (e.g. gernsbacher, 1990; johnson-laird, 1983; kintsch, 1998) the processes of integration and inference are important in the construction of a coherent model of the text, both locally and globally. in such models, working memory acts as a buffer for the most recently read propositions in a text, so that they can be integrated with the model of the text so far, and also holds information activated from long term-memory to facilitate its integration with the currently active text (cooke, halleran & o’brien, 1998; graesser, singer & trabasso, 1994). it follows that individuals with limited working memory capacities should be less able to undertake these types of processing than those with greater storage and processing capacities. however, daneman and carpenter's original reading span test itself requires reading, so performance on the test may be partly, or even largely, dependent on general reading ability, which is known to be correlated with reading comprehension skill, but which involves many components other than working memory. furthermore, people have to perform comprehension tasks, albeit simple ones, as an integral part of the reading span task. such considerations raise the question of what underlies the relation between reading span and reading comprehension. thus, given more general arguments for the role of working memory in comprehension, the question remains open as to whether the type of working memory implicated in text comprehension is a general one, one that is specific to language, or to the processing of symbolic information. on the one hand, several studies support the idea that it is the processing of symbolic information that is crucial. these studies show that verbal and numerical span tasks, but not spatial span tasks, predict performance on tests of reading comprehension and other measures of verbal ability (see, the differential relations between verbal, numerical / oakhill, yuill & garnham 85 e.g. daneman & tardif, 1987; shah & miyake, 1996) whereas spatial span, but not reading span, is a good predictor of performance on standardised visuo-spatial tests. on the other hand, domain-general accounts of working memory have been advanced by engle and his colleagues (e.g. engle, tuholski, laughlin & conway, 1999). in such accounts, individual differences are interpreted in terms of the quantity of resources available. turner and engle's (1989) results led them to describe working memory as a general capacity resource, in which it is the capacity to keep active a certain number of elements that is crucial. however, and importantly from our perspective, they did not include a measure of spatial working memory in their study, so it is impossible to know whether their findings would generalise to the spatial domain. in any case, these two views are not incompatible. by "domain-general” engle and colleagues mean that this capacity is not restricted to a certain type of task. furthermore, in a later study, kane, hambrick, tuholski, wilhelm, payne & engle (2004) found that a two-factor model (in which verbal and visuo-spatial memory were separated) was a slightly better fit than a one-factor model in which working memory was regarded as a single construct, although the verbal and visuo-spatial working memory constructs were highly correlated. thus, their data are consistent with a (weak) dissociation between verbal and visuo-spatial working memory capacity. in addition, kane et al. provide some possible reasons to be sceptical of the data that purport to support strong domain specificity and we return to those reasons at the end of the introduction, since they are particularly pertinent to the design of our own study. in a meta-analysis of the relations between wm and comprehension, carretti, borella, cornoldi and de beni (2009) suggest both domain-general and specific factors play a role, with verbal working memory being more predictive. however, they compared verbal working memory only with visuo-spatial tasks (not numerical working memory) and only three of the studies they review included more than one type of working memory task. finally, there is still some ambiguity about the relation between numerical working memory tasks and comprehension in adults. for example, waters and caplan (1996) found that adults’ comprehension was not significantly correlated with numerical working memory tasks, only with reading span tasks. in general, even if both sorts of task correlate with comprehension skill, it is the reading span tasks that show the stronger correlation. in children, as opposed to adults, a number of studies have shown a strong relation between working memory and children’s reading comprehension (e.g. leather & henry, 1994; oakhill, yuill & parkin, 1986; swanson & berninger, 1995; yuill, oakhill & parkin, 1989). this relation between working memory and reading comprehension has been found to hold with tasks that require the processing and storage of words (de beni, palladino, pazzaglia & cornoldi, 1998), sentences (engle, carullo & collins, 1991; seigneuric, ehrlich, oakhill & yuill, 2000) and numbers (yuill et al., 1989). other studies have compared listening and counting span (siegel & ryan, 1989; leather & henry, 1994). compared with the work on adults, however, there has been little research into domain-specificity of the relation between working memory and reading comprehension in children, and in particular the possible role of spatial working memory in children’s comprehension. swanson (1992; 1996) argued for a general resources model, based on similar correlations between verbal and spatial working memory tasks and comprehension skill. however, this argument is not compelling and, indeed, other work by swanson has produced less clear-cut results: swanson and berninger (1995) showed that, even with similar overall correlations between visuo-spatial working memory and comprehension skill, and verbal working memory and comprehension skill, verbal, but not visuo-spatial, working memory differentiated between groups of good and poor comprehenders. thus, the issue of whether international electronic journal of elementary education 86 skilled reading comprehension in children is associated with general working memory remains equivocal, and will be taken up in the present study. bayliss and colleagues (bayliss, jarrold, gunn & baddeley, 2003; bayliss, jarrold, baddeley, gunn, & leigh, 2005) also explored the relation between working memory and reading comprehension in children, using a sentence comprehension test (the nfer-nelson group reading test ii, 1998). they found moderate correlations between reading and both verbal and visuo-spatial span tasks (though not with a purely visuo-spatial task in their 2003 study). however, the reading comprehension measure was almost certainly confounded with word reading skills, which were not independently measured or controlled for. indeed, in both studies, digit span was also correlated with the assessment of reading. this fact strongly suggests that the reading comprehension test was also assessing word reading which, unlike comprehension, tends to be associated with digit span. an important, and novel, issue addressed in the present study is whether any of the working memory tasks are related to reading accuracy, as opposed to reading comprehension. we know of only two previous studies that explored the relation between reading comprehension and working memory in which assessments of reading comprehension skill were distinct from those of single word reading or decoding skills. seigneuric, et al. (2000) developed a test of spatial working memory: a simplified version of the tic-tac-toe task used by daneman and tardif (1987). they found that measures of working memory capacity – both verbal and numerical-predicted reading comprehension over and above vocabulary and decoding skills, but the spatial working memory task was not significantly related to comprehension skill. the present study builds on that of seigneuric et al. in two important ways. first, we explore these relations over a wider age range and with more participants and, second, we control for general ability in each of the domains of interest. the second study was conducted by nation, adams, bowyer-crain and snowling (1999). the findings from their first experiment support those from previous studies (oakhill et al., 1986; stothard & hulme, 1992) in showing that good and poor comprehenders do not differ in digit span and verbatim recall. also in keeping with previous studies, nation et al. found that good and poor comprehenders differed in verbal, but not in spatial, working memory and they argue that the poor comprehenders have specific problems with verbal processing and not more general capacity limitations. however, although nation et al. collected data on the children’s word reading accuracy, they did not look at the relation between working memory and word reading. the present study differs from theirs in various ways. first, we consider a wider age range, and include tasks of three different types (verbal, numerical and spatial). second, nation et al. compared the performance of groups who differed in comprehension ability on their working memory tasks, whereas we look at the relative contributions of working memory tasks in different domains, once general ability is controlled for. in the present study, we compared the relations of the various working memory tests to both accuracy and comprehension. we predicted that working memory would be more closely related to comprehension than to accuracy, and that it would predict variance in comprehension even when accuracy was controlled for. a further issue addressed by our study is whether working memory systems become more differentiated with age. contrasting views have been expressed by alloway, gathercole and pickering (2006) and hale, bronik and fry (1997), based, on the one hand, on correlational data and, on the other hand, on cross-task interference. it is possible that the relation between working memory and reading comprehension differs between children and adults. in particular, kennedy and murray (see, e.g. kennedy, 1987; murray & kennedy, the differential relations between verbal, numerical / oakhill, yuill & garnham 87 1988) have suggested that spatial working memory is important for place-keeping skills in text comprehension, which might develop, at least partially, separately from other aspects of comprehension. these place-keeping skills of fluent readers allow them to re-inspect text selectively, and children who are good readers are much better at re-inspecting text selectively than poor readers (see cataldo & oakhill, 2000). although spatial working memory does not predict text comprehension in adults, it might predict comprehension in children when these skills that depend on spatial working memory are developing, since the demands on the relevant memory systems may be higher. in this study, we assess the role of spatial and other measures of working memory in the reading comprehension performance of 6to 8and 9to 11-year-olds separately. in addition to the problems that we mentioned above, there is a more general problem of interpreting the relation between tests of working memory and assessments of reading. working memory tests in any modality inevitably require some basic abilities in that domain. for example, reading and listening span tasks require general vocabulary knowledge. we might expect tests of verbal working memory to correlate better with measures of reading than tests of numerical working memory because of this shared dependence on general verbal ability. of the previous studies, yuill et al. (1989) used contrasting groups matched on basic vocabulary skills, but only seigneuric et al. (2000) and swanson (1992) have directly assessed this possibility by controlling for vocabulary or other general skills. of course, tests of general ability are also likely to require some degree of working memory skills, but it would be an important indicator of the importance of working memory in reading if correlations with reading skills remained significant after performance on tests of general ability had been partialled out. in the present study, we include assessments of general ability in the three areas of interest: verbal, numerical and spatial, so that the particular working memory tasks can be assessed against the contribution of general ability in the relevant domain. another approach to this confounding was adopted by yuill, oakhill and parkin (1989) who developed a working memory test that required processing and storage of numbers rather than words and sentences. they found a significant correlation between performance on this test and reading comprehension in 7to 9-year-old children, but they did not directly compare the predictive power of their numerical task with that of a listening or reading span task, which is one of the aims of the present study. a subsidiary question is how the level of verbal complexity of a working memory task contributes to the relation between that task and reading comprehension. reading span tasks are complex, in that they require the simultaneous use of several skills: not only the verbal encoding of information and switching between storage and processing, but also the syntactic and semantic processing of sentences and the processing of word meanings (although the complexity of the processing component does not seem to relate very directly to the predictive power of the task: see lépine, barrouillet & camos, 2005). in the present study, we developed and validated a verbal measure for use with children that does not require sentence-level comprehension, and compared its predictive power with that of the listening span task. in addition, the inclusion of these two verbal tasks enabled us to explore how tasks with different processing components relate to comprehension skill. we also included two different numerical tasks for similar reasons, but also because the final digit task we have used previously (see yuill, et al., 1989), is not so strongly related as to comprehension skills as are verbal tasks (seigneuric et al., 2000). one possible explanation for this weaker relation is that the processing requirement of the final digit task is low (children simply have to read out a set of single-digit numbers). in the present study we therefore also included a second test of numerical working memory, with a more demanding international electronic journal of elementary education 88 processing requirement. we used only one spatial task, a version of that used successfully in the study by seigneuric et al. kane et al. (2004) point out some major limitations in previous (adult) studies that purport to show domain-specificity in working memory tasks, and we have attempted to overcome these criticisms in the present study, as follows. first, many studies have used small and quite homogeneous samples. we have used a large sample of children, across a wide age range. second, in some studies, the verbal and spatial working memory tasks differ markedly in difficulty. we have piloted and developed tasks that were similar in difficulty, and adapted the level of difficulty to give similar levels of average performance in the different age groups. third, it is not clear in previous studies whether it is the domain-specificity of the working memory construct that is important, or the domain-specificity of resources in that domain that are not specific to working memory tasks: for example, if verbal working memory is related to reading comprehension, is that something to do with the working memory task, or with the verbal nature of the task? in contrast to most previous studies of the dissociation between verbal and spatial working memory, we also included general measures of verbal, mathematical and spatial ability, so that the contributions of domainspecific working memory, as opposed to competence in a domain more generally, could be taken into account. in summary, the present study aimed to explore the relation between working memory skills in different domains (verbal, numerical and spatial) and reading comprehension in children. this work extends previous studies in two main ways. first, we include comparisons of all three areas. second, we explore the relation between working memory and reading skill when general ability in the relevant domain has been taken into account. third, we explore whether any links between working memory and reading ability are specific to reading comprehension, or apply to reading skill more generally. finally, we explore the way in which the level of complexity of the verbal and numerical tasks contributes to the relation between that task and reading skill. methods participants all available children (excluding those few identified by teachers as having language or behavioural problems) from 12 classes of 6 to 11-year-olds in 5 schools took part in the study. this produced a sample of 197 children, divided into two age groups: 97 6to 8year-olds and 100 9to 11-year-olds. overview of design and procedure each child completed 3 types of test, described in detail in the materials section: (1) reading tests (ra: accuracy and rc: comprehension tests of the neale analysis of reading ability revised (neale, 1997), (2) working memory tests of three types: verbal (2 tests, odd word out, vwm1, and reading span, vwm2), numerical (2 tests, highest number, nwm1, and final number, nwm2) and spatial (1 test, spwm) and (3) general ability tests selected from the cognitive abilities test (cat: thorndike & hagen, 1986) in three areas: verbal, numerical and spatial (nonverbal). the reading test was presented individually in the first session, followed by the 5 working memory tests, in a separate random order for each subject, spread over 3 sessions, each in a separate room by a male experimenter familiar to the children. finally the cats were presented to children in separate random orders in groups of 36. materials the differential relations between verbal, numerical / oakhill, yuill & garnham 89 reading test. the neale analysis requires children to read aloud a series of narrative passages of increasing difficulty, and then to answer from memory a mix of factual and inferential questions about each passage. children are corrected on words that are misread or not read, so that they are not disadvantaged on comprehension questions, but testing is stopped when children make a pre-set number of reading errors on a particular passage. separate norm-referenced scores are computed for accuracy (number of words read correctly) and comprehension (number of questions correct). this test, and its predecessor, have been shown to predict a range of differences in abilities between good and poor comprehenders (e.g. see yuill & oakhill, 1991). we used the raw scores for reading accuracy (ra) and reading comprehension (rc) in all analyses. working memory tests. all five of these tests had certain characteristics in common. all required the simultaneous storage and processing of information. for each of the tests, there were four levels of storage difficulty presented in order of increasing difficulty, each level containing three trials (except the pre-existing final number task devised by yuill et al., which was the least demanding in terms of storage and processing, and had eight trials). the first storage level contained two recall items and for each of the next levels the number of recall items was increased by one, with the final storage level having a maximum of five recall items. where appropriate (in tests v1, v2, n1 and sp) the position of correct responses was counterbalanced. children practised at each storage level until it was clear they understood the processing requirements of the task. this never required more than three trials at any of the storage levels. in all of the tests a strict scoring procedure was used: children were required to recall the correct items in the order of presentation. in all analyses, we used the proportion of items correct out of the possible maximum score as the independent variable. we piloted the materials for the new or adapted tests (v1, v2, n1 and sp) with 40 6to 11year-olds, in order to ensure that children could provide the correct responses for the processing component, and only used items on which 90% of the youngest children were correct. (recall of these correct responses was, naturally, considerably less than 90%.) the second aim of the pilot was to ensure that the tests produced similar mean scores and standard deviations both across the different modalities and across the age range. this aim was achieved, both in the pilot and in the main study, as shown in table 1. in all the working memory tests, the older children were presented with more trials in order to ensure that the tasks were at an appropriate level of difficulty. the younger age groups received three trials at each of three levels (two, three or four items to recall) and the older children received an additional set of trials with five items to recall. thus, the younger children were given a total of nine trials (3 trials at each of 3 levels of difficulty) and the older children received a total of 12 trials (3 trials at each of 4 levels of difficulty). all children attempted all trials appropriate for their age group. because different children received different numbers of trials, the scores entered into the analyses were proportions correct. the exception was the spatial working memory task, which proved to be sufficiently difficult for all children with a recall demand of four. aural word span: odd word out (vwm1). this newly-devised test consists of series of single words of one or two syllables in groups of four. three of the words are in the same category (e.g. names of fruits or colours) and the fourth is from a different category. the words within each group of four are presented in a fixed random order. children listen to the four words and have to detect the 'odd word out'. they then have to recall the odd words in each series. international electronic journal of elementary education 90 an example of a three-item series is: whale shark dolphin scarf cowboy curtain indian sheriff egg aunt cousin uncle the correct response for this series is: “scarf, curtain, egg”. aural reading span (vwm2). this test is our uk english adaptation of the test used by siegel and ryan (1989). we adapted the test because we found that english children in some cases did not give the same completions as the original north american sample did. the child listens to a set of unrelated sentences and supplies the final word in each, then recalls these final words. the final words are highly constrained by the context. for example, the sun shines during the day, the moon at _____. at the library people read ______. an apple is red, a banana is ______. the correct response is: “night, books, yellow”. highest number task (nwm1). in this new test, children inspect sets of three numbers, shown on a card and read aloud by the experimenter. they have to pick the highest number and then recall the highest numbers from each set. all the numbers are between 1 and 19 and each set contains one number below 10, and two between 10 and 20. for example, a 3-item set is: 14 9 17 10 11 4 15 3 12 to which the answer is: “17, 11, 15”. final number test (nwm2). this task was the one developed by yuill et al. (1989). children are required to read sets of three-digit numbers and to recall the last numeral in each number. for example, a three-item set is: 528 434 489 to which the answer is: “8, 4, 9”. it should be noted that, though we refer to the two above tests as tests of “numerical” working memory, they might better be described as tasks that require numerical processing, but verbal storage. we return to this point in the discussion section. spatial working memory test (spwm). daneman and tardif (1987) described a spatial test using three-dimensional tic-tac-toe. we adapted this test to make it suitable for children. children were shown a series of 3 by 3 matrices, one at a time, each containing two noughts, and had to point to the cell where a nought should be inserted to make a winning line. each grid has noughts of a different colour, in order to facilitate children's recall of the positions in the correct order. after seeing all the matrices, children had to place strips of corresponding colours onto an adhesive grid, to indicate the positions of the winning lines. the colours of the lines could thus be linked to the order in which the grids had been presented: the sequence of colours was the same in each trial (e.g. the trials of 3 items showed orange, then the differential relations between verbal, numerical / oakhill, yuill & garnham 91 blue, then green noughts, and the trials of 4 items added a set of pink noughts to this sequence). although the probability of guessing the correct lines is 1 in 8 (as there are only 8 possible winning lines in a 3 by 3 grid), the probability of matching colours to positions is much lower. cognitive abilities tests (cats). these are a series of standardised pencil-and-paper multiplechoice tests tapping general ability in different areas. the full cat consists of four verbal subtests, three mathematical (‘quantitative’) subtests and three spatial ('nonverbal') subtests. we used one of each type of subtest: verbal 2, quantitative 1 and nonverbal 2, all preceded by two to three practice items. these subtests were chosen on the basis that each correlated most highly with the other subtests in its battery and was most highly loaded on the relevant factor in a factor analysis (thorndike, hagen & france, 1986). there are two levels of each test: level a consists of the first 25 questions in the test, which increase in difficulty through the test, and level b consists of the first 30 questions of the same test. all children took level b, except for the 7-year-old group, who were given level a. an example from each of these tests is given below. verbal (catv): the fire is ....... (possible choices: wet, green, hot, running, round) quantitative (catn): which is greater: 1 1 + 1 1 or 1 1 + 1 ? spatial (cats): respondents have to make analogies between diagrams, for example: large square is to small square as large circle is to ....?, with choices of a small circle, a large semicircle, a filled square and two triangles joined at the apices. results descriptive statistics each child had a maximum of 10 test scores (though because of time constraints and absences, it was not always possible to collect the full set of data for every child; where data were missing, correlations and regression analyses were conducted using all available data: the actual numbers of children included in the correlation analyses is shown in the relevant tables): two reading scores, neale reading accuracy (nra) and neale reading comprehension (nrc), five working memory scores, (verbal, odd word; verbal reading span; numerical highest number; numerical final digit and spatial) which were calculated as proportion of total possible score, and three general ability scores (cat verbal, cat numerical and cat spatial). we also calculated chronological age at time of test in months (ca). the means and standard deviations of each score for the two age groups are shown in table 1. the working memory tests, including the new ones, showed a reasonable spread of performance (all means within the range 35-59% correct) and similar levels of variability (alls s.d.s in the range .12 to .16). importantly, the tests that turned out to be the strongest predictors of comprehension skill were not differentiated from the other tests by their particular ease or difficulty. in te rn a ti o n a l e le c tr o n ic j o u rn a l o f e le m e n ta ry e d u c a ti o n 9 2 t a b le 1 . s u m m a ry o f m e a n s a n d s ta n d a rd d e v ia ti o n s o f o v e ra ll s co re s o n t h e a ss e ss m e n ts , a n d w it h in e a c h a g e -g ro u p . a g e g ro u p : a g e n e a le a c c u ra c y n e a le c o m p re h . c a t : v e rb a l c a t : n u m e ri c a l c a t : sp a ti a l v w m : o d d w o rd v w m : r e a d in g sp a n n w m : h ig h e st n u m b e r n w m : fi n a l d ig it s p a ti a l w m 6 -8 y e a rs m e a n 9 4 .8 4 3 6 .7 0 1 2 .3 8 1 2 .0 7 1 1 .1 8 1 2 .7 3 .3 5 .5 2 .4 4 .4 8 .4 4 n 8 3 8 4 8 4 8 2 8 2 8 1 7 8 8 1 7 7 8 0 7 5 s. d . 6 .9 8 1 7 .6 4 5 .3 2 5 .7 2 5 .7 1 8 .2 9 .1 3 .1 4 .1 4 .1 4 .1 4 9 -1 1 y e a rs m e a n 1 2 1 .1 0 6 2 .3 6 2 0 .9 5 2 1 .2 0 2 1 .0 5 2 0 .8 4 .4 6 .5 9 .5 3 .5 5 .5 3 n 9 5 9 5 9 5 9 5 9 4 9 5 6 6 8 1 6 5 7 7 8 3 s. d . 7 .5 3 2 1 .5 9 7 .7 3 5 .6 8 5 .4 4 8 .9 2 .1 2 .1 2 .1 6 .1 4 .1 5 o v e ra ll m e a n 1 0 8 .8 5 5 0 .3 2 1 6 .9 3 1 6 .9 7 1 6 .4 5 1 7 .1 1 .4 0 .5 5 .4 8 .5 1 .4 9 n 1 7 8 1 7 9 1 7 9 1 7 7 1 7 6 1 7 6 1 4 4 1 6 2 1 4 2 1 5 7 1 5 8 s. d . 1 4 .7 3 2 3 .4 5 8 .1 9 7 .0 3 7 .3 6 8 .5 4 .1 2 .1 4 .1 6 .1 4 .1 5 n o te : c h ro n o lo g ic a l a g e is in m o n th s; n e a le a n d c a t s c o re s a re r a w s c o re s; w o rk in g m e m o ry t a sk s a re p ro p o rt io n c o rr e ct . the differential relations between verbal, numerical / oakhill, yuill & garnham 93 the reliability of the different working memory measures was calculated using cronbach’s alpha. it will be recalled that older children were required to complete more trials to ensure that the tasks were sufficiently difficult for them., so the total number of trials was 12 for the older children and 9 for the younger ones. however, estimates of reliability could be obtained only over the items that were completed by all the participants (n = 9 items) and, given the small number of items, were acceptable. the levels of cronbach’s alpha ranged between .66 and .73 for the verbal and numerical tasks, but were slightly lower for the spatial working memory task (.61). it was not appropriate or necessary to calculate reliability in the case of the neale analysis scores or the cats scores, since these are standardised tests with published reliability statistics. correlational analyses the bi-variate correlations between the measures are shown overall, and separately for each age group, in table 2. because of the large number of correlations, we adopted a conservative (.01) level of significance. all five working memory measures were significantly correlated with both reading comprehension (correlations between .34 and .46) and reading accuracy (correlations between .36 and .47), and all the working memory measures were significantly correlated with each other (correlations between .34 and .59). some of these correlations held up in both age groups separately, but it was only the verbal reading span and numerical final digit task that were strong and consistently related to reading comprehension in both age groups. we return to this point in the regression analyses, presented below. in addition, the general ability measures (cats) were significantly correlated with both the working memory measures and the reading measures in the data overall, and the cats scores were correlated with the two reading ability measures in both age groups, with the exception of the cat spatial which did not correlate with rc in the younger group. the relation between the cats scores and the working memory assessments in the two age groups considered separately were less consistent. in te rn a ti o n a l e le c tr o n ic j o u rn a l o f e le m e n ta ry e d u c a ti o n 9 4 t a b le 2 . in te rc o rr e la ti o n s a m o n g a g e a n d a b il it y v a ri a b le s a g e n e a le a c c u ra c y n e a le c o m p re h e n si o n v w m : o d d w o rd v w m : re a d in g sp a n n w m : h ig h e st n u m b e r n w m : fi n a l d ig it s p a ti a l w m : c a t : v e rb a l c a t : n u m e ri c a l c a t : s p a ti a l a g e 1 .6 0 2 ** .5 8 1 ** .4 2 7 ** .2 5 0 ** .3 3 4 ** .2 1 1 ** .4 1 3 ** .7 1 6 ** .7 3 4 ** .4 8 3 ** n 1 7 8 1 7 8 1 4 3 1 6 2 1 4 1 1 5 7 1 5 8 1 7 6 1 7 5 1 7 5 n e a le r a 1 .7 1 3 ** .4 7 0 ** .3 7 4 ** .4 2 1 ** .4 0 0 ** .3 5 9 ** .8 1 3 ** .7 1 1 ** .5 7 8 ** n 1 7 9 1 4 4 1 6 2 1 4 2 1 5 7 1 5 8 1 7 7 1 7 6 1 7 6 n e a le r c 1 .4 6 4 ** .4 6 4 ** .3 4 5 ** .4 4 0 ** .3 4 4 ** .7 1 5 ** .6 1 2 ** .5 2 0 ** n 1 4 4 1 6 2 1 4 2 1 5 7 1 5 8 1 7 7 1 7 6 1 7 6 v w m 1 1 .5 9 0 ** .5 1 5 ** .5 0 6 ** .4 7 0 ** .4 9 2 ** .4 6 4 ** .4 8 8 ** n 1 3 3 1 3 9 1 3 1 1 2 9 1 4 2 1 4 1 1 4 1 v w m 2 1 .4 7 2 ** .5 4 2 ** .3 3 9 ** .3 7 7 ** .3 6 9 ** .3 4 4 ** n 1 3 1 1 5 7 1 4 6 1 6 0 1 5 9 1 5 9 n w m 1 1 .5 5 9 ** .4 8 3 ** .3 9 4 ** .5 1 1 ** .3 8 9 ** n 1 2 7 1 2 6 1 4 0 1 4 0 1 3 9 n w m 2 1 .3 5 9 ** .3 6 1 ** .4 6 3 ** .3 4 4 ** n 1 4 1 1 5 5 1 5 4 1 5 4 s p w m 1 .3 8 4 ** .4 5 4 ** .4 3 9 ** n 1 5 6 1 5 5 1 5 5 c a t -v 1 .7 7 2 ** .6 6 6 ** n 1 7 6 1 7 6 c a t -n 1 .5 9 2 ** n 1 7 5 ** p < 0 .0 0 1 * p < 0 .0 1 t h e d if fe re n ti a l r e la ti o n s b e tw e e n v e rb a l, n u m e ri c a l / o a k h il l, y u il l & g a rn h a m 9 5 t a b le 2 a . in te rc o rr e la ti o n s a m o n g v a ri a b le s w it h in t h e y o u n g e r a g e g ro u p a g e n e a le a c c u ra c y n e a le c o m p re h . v w m : o d d w o rd v w m : re a d in g sp a n n w m : h ig h e st n u m b e r n w m : fi n a l d ig it s p a ti a l w m c a t : v e rb a l c a t : n u m e ri c a l c a t : s p a ti a l a g e 1 .3 0 0 * .2 1 5 .2 7 1 .2 6 7 .2 1 0 .0 5 6 .3 2 8 * .5 8 0 ** .4 0 8 ** .2 1 6 n 8 3 8 3 7 7 8 1 7 6 8 0 7 5 8 1 8 1 8 0 n e a le r a 1 .6 5 2 ** .3 8 5 ** .4 5 8 ** .2 2 8 .4 5 6 ** .2 7 3 .7 3 9 ** .5 2 3 ** .2 9 9 * n 8 4 7 8 8 1 7 7 8 0 7 5 8 2 8 2 8 1 n e a le r c 1 .4 2 8 ** .4 6 6 ** .2 8 0 .4 0 6 ** .1 8 4 .5 8 0 ** .3 4 8 * .2 4 4 n 7 8 8 1 7 7 8 0 7 5 8 2 8 2 8 1 v w m 1 1 .6 1 3 ** .5 0 3 ** .4 5 1 ** .5 0 0 ** .4 3 3 ** .3 1 5 * .4 9 1 ** n 7 5 7 7 7 4 7 1 7 6 7 6 7 5 v w m 2 1 .5 0 1 ** .4 2 7 ** .3 9 1 ** .4 0 3 ** .3 4 4 * .3 0 8 * n 7 4 8 0 7 4 7 9 7 9 7 8 n w m 1 1 .3 7 2 ** .3 9 8 ** .2 4 1 .3 6 9 ** .2 9 0 n 7 3 7 0 7 5 7 5 7 4 n w m 2 1 .3 7 2 ** .3 9 8 ** .2 4 1 .3 6 9 ** .2 9 0 n 7 3 7 0 7 5 7 5 7 4 s p w m 1 .2 4 9 .2 9 1 .2 8 5 n 7 3 7 3 7 2 c a t -v 1 .5 8 5 ** .4 6 6 ** n 8 2 8 1 c a t -n 1 .3 5 5 ** n 8 1 ** p < 0 .0 0 1 * p < 0 .0 1 in te rn a ti o n a l e le c tr o n ic j o u rn a l o f e le m e n ta ry e d u c a ti o n 9 6 t a b le 2 b . i n te rc o rr e la ti o n s a m o n g v a ri a b le s w it h in t h e o ld e r a g e g ro u p a g e n e a le a c c u ra c y n e a le c o m p re h . v w m : o d d w o rd v w m : re a d in g sp a n n w m : h ig h e st n u m b e r n w m : fi n a l d ig it s p a ti a l w m c a t : v e rb a l c a t : n u m e ri c a l c a t : s p a ti a l a g e 1 .3 1 4 * .3 1 2 * -. 0 3 0 -. 1 7 6 .1 1 9 -. 0 6 5 .3 3 7 * .3 3 9 ** .4 4 1 ** .2 6 8 * n 9 5 9 5 6 6 8 1 6 5 7 7 8 3 9 5 9 4 9 5 n e a le r a 1 .5 6 7 ** .2 9 5 .1 3 2 .4 1 0 ** .2 2 4 .2 3 3 .7 1 5 ** .5 8 6 ** .5 6 0 ** n 9 5 6 6 8 1 6 5 7 7 8 3 9 5 9 4 9 5 n e a le r c 1 .2 4 2 .3 6 9 ** .2 0 0 .3 6 8 ** .2 6 2 .5 8 7 ** .4 5 3 ** .4 6 4 ** n 6 6 8 1 6 5 7 7 8 3 9 5 9 4 9 5 v w m 1 1 .4 4 6 ** .3 9 9 ** .4 1 3 ** .3 0 4 .2 1 6 .2 6 1 .2 8 4 n 5 8 6 2 5 7 5 8 6 6 6 5 6 6 v w m 2 1 .3 5 3 * .6 1 5 ** .1 8 1 .1 5 9 .2 0 0 .2 3 5 n 5 7 7 7 7 2 8 1 8 0 8 1 n w m 1 1 .6 8 8 ** .4 8 1 ** .3 3 1 * .5 0 5 ** .3 2 9 * n 5 4 5 6 6 5 6 5 6 5 n w m 2 1 .3 3 2 * .2 7 3 .4 2 7 ** .2 1 2 n 6 8 7 7 7 6 7 7 s p w m 1 .2 8 9 * .4 3 0 ** .4 1 9 ** n 8 3 8 2 8 3 c a t -v 1 .6 3 7 ** .6 4 7 ** n 9 4 9 5 c a t -n 1 .5 3 7 ** n 9 5 ** p < 0 .0 0 1 * p < 0 .0 1 international electronic journal of elementary education 97 since our prediction in relation to age differences was not upheld (that there may be a different relation between comprehension skill and visuo-spatial working memory in the two age groups) we conducted all further analyses on the entire data set. regression analyses the regression analyses enabled us to assess the relative importance and specificity of the various predictors in relation to the measures of reading ability. we were particularly interested in comparing the predictive power of the numerical, verbal and spatial tasks. the first goal was to determine whether working memory was a predictor of reading comprehension when age and general ability in the relevant domain were controlled. a first set of stepwise hierarchical multiple regression analyses were conducted with comprehension as the independent variable, in which the different working memory measures were entered at the final step. in each analysis, three variables were entered: age, performance on the relevant cognitive abilities test (verbal, numerical or spatial, depending on which working memory task was entered) and one of the working memory tasks. thus, five different models were tested – each with a different working memory measure. in all of these analyses, age and the relevant cat were highly significant predictors of comprehension skill. however, the results showed that three of the four verbal and numerical working memory tests (but not the spatial test) accounted for variance in comprehension skill over and above that accounted for by age and the relevant cat score. the reading span task and the final digit task were the strongest predictors of comprehension skill, and the other verbal task (odd word out) was only marginally predictive. the results of these regression analyses are shown in table 3. table 3: stepwise multiple regression analyses predicting reading comprehension. table 3a: vwm1 (odd word) entered in final position independent variable r square r square change f change d.f. sig. f change 1. age .306 .306 61.222 1,139 .001 2. cat-v .534 .228 67.589 1,138 .001 3. vwm 1 .546 .012 3.684 1,137 .057 table 3b: vwm2 (reading span) entered in final position independent variable r square r square change f change d.f. sig. f change 1. age .327 .327 76.710 1,158 .001 2. cat-v .506 .180 57.128 1,157 .001 3. vwm 2 .553 .047 16.371 1,156 .001 international electronic journal of elementary education 98 table 3c: nwm1 (highest number) entered in final position independent variable r square r square change f change d.f. sig. f change 1. age .316 .316 63.342 1,137 .001 2. cat-n .397 .081 18.277 1,136 .001 3. nwm 1 .399 .002 .486 1,135 .487 table 3d: nwm2 (final digit) entered in final position independent variable r square r square change f change d.f. sig. f change 1. age .320 .320 71.644 1,152 .001 2. cat-n .389 .068 16.854 1,151 .001 3. nwm 2 .444 .056 15.007 1,150 .001 table 3e: spatial working memory span entered in final position independent variable r square r square change f change d.f. sig. f change 1. age .332 .332 76.172 1,153 .001 2. cat-sp .421 .089 23.375 1,152 .001 3. spwm .422 .001 .250 1,151 .618 thus far, the results closely parallel those of seigneuric et al. but also go beyond them in important ways, in that we control for measures of general ability in the relevant domain, whereas they did not. that is, even after performance on the relevant assessment of general ability measure had been entered, both verbal working memory measures, and the numerical final digit measure accounted for significant variance in comprehension skill. it is particularly impressive that the tests of verbal working memory accounted for variance in comprehension skill over and above the contribution of general verbal ability, since that variable alone accounted for around 20% of unique variance in comprehension skill. since the reading scores were highly correlated in this sample (r = .71) and each of the working memory tasks was correlated with accuracy in the sample overall (all rs ≥ .36) it is important to establish whether working memory is a predictor of comprehension skill specifically, or reading more generally. in order to do this, we conducted a parallel set of regression analyses to those above, with reading accuracy as the dependent variable. after controlling for chronological age and the relevant measure of general ability, only one of the the differential relations between verbal, numerical / oakhill, yuill & garnham 99 working memory measures predicted significant variance in word reading. that was the final digit task, which accounted for 1.6% of variance in accuracy (compared with 5.6% in comprehension). because at least one of the working memory tasks was related to reading accuracy, over and above the effects of age and the general ability measure, we re-ran the regression analyses in which comprehension was the dependent variable, but controlled for reading accuracy as well as chronological age and the relevant general ability measure. despite this very strong test of the predictive power of the measures of working memory, the verbal (reading span) and numerical (final digit) working memory measures continued to predict variance in comprehension skill. this first set of analyses enables us to provide a clear answer to the first question, which is whether working memory predicts comprehension skill in children over and above measures of general ability. the answer is that three of the four verbal and numerical working memory measures account for significant (or marginally significant) variance in comprehension skill, over and above the effects of age and a relevant general ability measure. it replicates and extends seigneuric et al.’s (2000) finding that spatial working memory was not related to comprehension skill. the second question concerns the nature of the working memory resources involved in reading comprehension. we wanted to determine whether the working memory system that is related to reading comprehension in children is a general system, or a symbolic system specialised for language processes. the results of the previous analyses provide some indications. as we saw above, only the verbal working memory tasks and one of the numerical tasks were significantly related to comprehension skill in both age groups once age and general ability had been partialled out, whereas the spatial working memory task was not related to comprehension skill over and above age and general ability. thus, these analyses seem to support the "symbolic resource model" that we describe in the introduction. in order to test this hypothesis more directly, we need to assess whether the verbal and numerical tasks draw on the same pool of symbolic resources to predict reading comprehension. therefore we carried out a further set of analyses. in these analyses, performance on the stronger verbal working memory task (reading span) and the stronger numerical working memory task (final digit) was compared. the variables were again entered in a fixed order, and the order of entry of the verbal and the numerical working memory measures was reversed in order to assess the shared and the unique variance explained by each measure. support for the symbolic system hypothesis would come from results showing the contribution of a verbal or numerical task to be substantially reduced when the effect of the other (numerical or verbal) task was previously entered into the regression equation. in these analyses, the spatial task was not considered further since it did not account for significant variance in reading comprehension, over and above age and a general measure of spatial ability. because of the wide age range, and the general improvement with age on the working memory tasks, age was entered first in the regression analyses. it was not obvious which of the general ability measures (verbal or numerical) to enter in these analyses, but in fact the results showed an identical pattern whichever was used. the results including the verbal cat data are presented. the results of these regression analyses are shown in table 4. these analyses indicate that the verbal and numerical measures contributed independently to variance in comprehension skill, even after controlling for domain-relevant ability. we note here that this result is contrary to that obtained by seigneuric et al., who found that neither of their numerical tasks explained variance over and above that contributed by one of the verbal measures. however, our data are probably more reliable since the sample was much larger international electronic journal of elementary education 100 (in these particular analyses 153 participants were entered as opposed to only 48 in seigneuric et al.’s study). despite the apparent difference in conclusion, however, we found that whichever working memory assessment was entered last contributed a very small percentage of additional variance (between 1.5 and 1.7%, see table 4) though, of course, the preceding variables had already taken up about 55% of the variance. thus, although the contribution of whichever task is entered at the final step is significant, there is also a very substantial amount of shared variance between the verbal and numerical working memory tasks. table 4: fixed order regressions with the verbal (reading span) and a numerical (final digit) working memory measure as predictors; reading comprehension as the dependent variable independent variable r square r square change f change d.f. sig. f change 1. age .321 .321 72.355 1,153 .001 2. cat-v .505 .184 56.505 1,152 .001 3. vwm 2 .550 .045 15.148 1,151 .001 4. nwm 2 .565 .015 5.109 1,150 .025 3. nwm 2 .548 .043 14.309 1,151 .001 4. vwm 2 .565 .017 5.896 1,150 .016 a subsidiary question, which we addressed in a further analysis, was the way in which the level of verbal complexity of a working memory task contributes to the relation between that task and reading comprehension. reading span requires not only switching between storage and processing, verbal encoding of information and phonological storage, but also requires the syntactic and semantic processing of sentences and the processing of word meanings. the odd-word-out task was designed to tap the same processes, except that it did not require sentence processing. thus, if it is the shared verbal component of the tasks that is important in predicting comprehension, then we might expect that the reading span task would not predict additional variance over and above that predicted by the odd-word-out task. however, if the sentence-level processing is an important additional aspect of the predictive power of the reading span task, over and above the more general verbal component, then we might expect that it would account for additional variance even after performance on the odd-word-out task is controlled for. in the analyses in which we compared the predictive power of the two verbal tasks, we found the latter pattern of results (see table 5). as can be seen, the reading span task accounted for an additional, highly significant, 3% of variance even when entered last, whereas the word span task, when entered last, did not account for significant additional variance the differential relations between verbal, numerical / oakhill, yuill & garnham 101 table 5: fixed order regression analyses to compare the predictive power of the two verbal working memory tests (reading span and odd word out) independent variable r square r square change f change d.f. sig. f change 1. age .312 .312 58.592 1,129 .001 2. cat-v .535 .223 61.372 1,128 .001 3. vwm 1 .552 .017 4.740 1,127 .031 4. vwm 2 .580 .028 8.420 1,126 .004 3. vwm 2 .579 .044 14.309 1,127 .001 4. vwm 1 .580 .001 5.896 1,126 .608 overall, these results provide support for the idea that working memory capacity is a strong predictor of reading comprehension in children. both verbal and numerical (but not spatial) working memory tasks contributed substantially to the prediction of reading comprehension even when age and relevant general ability had been taken into account, but there was some indication that they are making independent contributions to this prediction. the results also provide some support for the idea of a symbolic capacity model since the verbal and numerical tasks were far more strongly related to comprehension skill than was performance on the spatial task. discussion for the purposes of this study, we produced working memory tests for children across a wide age range. the tests were calibrated in such a way that they produced similar means and standard deviations from 6 through to 11-year olds. the new tests we have developed (the odd-word out task, the largest number task and the spatial task) and the comparison of these with tests of working memory we have used previously (the reading span and final-number tests) provide useful comparative data, and also provide new evidence on the relation of working memory to children’s reading skills: both comprehension and reading accuracy. an important theoretical aspect of this study is that we have explored the predictive power of the various measures of working memory (verbal, numerical and spatial) once performance on the relevant general ability has been controlled. we found that, even after discounting age and relevant general ability, two of the tests (the reading span and final digit tasks) strongly predicted, and one (the odd-word-out task) marginally predicted, performance on the comprehension assessment. of course, although we refer to two of the tests as “numerical tests”, they do require verbal storage (it is only the processing component that is numerical), so it is not surprising they are related to verbal tests. indeed, the numerical tasks were significantly correlated with the verbal tasks, and more highly than they were with the spatial tasks. the spatial working memory test was moderately correlated with both accuracy and comprehension overall, but not predictive of comprehension once age and spatial ability had been partialled out. these findings are consistent with those of leather and henry, who found a reading span task to be a significant predictor of comprehension in 7-year-olds, while yuill et al. (1989) international electronic journal of elementary education 102 found a high correlation between comprehension skill in 7to 8-year-olds and the same digit working memory test that we used in the present study. our results suggest that, overall, the verbal tasks are better predictors than numerical ones, but that the difference is not striking. indeed, there was little difference in the predictive power of the two stronger numerical and verbal working memory tests: both accounted for about 5% of unique variance in comprehension skill, over and above the contributions of age and the relevant measure of ability in that domain. this pattern of findings suggests that verbal tasks, even ones that require sentence comprehension, are not particularly privileged in their relation to reading comprehension. importantly, it argues against the idea that reading span measures relate to comprehension skill only because they have a comprehension component (see also, lépine, et al, 2005). these conclusions are consistent with the suggestion from previous (adult) studies (e.g. daneman & tardif, 1987) that the working memory system that is implicated in language comprehension is a system specialized for the processing of symbolic (particularly verbal) information, and that spatial working memory does not have a role in text processing. in the present study, we were able to compare the relative predictive power of two verbal working memory tests, which we designed to have different characteristics, crucially their level of verbal complexity. the reading span task requires not only switching between storage and processing, the verbal encoding of information and phonological short-term storage, but also the syntactic and semantic processing of sentences, including word meanings. the odd-word-out task, in contrast, does not require any sentence processing (though it does require knowledge of word meanings and categories since the child has to select and remember the word in each set that comes from a different semantic category). thus, if the sentence-level processing is an important additional aspect of the predictive power of the reading span task, then we might expect that it would account for additional variance even after performance on the other verbal task (the odd-word-out task) was controlled for, and this is what we found. however, as we have argued above, any explanation of the role of wm in comprehension that focuses entirely on the comprehension requirement of the reading span task does not hold up, given that the final digit task was as strong a predictor of reading comprehension. furthermore, a comparison of the two numerical tests showed that the final digit test was a considerably stronger predictor than the highest number task, even though the final digit task had a low processing requirement (reading out digits) and did not make demands on sentence comprehension processes. thus, our more general prediction about the relation between working memory and reading comprehension being dependent on the level of processing difficulty was not supported by the data (see also lépine et al., 2005). the ability of one of the verbal and one of the numerical working memory tests to predict comprehension skill remained strong after tests of general ability had been partialed out. this method of analysis represents a very conservative test of the hypothesised relation between working memory and comprehension, since ability in a particular modality will inevitably influence children's performance on working memory tests in that modality. as we noted previously, tests of general ability, including the cat, can in turn be expected to tap working memory capacity to a limited degree. importantly, the pattern of relations reported above was specific to comprehension and did not apply to reading accuracy. although working memory in each of the different tasks was correlated with reading accuracy, both overall and in each age group separately, most of the predictive power was lost when age and the relevant ability score were first entered into the regression equation. only one of the working memory tasks (the final digit task) the differential relations between verbal, numerical / oakhill, yuill & garnham 103 predicted a very small proportion of variance in word reading accuracy over and above age and the relevant cat. our finding that performance on particular working memory tasks was related specifically to reading comprehension, and not word reading accuracy, is consistent with some previous results (e.g. swanson & jerman, 2007; oakhill, cain & bryant, 2003; yuill et al., 1989). swanson and colleagues have demonstrated that reading comprehension and growth in reading comprehension are best predicted by executive function tasks (i.e. working memory tasks), rather than short-term memory tasks involving phonological coding, which are more likely to be related to word recognition skills. although spatial working memory and reading comprehension were correlated, spatial working memory was not predictive of reading comprehension once age and spatial ability had been partialled out. consistent with the results of nation et al., (1999) and seigneuric et al. (2000), there is no evidence, even within this wide age group, that the spatial working memory system plays a role in comprehension processes, and neither is it related more strongly to comprehension skill in younger than in older children, as we hypothesized it might be. although we used the same visuo-spatial task as seigneuric et al., this was different to the task used by nation et al., thus demonstrating that the pattern of results in children generalizes over more than one task. we have suggested that the different predictive power of the spatial vs. the numerical and verbal working memory tests might be explained in terms of the verbal and numerical tests requiring processing of symbolic information (letters and numbers). however, an alternative explanation of the difference, which might be considered in further work, is that the verbal and numerical tasks, unlike the spatial task, depend on retrieval of information from long-term memory (which is, of course, also a characteristic of skilled reading comprehension. we also suggested, following kennedy and murray, that spatial working memory skills might be specifically related to the skill of place keeping in comprehension. however, it could be that the comprehension assessment is not sufficiently demanding of these skills. in any case, the time course of typical uses of place holding skills in reading is perhaps beyond the bounds of what is traditionally thought of as working memory processes. thus, such skills might be more closely linked to spatial ability, rather than working memory per se, which is consistent with our finding that general spatial ability was a better predictor of reading comprehension in our sample than was the measure of spatial working memory. as outlined in the introduction, working memory skills are likely to be important in reading (and listening) comprehension (in both children and adults). the processes of integration and inference are important to the construction of an integrated and coherent model of a text (i.e. a mental model or a situation model: gernsbacher, 1990; johnson-laird, 1983; kintsch, 1998), and these processes require that the relevant information, either from the text or world knowledge, is both available and accessible. working memory is proposed to serve as a buffer for the most recently read propositions in a text, enabling their integration to establish coherence, and for information retrieved from long-term memory to enable its integration with the currently active text (see e.g. cooke, et al., 1998; graesser, et al., 1994). this study, along with several previous studies, demonstrates a strong relation between memory and children’s reading comprehension. the majority of this work suggests that the relation between memory and reading comprehension is specific to working memory tasks that require the simultaneous storage and processing of symbolic information (both verbal and numerical), rather than memory tasks that simply assess the passive storage of such information (e.g. leather & henry, 1994; oakhill et al., 1986; swanson & berninger, 1995; yuill et al., 1989). furthermore, and consistent with the present findings, the working memory resources that are related to reading comprehension appear to be specialised for language international electronic journal of elementary education 104 processing: tasks that require the manipulation of shapes and patterns do not explain variance in reading comprehension skill (nation et al., 1999; seigneuric et al., 2000). this work does not establish directly whether it is the controlled attention aspect of working memory, or the storage function (or stm) that is important in reading comprehension. both stm and working memory deficits have been shown to make contributions to reading problems (e.g. de jong, 1998; swanson & ashbaker, 2000). however, stm tasks do not discriminate between good and poor comprehenders who are matched for word recognition (e.g. yuill & oakhill, 1991), and swanson & jerman (2007) found that it was working memory and not stm that predicted growth in reading comprehension. of course, working memory tasks have an stm requirement, but there must be something over and above this storage component that is important to comprehension skill and its development. in addition, although working memory capacity assessed by symbolic processing tasks explains individual differences in children’s text comprehension over and above other well established predictors of reading comprehension, such as word recognition skill and vocabulary knowledge (e.g., swanson & berninger, 1995; yuill et al., 1989), some researchers have suggested that the reported relation between children's working memory and text comprehension is the result of underlying levels of verbal and semantic skills. for example, nation et al. (1999) argue that poor comprehenders have a specific semantic weakness that restricts their ability to store verbal information in short-term memory and that this weakness, in turn, impairs their performance on verbally mediated working memory tasks. a similar position was adopted by stothard and hulme (1992), who proposed that working memory differences between good and poor comprehenders would disappear if differences in verbal iq were controlled. the present results argue against this position, since at least two of the working memory tasks were strongly predictive of comprehension skill over and above the contribution of general verbal (or numerical) ability, and demonstrate that working memory tasks that require symbolic processing are important predictors of comprehension skill in young children, over and above the cognitive skills that contribute to the tasks. • • • jane oakhill, is professor of experimental psychology at the university of sussex, brighton, uk. she worked briefly as a primary school teacher, and her interest in children’s reading comprehension problems led to a d.phil. on that topic. she has worked on various research projects (including deductive reasoning in children and adults, circadian variations in human performance, and adult language comprehension) but has always maintained a research interest in children’s reading comprehension (in particular, individual differences). nicola yuill is head of the children and technology lab (chatlab) at the university of sussex, uk. she has worked for many years on children's reading comprehension, especially intervention, and studies the role of social interaction and collaboration, particularly supported by technology, in understanding and developing language and communication in typical and atypical populations. alan garnham is professor of experimental psychology in the school of psychology at the university of sussex, uk, where he has worked for most of his professional career. his research interests are primarily in text comprehension, in particular anaphor processing. he has also worked on syntactic processing and on deductive reasoning. the differential relations between verbal, numerical / oakhill, yuill & garnham 105 references alloway, t. p., gathercole, s. e., & pickering, s. j. (2006). verbal and visuospatial short-term and working memory in children: are they separable? child development, 77, 1698-1716. baddeley, a.d. & hitch, g.j. (1974). working memory. in g.a. bower (ed.) the psychology of learning and motivation, vol. 8 (pp. 4790). new york: academic press. bayliss, d. m., jarrold, c., baddeley, a. d., gunn, d. m., & leigh, e. (2005). mapping the developmental constraints on working memory span performance. developmental psychology, 41, 579-597. bayliss, d. m., jarrold, c., gunn, d. m., & baddeley, a. d. (2003). the complexities of complex span: explaining individual differences in working memory in children and adults. journal of experimental psychology: general, 132, 71-92. caplan, d. and waters, g. s. (1999) verbal working memory and sentence comprehension. behavioral and brain sciences, 22, 77-126 cantor, j., engle, r.w., & hamilton, g. (1991). short-term memory, working memory, and verbal abilities: how do they relate? intelligence, 15, 229-246. carretti, b., borella, e., cornoldi, c. & de beni, r. (2009). role of working memory in explaining the performance of individuals with specific reading comprehension difficulties: a meta-analysis. learning and individual differences, 19, 246–251 cataldo, m.g. and oakhill, j.v. (2000) why are poor comprehenders inefficient searchers? an investigation into the effects of text representation and spatial memory on ability to locate information in a text. journal of educational psychology, 92, 791-799. cooke, a. e., halleran, j. g., & o'brien, e. j. (1998). what is readily available during reading? a memorybased view. discourse processes, 26, 109-129. daneman, m. & carpenter, p.a. (1980) individual differences in working memory and reading. journal of verbal learning and verbal behavior, 19, 450-466. daneman, m. & carpenter, p.a. (1983). individual differences in integrating information between and within sentences. journal of experimental psychology: learning, memory and cognition, 9, 561584. daneman, m. & merikle, p.m. (1996). working memory and language comprehension: a metaanalysis. psychonomic bulletin & review, 3, 422-433. daneman, m. & tardif, t. (1987). working memory and reading skill reexamined. in m. coltheart (ed.), attention and performance, 7.pp. 491-508. london: erlbaum. de beni, r., palladino, p., pazzaglia, f., & cornoldi, c. (1998). increases in intrusion errors and working memory deficit of poor comprehenders. quarterly journal of experimental psychology: human experimental psychology, 51a, 305–320. de jong, p. (1998). working memory deficits of reading disabled children. journal of experimental child psychology, 70, 75–95. dixon, p., lefevre, j. & twilley, l.c. (1988). word knowledge and working memory as predictors of reading skill. journal of educational psychology, 80, 465-472. engle, r. w., carullo, j. j., & collins k. w. (1991). individual differences in working memory for comprehension and following directions. journal of educational research, 84, 253-262. engle, r. w., nations, j. k., & cantor, j. (1990). is "working memory capacity" just another name for word knowledge? journal of educational psychology, 82, 799-804. engle, r. w., tuholski, s. w., laughlin, j. e., & conway, a. r. a. (1999b). working memory, short-term memory, and general fluid intelligence: a latent variable approach. journal of experimental psychology: general, 125, 309 – 331. gernsbacher, m. a. (1990). language comprehension as structure building. hillsdale, nj: lawrence erlbaum associates. graesser, a. c., singer, m., & trabasso, t. (1994). constructing inferences during narrative text comprehension. psychological review, 101, 371-395. hale, s., bronik, m. d., & fry, a. e. (1997). verbal and spatial working memory in school-age children: developmental differences in susceptibility to interference. developmental psychology, 33, 364–371. johnson-laird, p. n. (1983). mental models. cambridge: cambridge university press. just, m. a., & carpenter p.a. (1992). a capacity theory of comprehension: individual differences in working memory. psychological review, 99, 122-149. international electronic journal of elementary education 106 kane, m. j., hambrick, d. z., tuholski, s. w., wilhelm, o., payne, t. w., & engle, r. w. (2004). the generality of working memory capacity: a latent-variable approach to verbal and visuospatial memory span and reasoning. journal of experimental psychology: general, 133, 189-217. kennedy, a. (1987). eye movements, reading skill and the spatial code. in j.r. beech and a.m. colley (eds.) cognitive approaches to reading. chichester: wiley. kintsch, w. (1998). comprehension: a paradigm for cognition. new york: cambridge university press. leather, c.v. & henry, l.a. (1994). working memory span and phonological awareness tasks as predictors of early reading ability. journal of experimental child psychology, 58, 88-111. lépine, r., barrouillet, p., & camos, v. (2005). what makes working memory spans so predictive of high-level cognition? psychonomic bulletin and review, 12, 165-170. murray, w.s. & kennedy, a. (1988). spatial coding in the processing of anaphor by good and poor readers: evidence from eye movement analyses. quarterly journal of experimental psychology, 1988, 40a, 693-718. nation, k., adams, j. w., bowyer-crane, c.a. & snowling, m. j. (1999) working memory deficits in poor comprehenders reflect underlying language impairments. journal of experimental child psychology, 73, 139-158. neale, m.d. (1997) the neale analysis of reading ability: revised, london: macmillan education. oakhill, j. v., cain, k.e. & bryant, p.e. (2003). dissociation of single-word reading and text comprehension skills. language and cognitive processes, 18, 443-468. oakhill, j. v., yuill, n. m., & parkin, a. (1986). on the nature of the difference between skilled and lessskilled comprehenders. journal of research in reading, 9, 80–91. seigneuric, a., ehrlich, m-f., oakhill, j.v. & yuill, n.m. (2000). working memory resources and children's reading comprehension. reading and writing, 13, 81-103. shah, p., & miyake, a. (1996). the separability of working memory resources for spatial thinking and language processing: an individual differences approach. journal of experimental psychology: general, 125, 4-27. siegel, l.s., & ryan, e.b. (1989). the development of working memory in normally achieving and subtypes of learning disabled children. child development, 60, 973-980. stothard, s.e. & hulme, c. (1992) reading comprehension difficulties in children: the role of language comprehension and working memory skills. reading and writing, 4, 245-256. swanson, h.l. (1992) generality and modifiability of working memory among skilled and less skilled readers. journal of educational psychology, 84, 473-488. swanson, h.l. (1996). individual and age-related differences in children's working memory. memory & cognition, 24, 70-82. swanson, h. l., & ashbaker, m. (2000). working memory, short-term memory, articulation speed, word recognition, and reading comprehension in learning disabled readers: executive and/or articulatory system? intelligence, 28, 1–30. swanson, h.l., & berninger v.w. (1995). the role of working memory in skilled and less skilled readers’ comprehension. intelligence, 21, 83-108. swanson, h.l. & jerman, o. (2007) the influence of working memory on reading growth in subgroups of children with reading disabilities. journal of experimental child psychology , 96, 249–283 thorndike, r.l. & hagen, e. (1986) cognitive abilities test levels af. second edition. nfer nelson: windsor, berks. thorndike, r.l., hagen, e. & france, n. (1986) cognitive abilities test: administration manual. second edition. nfer-nelson: windsor, berks. turner, m.l., & engle, r.w. (1989). is working memory capacity task dependent? journal of memory and language, 28, 127-154. waters, g.s. & caplan, d. (1996). the measurement of verbal working memory capacity and its relation to reading comprehension. quarterly journal of experimental psychology, 1996, 49a, 51-79. yuill, n. & oakhill, j. (1991). children's problems in text comprehension: an experimental investigation. cambridge: cambridge university press. yuill, n., oakhill, j. & parkin, a. (1989). working memory, comprehension ability and the resolution of text anomaly. british journal of psychology, 80, 351-361. microsoft word 12_iejee_4_1_baker_et_al international electronic journal of elementary education, 2011, 4(1), 197-227. issn:1307-9298 copyright © iejee www.iejee.com children’s comprehension of informational text: reading, engaging, and learning linda baker∗∗∗∗ university of maryland, baltimore county, united states mariam jean dreher university of maryland, college park, united states angela katenkamp shiplet university of maryland, baltimore county, united states lisa carter beall university of maryland, baltimore county, united states anita n. voelker university of maryland, college park, united states adia j. garrett university of maryland, baltimore county, united states heather r. schugar university of maryland, college park, united states maria finger-elam university of maryland, baltimore county, united states abstract the reading, engaging, and learning project (real) investigated whether a classroom intervention that enhanced young children's experience with informational books would increase reading achievement and engagement. participants attended schools serving low income neighborhoods with 86% african american enrollment. the longitudinal study spanned second through fourth grades. treatment conditions were: (1) text infusion/reading for learning instruction -students were given greater access to informational books in their classroom libraries and in reading instruction; (2) text infusion alone -the same books were provided but teachers were not asked to alter their instruction; (3) traditional instruction -students experienced business as usual in the classroom. children were assessed each year on measures of reading and reading engagement, and classroom instructional practices were observed. on most measures, the informational text infusion intervention did not yield differential growth over time. however, the results inform efforts to increase children’s facility with informational text in the early years in order to improve reading comprehension. keywords: reading comprehension, informational text, reading instruction ∗ linda baker, department of psychology, umbc 1000 hilltop circle baltimore, md 21250, usa. phone: 410455-2370. e-mail: baker@umbc.edu international electronic journal of elementary education 198 introduction the tremendous nationwide emphasis on early reading achievement that came about through no child left beyond had as a primary focus the development of the basic skills of word recognition. the ability to read words fluently and automatically is critical to achievement, but it is not sufficient. nor is the ability to comprehend stories sufficient. although stories comprise a major proportion of the materials children encounter in early reading instruction (duke, 2000; hoffman et al., 1994; jeong, gaffney, & choi, 2010, moss, 2008; moss & newton, 2002; ness, 2011), older students are expected to comprehend and learn from informational text, and they must continue to do so as they move through school and into the job market (common core state standards (ccss), 2010; salinger, kamil, kapinus, & afflerbach, 2005; white, chen, & forsyth, 2010; venezky, 2000). thus, facility with informational text is also critical to achievement, in school and beyond. the reading, engaging, and learning project (real) that is the focus of this article was designed to respond to the national goals of raising reading achievement and closing the achievement gap. the intervention study was funded by the spencer foundation in 2000, at a time when national experts had begun to recommend that children be given more exposure to informational text in the early years of schooling (e.g., snow, burns, & griffin, 1998), but when support for this recommendation was mostly anecdotal and correlational (campbell, kapinus, & beatty, 1995; caswell & duke, 1998). the real project was a multicomponent intervention study that yielded information about a variety of facets of children’s experiences with informational text, including comprehension of text as assessed on a standardized multiple-choice instrument and on a performance assessment, ability to use the features of informational text that enhance comprehension, reading motivation, out-ofschool reading activity, instructional practices of the classroom teachers, preferences for reading, and gender differences. the three-year longitudinal study, spanning grades two through four, yielded only limited evidence of an effect of the intervention to increase students’ access to and comprehension of informational text (baker & dreher, 2005; dreher & baker, 2005). for that reason, we did not pursue publication of the real study at the time it concluded in 2005. however, this decision did not mean that we thought efforts to enhance children’s comprehension of informational text were bound to be unsuccessful. rather, we attributed the null effects primarily to school and teacher factors beyond our control. with hindsight, we have come to realize that the study provides much valuable information about children’s understanding of informational text, independent of intervention effects. accordingly, the purpose of this article is to describe the study in the context of the growing body of knowledge on informational text comprehension and use in the early elementary grades. traditionally, learning through reading has been delayed until children have learned how to read, using familiar topics in stories. but expert opinion has converged on the notion of integrating learning to read and reading to learn (e.g., ccss, 2011; international reading association/national council of teachers of english, 1996; national council for the social studies, 1994; national council of teachers of mathematics, 1989; national research council, 1994; snow et al., 1998; salinger et al., 2005). changes have begun to appear in the past decade, but the elementary school reading experience continues to be predominantly stories (jeong, et al., 2010, moss, 2008, ness, 2011). analyses of classroom reading materials suggest that up to 90% of what is read in the classroom consists of stories (dreher, 2000), yet there is no compelling reason why this should be so, even in the primary grades. research shows that young children can and do appreciate and understand informational text (cervetti, bravo, hiebert, pearson, & jaynes, 2009; diakidoy, stylianou, karefillidou, & children’s comprehension of informational text / baker, et al. 199 papageorgiou, 2005; horowitz & freeman, 1995; pappas, 1993; smolkin & donovan, 2001). furthermore, they can be taught strategies for learning from it (aarnoutse & schellings, 2003; culatta, hall-kenyon, & black, 2010; duke & carlisle, 2011; hall, sabey, & mclellan, 2005; o’hara, 2007; williams et al., 2005; williams, stafford, lauer, hall, & pollini, 2009). the new common core state standards (ccss, 2010) specify that children should acquire competencies with informational text beginning in kindergarten. for example, with prompting and support, kindergartners should be able to ask and answer questions about key details in a text, ask and answer questions about unknown words in a text, and identify basic similarities in and differences between two texts on the same topic (e.g., in illustrations, descriptions, or procedures). the fact that informational text processing skills are to be part of the standard language arts curriculum is a powerful indicator of the importance of fostering children’s comprehension of such text from the earliest years of formal schooling. indeed, early attention to informational text is needed if by fourth-grade children are to be “capable—independently and productively—of reading to learn” (snow et al., 1998, p. 207). the national assessment of educational progress (naep) now reflects this expectation (foorman & connor, 2011). beginning with the 2009 reading assessment, the naep framework for fourth grade specifies a 50-50 proportion of total testing time devoted to reading informational vs. literary text (national assessment governing board, 2008), compared to a 45-55 allocation in the 1992 to 2007 frameworks. in addition, for the first time, the 2009 naep reports scores separately by genre. performance on the naep, however, indicates the schools have far to go. the 2009 naep found two thirds of american fourth graders can only read at or below a basic level (33% below basic and 34% at basic), meaning they exhibit only partial mastery of reading skills (nces, 2009). this figure has decreased only slightly since 1992, when the percentage at or below basic was 71%, indicating that the recent federal initiatives to improve early reading skills have met with limited success. in fact, evaluations of reading first have revealed that benefits do not extend beyond decoding to reading comprehension (gamse, jacob, horst, boulay, & unlu, 2008). the 2009 naep results also document that certain subgroups of fourth graders continue to perform lower than the national average. for example, 52% of black and 51% of hispanic children scored below a basic level in reading, whereas 22% of white and 20% of asian/pacific island children did so. furthermore, 71% of english language learners (ell) scored below basic in comparison to 30% of non-ells, as did 49% of children eligible for free or reduced-cost meals, in comparison to 20% of those not eligible (http://nationsreportcard.gov/reading_2009/). thus, overall fourth-grade reading performance needs improvement, but for poor and minority children the situation is particularly urgent. to pursue our specific interest in comprehension of informational text, we accessed the naep data base and used their statistical tools to compare scale scores achieved on literary and informational text across selected demographic subgroups. the data for fourth graders on the 2009 assessment are provided in table 1. overall, students scored better on the literary passages than the informational. note that the demographic disparities in performance are even greater on informational text than on literary text. for example, children eligible for free lunch scored 28 points lower than non-eligible children on informational text, compared to 25 points lower on literary text. students from high-poverty areas often lack exposure to the academic vocabulary needed for reading comprehension (chall, jacobs, & baldwin, 1990; neuman, 2006). informational text is a major source of difficult, abstract, specialized, and technical words. international electronic journal of elementary education 200 the narrow focus on word recognition so typical in early literacy instruction should be replaced by content-rich environments that include informational books, given their considerable potential for increasing children’s background knowledge and conceptual development (foorman & connor, 2011; neuman, 2001; 2006; 2010). focusing on both learning to read and reading for learning -with early, balanced attention to both stories and informational text – may increase children’s reading achievement (ccss, 2010; dreher & voelker, 2004). support for this suggestion comes from the naep, where it was found that on the 1992 assessment, fourth graders who reported reading not only stories, but also magazines and informational books had the highest achievement (campbell et al., 1995). table 1. mean scale scores for fourth graders on the 2009 naep reading assessment by type of text and selected demographic variables demographic variable subgroup literary text informational text full sample -----222 219 income eligible for free lunch 208 203 not eligible 233 231 language ell 190 185 non-ell 225 222 race white 231 229 black 207 202 hispanic 208 202 asian/pacific isl. 236 234 gender girls 226 222 boys 218 216 note. maximum scale score is 500. differences as a function of income level, language status, and gender were statistically significant at p < .001 for each text type. for the race variable, asian/pacific islanders had the highest scores, followed by whites; blacks and hispanics did not differ significantly. differences across text types were not analyzed due to limitations of the naep online software. an important enabler of reading comprehension is reading engagement (baker, dreher, & guthrie, 2000). engaged readers read widely and frequently, and they seek opportunities to learn from reading. their motivations include the beliefs, desires, and interests that lead them to choose to read. yet reading motivation drops as children move through the grades (chapman & tunmer, 1997; mckenna, ellsworth, & kear, 1995; wigfield et al., 1997). this decline occurs for all achievement levels but is most pronounced for children of lower reading ability. effective use of more diverse material, including informational books, may help to counteract this drop in motivation to read by arousing children’s interest and curiosity, which in turn should increase independent reading activity (baker & wigfield, 1999; duke & carlisle, 2011; wigfield & guthrie, 1997). guthrie and wigfield and their colleagues have provided compelling evidence that classroom interventions that increase reading engagement also increase reading comprehension (guthrie, wigfield, barbosa et al., 2004). furthermore, the more motivated the reader, the more growth in comprehension over time (guthrie, et al., 2007). children’s comprehension of informational text / baker, et al. 201 a variety of frameworks have been used to categorize text types as fiction vs. nonfiction or literary vs. informational. the guidelines of kletzien and dreher (2004) were used in the real project. the informational text category includes narrative-informational, expository, and mixed. narrative-informational text uses a story or narrative format to convey factual information. expository texts do not include story elements such as characters, goals, and resolutions. instead they might be characterized as reports, using text structures such as cause and effect, comparison and contrast, sequence, description, and problem and solution. they explain things about the natural and social world such as animals, places, and cultural groups. mixed texts, also referred to as dual-purpose, blended, or hybrid texts, mix narrative and expository writing in the same texts. for example, in the popcorn book (depaola, 1984), the story of brothers making popcorn is accompanied by encyclopedia-like facts about popcorn which one brother reads aloud. the real project was based on the premise that increasing students’ access to informational text would enhance their knowledge and comprehension. accordingly, the central thrust of the project involved an infusion of informational books into classroom libraries. classroom libraries are frequently limited in the resources they offer, and this is particularly true in schools serving low income neighborhoods (neuman, 2006). the effective classroom library provides a variety of book genres including informational books, stories, poetry, references, and multimedia, and it offers a range of difficulty levels and caters to a range of interests (dreher & voelker, 2004). given the opportunity to access diverse materials in the classroom library, student interest in reading should increase, the amount of time spent reading should increase, and gains in reading skills and strategies should be realized (chambliss & mckillop, 2000). of course, at the same time, teachers need to provide instruction designed to help children read and learn from diverse genres. accordingly, one of the treatment conditions included professional development for teachers on reading for learning. the project was a three-year longitudinal study, commencing in second grade and continuing through fourth grade. second grade was selected as the starting point because children at this level have begun to master the skills of word recognition and are capable of comprehending other than the simplest of texts. fourth grade was selected as the ending point because it is at this level that many children begin to struggle, as the demands shift more from learning to read to reading to learn. it is not our intention in this article to provide a comprehensive empirical report of the entire real project but rather to describe certain components of the project in greater depth than others, consistent with the theme of this special issue on reading comprehension. students within classrooms were assigned to one of three treatment conditions. it was expected that students who experienced both text infusion and reading for learning instruction would exhibit the greatest gains in reading achievement and engagement; students who received text infusion alone would also benefit relative to those receiving traditional instruction, but not to the same extent as those in the combined treatment condition. overview of the methods of the real project treatment conditions informational text infusion/reading for learning (rfl) instruction (text infusion/rfl instruction). classroom libraries were enhanced with informational books, and teachers participated in professional development sessions on reading for learning, as described below. students were assessed on reading comprehension using both standardized and researcherdeveloped measures, motivation for reading, and reading activity each year of the project. students kept logs of books read during independent reading time, and teachers kept logs international electronic journal of elementary education 202 of books read aloud to the students. classrooms were visited regularly to observe instruction and collect logs. informational text infusion/traditional instruction (text infusion alone). to control for the possibility that the informational text infusion alone might be responsible for any positive outcomes (i.e., book flood effects, ingham, 1982), classroom libraries were enhanced with the same informational books as in the rfl instructional condition, but teachers were not asked to modify their usual instructional practices. all of the same data were collected. traditional instruction. classroom libraries were not supplemented with informational text, and teachers were not asked to modify their usual instructional practices. the same assessments were used, but students and teachers did not keep logs of their reading as this in itself might influence outcomes. however, classroom observations were made to provide data on informational book availability and instruction relating to informational text. (teachers received books of their choice at the end of the school year for use with the next year’s students.) participants the study was conducted within three demographically similar public elementary schools in a large suburban school system with a majority african american student population. the three schools that were selected also had a majority of children receiving free or reduced price lunch and a majority scoring below average on statewide assessments. the ethnicity distribution of participants at project outset was 86% african-american, 10% europeanamerican, 2% hispanic-american, and 1% asian-american. during the first year of the project, 222 second-grade children participated. in subsequent years, the number of participants was affected by withdrawals, new enrollees, and redistricting. in year 2, 195 third graders participated, and in year 3, 209 fourth graders. each school was randomly assigned to one of the three treatment conditions. table 2 shows the distribution of classrooms and teachers across conditions and years, as well as the number of students remaining in the real project for all three years. table 2. number of classes and students in the reading, engaging, and learning (real) project by treatment condition project year text infusion/rfl instruction text infusion alone traditional instruction classes students classes students classes students 1: grade 2 2 60 3 77 4 85 2: grade 3 3 62 4 79 3 69 3: grade 4 2 51 4 69 3 89 note. the number of students participating in the project across all three project years was 28 in text infusion/rfl, 51 in text infusion alone, and 30 in traditional instruction. teacher training at the beginning of each school year, brief orientations were given to the teachers who were participating in the project at all three schools, followed by in-service sessions with teachers in the text infusion schools. for the text infusion alone teachers, a single short session focused on procedures for completing the teacher and student reading logs. text infusion/rfl instruction teachers received in-service sessions on rfl instruction, one in the fall at the beginning of the intervention and one after winter vacation. in addition, we held informal discussions during our regular visits to the classrooms and via email. key instructional components were selected on the basis of research evidence attesting to their promise: (1) balancing the use of informational books in daily read-alouds; (2) promoting children’s comprehension of informational text / baker, et al. 203 diverse daily independent reading; and (3) teaching children strategies and text features for comprehending and using informational text. at second-grade, the fall session included modeling and providing guidelines on how to use information books in read-alouds (e.g., teach children to use text access features, activate prior knowledge, highlight new vocabulary, engage children in discussion). we explained the questioning the author (qta) approach to enhance children’s comprehension (beck mckeown, hamilton, & kucan, 1997). we also discussed examples of different types of informational text (narrative-informational, expository, and mixed) and the importance of instruction on expository text. in the second-grade winter session, we continued working with teachers to offer suggestions and address any concerns on the earlier topics. in addition, we introduced ways to use informational books to encourage expository writing. at third and fourth grades, in-service sessions continued attention to the same topics but with additional foci. at third-grade, we modeled the use of a variety of i-charts (hoffman, 1992) for organizing new information. we also provided additional techniques to support vocabulary development, and more ways to encourage expository writing such as paragraph frames (lewis, wray, & rospigliosi, 1994). at fourth grade, we provided teachers with a strategy instruction plan, overviewed key comprehension strategies for use with informational text, reviewed common expository text structures, and modeled how to help children search for and monitor their hunt for information (kletzien & dreher, 2004). in all grades, teachers received handouts, and a notebook in which to keep them, including a summary of research support for each topic, specific guidelines, and examples. availability of informational text in classrooms classroom libraries were inventoried each year, at the beginning before books were provided in the text infusion conditions and at the end. books were counted, listed, and categorized by type. classroom libraries varied greatly in size, but most were quite modest. as second grade began, children in one class had no classroom library at all, whereas in another class, the teacher had over 800 books. the books in these classroom libraries were predominantly fiction. this same variability was apparent in subsequent years of the project, as we worked in grade 3 and then 4. for example, in grade 4, the average classroom library at the traditional instruction school had 276 books, of which 21% were informational books. at the text infusion alone school the average was 169, of which 20% were informational. at the text infusion/rfl instruction school, one teacher had no classroom library and the other teacher had only 24 books, of which 17% were informational. this limited availability of informational text was similar to data reported in a number of other studies conducted prior to the beginning of the real project (duke, 2000; moss & newton, 1998), but recent analyses have yielded similar results. for example, jeong et al. (2010) inventoried five classroom libraries at each of grades 2, 3 and 4. the percentages of informational text were 22%, 18%, and 19% across the three grade levels. in a study of 318 teachers in k-5 classrooms conducted in 2007-2008 by ness (2011), teachers reported on average that 33% of the books in their classroom libraries were informational texts. across grades 2, 3, and 4, means were 36%, 35%, and 37%, respectively, with ranges from 15% to 75%. informational books were provided in the real text infusion classrooms in two phases. in the fall, we selected books from booklists to appeal to the interests of children at each grade level, and whenever possible to correspond to the science and social studies themes in the county curriculum. most of these contained expository writing, with text features such as tables of contents, indexes, and captions. after winter break, children and teachers both played a role in selecting additional books. children received “catalogs” of informational international electronic journal of elementary education 204 books and placed stickers on their top choices, and, in addition, they indicated what topics they would like to see addressed in other books. teachers were also given a list of books from which to make selections. as an illustration, we describe here the books added to classroom libraries in fourth grade. sixty-six informational books were purchased for each classroom, including 61 expository, 4 narrative-informational, and 1 mixed text book. eleven of the books were selected to reinforce topics in the social studies curriculum. others featured animals, famous people, sports, nature, and science. most of the books had text access features. the selection of books included those that were appropriate for students at a range of reading levels. student reading comprehension in the real project reading achievement was measured using a standardized assessment and two researcherdevised assessments. the standardized test was the gates-macginitie reading test (gmrt) 4th edition (macginitie, macginitie, maria, & dreyer, 2000). at level 2, the gmrt has subtests for decoding (word recognition), word knowledge, and comprehension; at levels 3 and 4 it has vocabulary and comprehension subtests. although the gmrt includes both fiction and informational passages on its comprehension assessment, scores are not broken down by type of passage. because of our primary interest in comprehension of informational text, we conducted supplementary analyses where we categorized passages according to text type so that we could compare scores on fiction and nonfiction (ruetschlin, finger, & dreher, 2005). one of the researcher-devised comprehension assessments was based on openended written responses to extended informational texts, and the other tapped student competencies in the use of text access features specific to informational text (e.g., indexes and glossaries). because of the small scale of the study, with only one school per treatment, and the limited number of teachers within each condition, it was not possible to conduct multi-level analyses. this is a significant limitation because differences among teachers were strong, even within the same school/treatment condition. moreover, the clustering of children within classrooms varied across the years; that is, students did not remain in intact classes as they moved from grade to grade. the longitudinal analyses are also limited by the decreasing sample size due to attrition from fall of grade 2 through spring of grade 4. the gmrt assessments in the first year of the study, children took the gmrt in the fall to provide baseline information and again in the spring. in subsequent years, they took it only in the spring. analyses of variance using extended scale scores (ess) were conducted, with time of test as the repeated measure and treatment condition (text infusion/rfl instruction, text infusion alone, and traditional instruction) as the between-subjects factor. extended scale scores allow one to compare achievement across the entire range of grade levels tested on the gmrt. a score of 500 represents the normative achievement of a 5th grader at the beginning of the school year; the normal curve equivalent score (nce) would be equal to 50. the same pattern of results was obtained in the analyses on each of the gmrt subtests and on total reading. we focus here on the comprehension assessment. mean scale scores are shown in table 3. during the first year of the project, in second grade, children improved significantly from fall to spring, as one would expect given regular classroom instruction, f(1, 176) = 271.72, p < .001, partial η2 = .61. children in all three conditions had similar scores at the outset of the project and showed comparable gains in comprehension. thus, contrary to predictions, reading achievement as indexed by a widely-used standardized assessment was not greater as a result of enhanced exposure and use of informational books in the classroom during second grade. results were similar in the longitudinal analyses; children children’s comprehension of informational text / baker, et al. 205 continued to improve in their reading skills through fourth grade, f (103) = 239.45, p < .001, partial η2 = .70, but with no differential effects related to treatment condition. in order to better contextualize the reading comprehension abilities of the children in the real project, we relate their performance at the end of grade 3 to national norms. this time point corresponds to the “reading by 9” initiative popularized in the late 1990s that emphasized the need to be ready to transition to “reading to learn” in 4th grade. children who have a grade equivalent (ge) of 3.9 in comprehension on the gmrt would have an ess of 477 and an nce of 52. in the real sample, only 25% of the children were reading at or above 3.9 on the spring of grade 3 assessment. the median ge was only 2.8. moreover, 25% of the students were reading at or below a ge of 2.3. clearly, as intended when we selected the district and the schools within it, children’s reading comprehension was low relative to national norms. table 3. extended scale scores for gmrt comprehension by treatment condition and grade level (sds in parentheses) treatment n fall gr. 2 spring gr. 2 spring gr. 3 spring gr. 4 text infusion/rfl 28 383.71 (34.19) 422.46 (34.27) 445.82 (27.60) 470.50 (21.78) text infusion alone 48 395.63 (37.99) 416.63 (36.46) 447.15 (38.83) 468.27 (33.23) treatment n fall gr. 2 spring gr. 2 spring gr. 3 spring gr. 4 traditional 30 392.60 (58.08) 422.37 (44.88) 458.07 (30.03) 475.77 (31.69) to address our specific interest in informational text comprehension, the passages on all four forms of the gmrt comprehension subtests were classified as fiction or nonfiction, with nonfiction further categorized as narrative-informational or expository. (no instances of mixed text were identified.) two raters independently categorized each passage and established good inter-rater reliability. the classifications were subsequently compared to those reported in the gmrt technical manual. for levels 2 through 4, the gmrt categorizes passages by content as fiction, social sciences, or natural sciences, and by type as narrative, expository, or setting. they used setting for “passages that seemed characteristic of those sections of stories that do not actually move the account forward in time” (macginitie et al., 2002, p. 21), and we classified all the setting passages as fiction. our passage classification matched on 36 of the 42 passages (86%) across the four forms. for two of the passages on which there was disagreement (one on each of the level 2 forms), the gmrt technical manual indicated that the source material was social studies exposition. however, when taken out of context as in an assessment, these passages appeared to be stories and we coded them as such. on level 2t, two other mismatches occurred for (a) a passage the gmrt considered social studies narrative but appeared to be a story so we coded it fiction, and (b) a passage the gmrt classified as natural science narrative but we classified as expository. on level 4s, there were two mismatches. in one case, the gmrt classified a passage as social studies narrative, but we classified it as exposition. in the other case, the passage was labeled exposition, but we judged it to be narrative-informational. although there were few mismatches, those that occurred underscore the difficulty of classifying short segments out of context. table 4 shows the number of passages of each type on each form coded according to our system, as well as the number of test items of each type. the proportion of nonfiction passages increases with grade level, as one would expect. unexpectedly, the two types of nonfiction passages, expository and narrative-informational, were unevenly distributed across forms and grade levels. proportion correct of each item type was calculated for each international electronic journal of elementary education 206 student, and these data were used to test for internal consistency reliabilities for each of the item subsets; cronbach’s alphas are presented in table 4. (in an additional set of analyses not described here, we determined that the nature of the questions changes from predominantly literal in grade 2 to predominantly inferential in grades 3 and 4 and that students performed better on literal questions at all four testing points.) within-subject analyses of variance were conducted to compare comprehension on the different passage types. treatment condition was not included as a factor in order to increase power. all children who completed each test were included in the analyses, regardless of how long they had been enrolled at the participating schools. mean proportions correct on fiction and nonfiction items are shown in table 5. in the fall of grade 2, on form 2s, and in the spring of grade 2, on form 2t, students scored significantly better on the fiction questions than the nonfiction, f (1,222) = 223.70, p < .001, partial η2 =.50, and f (1, 211) = 42.23, p < .001, partial η2 = .17, respectively. grade 3 form s data also indicated significantly better performance on fiction than on nonfiction, f (1, 183) = 33.09, p < .001, partial η2 = .15. however, in grade 4 form t, performance was comparable on the two text types, f (1, 206) < 1. the commonplace generalization that children comprehend fiction better than nonfiction was supported in grades 2 and 3, but by grade 4 informational text was comprehended as well as fiction. this latter outcome contrasts with the 2009 naep results, which showed an advantage for literary over informational comprehension for fourth graders. the pattern also varies from that reported by diakidoy et al. (2005) who found that fourth graders had better scores on expository than on narrative, but second graders were comparable. outcomes on reading assessments may vary depending on the nature of the nonfiction passages that are used (e.g., narrative-informational and/or expository). because the grade 4 test had sufficient numbers of items of each type, with good internal consistency reliability on each subscale, an additional analysis was conducted comparing comprehension of fiction, narrative-informational, and expository text types. the overall main effect was significant, f(2,206) = 6.90, p < .001, partial η2 = .06, with follow up analyses showing that expository text comprehension was significantly better than comprehension of narrativeinformational text; comprehension of fiction was at an intermediate level, not significantly different from either of the other two text types. a parallel analysis was conducted for grade 2 (form s), but results should be interpreted cautiously given the low reliability of the expository text subscale. this analysis also revealed an overall effect of text type, f (2,222) = 129.64, p < .001, partial η2 = .37. comprehension was best on fiction, with narrativeinformational text in the middle, and with expository text comprehension the weakest; all pairwise comparisons were significantly different. these data are also shown in table 5. several issues must be kept in mind in interpreting these results, the most important of which is that the passages were very short, averaging from 82 words in the grade 2 tests to 93 words in grade 4. classification as to text type is more accurate with longer passages. moreover, students reading short expository passages often do not encounter the same text features they would encounter in longer expository texts, such as bold-faced words, headings, and glossaries. also, complex organizational text structures are not as evident in short expository passages as they are in longer expository texts. a final limitation of this analysis is that the developers of the gmrt did not design the test to yield separate scores on comprehension of fiction and non-fiction. although we found acceptable internal consistency reliability on all but one of the subscales, we were not using the test as the developers intended. children’s comprehension of informational text / baker, et al. 207 table 4. number of passages and items of different text types on the gmrt comprehension test and internal consistency reliabilities of the derived sub-scales table 5. proportion of correct responses (sds in parentheses) on the fiction and nonfiction items of the gmrt comprehension test type of test item form n fiction overall nonfiction narrative info expository grade 2 (s) 223 .53 (.25) .36 (.22) .41 (.27) .30 (.22) grade 2 (t) 212 .61 (.23) .54 (.25) .. .54 (.25) grade 3 (t) 184 .51 (.24) .45 (.23) .. .45 (.23) grade 4 (s) 207 .51 (.24) .51 (.20) .48 (.22) .52 (.21) the analysis of passage types on the gmrt points to an important issue with regard to nonfiction text: how it is categorized. in particular, the distinction between expository and narrative-informational text is often problematic. for example, although the intent of the gmrt developers was that forms 2s and 2t present children with equivalent numbers of expository and narrative-informational passages, we found disparity between the forms. our classification indicates that each form has 15 nonfiction items; but 2t has 15 items on expository passages, whereas 2s has 7 items on expository passages and 8 on narrativeinformational. as noted, part of the difficulty may be that these passages are not only very short, but that they appear out of context. but to complicate matters, nonfiction is not categorized consistently in the literature. in the classification system used on the 2009 naep, form passage type fiction overall nonfiction narrative info expository grade 2 (s) 6 4 2 2 grade 2 (t) 6 4 0 4 grade 3 (s) 6 5 0 5 grade 4 (t) 4 7 3 4 item type fiction overall nonfiction narrative info expository grade 2 (s) 24 15 8 7 grade 2 (t) 24 15 0 15 grade 3 (s) 29 19 0 19 grade 4 (t) 15 33 17 16 cronbach’s alpha fiction overall nonfiction narrative info expository grade 2 (s) .87 .75 .78 .46 grade 2 (t) .87 .79 .. .79 grade 3 (s) .86 .80 .. .80 grade 4 (t) .79 .85 .70 .70 international electronic journal of elementary education 208 narrative-informational passages are included within the literary category, and expository passages are included within informational (nces, 2009). in contrast, the common core state standards include both types in their definition of informational text (ccss, 2010). this variability in classification schemes limits generalizability across studies. it is clear from our analyses that the distinction between narrative-informational and expository text is important, and these text types should be examined separately. moreover, it is not yet understood whether children process narrative-informational texts more similarly to fictional narratives or to nonfictional exposition. however, research reveals differences in the competencies that contribute to comprehension of the different genres. best, floyd, and mcnamara (2008) found that among third graders, comprehension of narrative text was most influenced by decoding skills, whereas comprehension of expository text was most influenced by world knowledge. the study provides confirmation of the widely-stated assertion that children need a solid knowledge base in order to understand expository text. reading-and-writing-to-learn assessments to more directly tap the comprehension skills that were the focus of the real project, performance assessments were developed for each grade level and were pilot tested prior to implementation. for second and third grade, two alternate form expository texts were created and were presented in counterbalanced order across fall and spring test sessions. the second grade texts dealt with either snakes or frogs, and each was bound in an 8-page booklet with color illustrations and a table of contents. the third grade texts dealt with either rome or egypt. booklet length was increased to 16 pages; the texts included color illustrations and text access features (a table of contents, glossary, and index). for fourth grade, the task increased in complexity with children receiving two booklets at each testing session, requiring them to integrate information across booklets. fourth graders received booklets on either boston and chicago, or hawaii and alaska, presented in counterbalanced order across fall and spring testing. the four booklets for fourth grade were each 18 pages long, including illustrations, a table of contents, glossary, and index. at all three grade levels, children responded in writing to open-ended prompts about the materials they read, as described below. the reading and writing to learn assessments differed across the years because of children’s increasing competencies. although we developed this assessment prior to the publication of the common core state standards, the expository text comprehension demands at each grade level were consistent with the standards. table 6 shows the specific prompts that were used in each assessment and examples of the qualitative scoring rubrics that were used. grade 2 performance assessment. students were presented with one of the two books and an accompanying response sheet. children’s responses to the prompts shown in table 6 were scored according to a 5-point scale ranging from 0 to 4. portions of the rubric for question 2 are also shown. each response was scored by two independent coders; reliability was strong, with correlations averaging about .90. the two questions were summed for a total score. although the scores were significantly correlated, internal consistency reliability was rather low, .52 in the fall and .47 in the spring. children’s comprehension of informational text / baker, et al. 209 table 6. prompts and sample scoring rubrics for the reading and writing to learn performance assessments grade 2 prompt 1. use your book to learn new information about frogs. find out interesting information about frogs. write down what you have learned on the lines below. today i learned about frogs. here are some interesting things that i learned. … 2. pretend your friend wants to pick one of these frogs as a pet. use your book to find out about these two frogs. which frog should your friend pick as a pet? explain why your friend should pick that frog. circle the frog your friend should pick (photographs were provided of the two frogs described in the text). my friend should pick this frog as a pet because …. . portions of the scoring rubric for question 2 5 points: answer is written in student’s own words, claims one of the animals as the best choice, uses information from more than one place in the text to support the choice, and includes a contrasting of the two animals to further explain and justify the choice. 1 point: answer is expressed as an opinion, or uses only picture clues, but does not include information from the text. grade 3 prompt pretend that you are going back in time to visit the people of rome/greece long ago. write what the people would be like. be sure to include information on how they would dress, what they would eat, what they do for fun, and any other information that would help you tell us what their life would be like. portions of the scoring rubric for the use of notes 4 points: notes are clear and serve as the primary basis for the details in the writing and the organizational structure. 1 point: notes are recorded but there is little or no use of notes in the writing. international electronic journal of elementary education 210 table 6 (continue). prompts and sample scoring rubrics for the reading and writing to learn performance assessments grade 4 prompt where do we go? today you will pretend your class has won an all-expense paid trip to 1 of 2 exciting places. your job is to research both locations and decide which place you think your class should visit. you will have two short books to read about these places. we will give you time to read the first book and take notes about what you like and don’t like. then we will give you time to read the second book and take notes. you will use the information you have gathered to choose where you think your class should go. after you have decided where you think your class should go, you will write a letter to your teacher persuading him or her to take your class to this place. be sure to include information from the 2 books to support your choice. portions of the scoring rubric for persuasion 4 points: the student takes a clear stand on where the class should go and fully supports it with reasons based on the reading and their prior knowledge. the student details why they should go to the chosen place, and why they should not go to the other place. 1 point: the student does not take a stand on the issue. the student presents some information but it is not clear where he/she would prefer to go, or why they came to his/her decision. table 7. mean scores on the reading and writing to learn assessments (sds in parentheses) by treatment condition, grade level, and assessment time grade treatment condition text infusion/rfl instruction text infusion alone traditional instruction 2 n = 47 n=67 n=65 fall 3.13 (1.41) 3.30 (1.76) 3.31 (1.89) spring 4.13 (1.75) 4.18 (1.57) 4.14 (1.94) 3 n=33 n=52 n=35 fall 12.45 (3.48) 12.58 (4.23) 13.69 (5.02) spring 14.64 (3.34) 14.73 (4.24) 16.60 (5.01) 4 n=27 n=30 n=31 fall 11.19 (2.94) 11.23 (2.81) 11.35 (3.05) spring 10.56 (3.03) 11.87 (3.14) 11.94 (3.35) table 7 shows the mean scores on the reading and writing to learn assessment in the fall and in the spring for each treatment condition. preliminary analyses revealed that scores on the two alternate forms were comparable and that scores were comparable across conditions at baseline. a repeated measures analysis of variance revealed significant growth over the school year f (1, 176) = 41.24, p < .001, partial η2 = .19 (fall mean = 3.26 and spring children’s comprehension of informational text / baker, et al. 211 mean = 4.15). contrary to predictions, however, children who received the text infusion with reading for learning instruction did not outperform children experiencing only the text infusion or traditional classroom instruction, even though this assessment was more closely aligned with the focus of the intervention than was the gmrt. grade 3 performance assessment. in grade 3, children responded to a single elaborated question, shown in table 6. along with the passages, they were provided with a grid for taking notes to help them prepare their response. the worksheet included spaces for recording information about clothes, food, dress, and other information. responses were scored according to four criteria: topic use, use of notes, accuracy of information, and quality of writing. scores on each could range from 0 to 4. see table 6 for a portion of the rubric for use of notes. each response was scored by two independent coders, and inter-rater reliability was strong, with correlations above .90. the four separate scores in the coding rubric were summed for a total score. cronbach’s alpha revealed good internal consistency reliability (.78 in the fall and .75 in the spring). mean scores are shown in the middle section of table 7. analysis of variance revealed significant improvement from fall to spring, f (1, 117) = 24.24, p < .001, partial η2 = .17, with a mean score of 12.87 in the fall and 15.25 in the spring. however, again contrary to predictions, children in the text infusion/rfl instruction condition did not achieve higher comprehension scores than children in the other two conditions, nor was there differential growth over the school year. grade 4 performance assessment. the grade 4 task required students to integrate information across two expository texts to derive their responses to a single question (see table 6). along with the passages, students were given a response sheet with columns for taking notes about what they liked and did not like about each of the two possible destinations for a class trip. a 5-element coding rubric was used for the responses, with 0 to 4 points on each element. students were scored on persuasiveness, organization, style, grammar, and use of notes. a portion of the rubric for persuasion is shown in table 6. each response was scored by two independent coders, with reliability again strong (correlations among raters averaged .88). the five elements in the coding rubric were summed for a total score. cronbach’s alpha revealed good internal consistency reliability (.82 in the fall and .85 in the spring). on this more challenging performance assessment, students showed no significant improvement from fall to spring, f (1, 85) < 1. the mean score in the fall was 11.26 and in the spring it was 11.45. once again, contrary to predictions, informational text infusion in the classroom, with or without reading for learning instruction, did not affect performance (see table 7). predictors of performance on the reading and writing to learn assessments. as was just demonstrated, a key question that motivated the real project was answered negatively; that is, the text infusion/rfl instruction intervention did not promote growth over time in children’s comprehension of expository text on the rwtl assessment. another important question concerns the role of early literacy competencies on subsequent achievement. as has been amply shown in the literature, children come to school with vast differences in background knowledge that are not easily supplemented in the classroom, and the comprehension skills that children have acquired by the end of first grade are strong predictors of performance throughout their school years (duke & carlisle, 2011; neuman, 2006). multiple regression analyses were therefore conducted to examine the extent to which competencies at the beginning of grade 2 predicted growth on the performance assessments from fall to spring in each of grades 2, 3, and 4. if children’s early facility in reading and writing about informational text continues to predict performance in international electronic journal of elementary education 212 subsequent years, above and beyond comprehension as measured by the gmrt, it would provide evidence that further efforts to enhance children’s experiences with informational text are indeed warranted. in each analysis, students’ scale scores on the fall of grade 2 administration of the gmrt were entered into a regression analysis, treating decoding, word knowledge, and comprehension as separate variables, along with the fall of grade 2 scores on the reading and writing to learn assessment. for the regression predicting spring of grade 2 rwtl scores, no other predictor variables were entered. for the regressions of spring of grade 3 and 4 rwtl scores, the fall rwtl scores were used for the respective grade level in order to test for growth across the year. the results of the three analyses are shown in table 8. the overall model for grade 2 was significant, f (4, 172) = 15.97, p < .001, accounting for 27% of the variance. significant variance was accounted for by the fall rwtl scores, by gmrt decoding, and gmrt word knowledge. the gmrt comprehension scores did not contribute additional variance beyond that accounted for by the other variables. at this early grade level, students’ entering decoding skills best predicted growth in the performance assessment. the overall model for grade 3 was also significant, f (5, 112) = 4.59, p < .001, accounting for 17% of the variance. however, no individual predictors were significant, with gmrt comprehension the only one that even approached significance at p < .10. of most interest was the longer-term analysis of predictors of reading and writing to learn at the end of grade 4. this analysis provided clear evidence of the importance of early informational text competencies. the model was significant, f (5, 78) = 18.44, p < .001, accounting for 54% of the variance. the fall of grade 2 rwtl score was the strongest individual predictor of rwtl growth in grade 4. the beta weight was even stronger than that of the fall of grade 4 rwtl score. none of the early gmrt scale scores significantly predicted unique variance, although decoding came close at p<.07. thus, the children who at the beginning of grade 2 already had the comprehension skills needed to read an expository text and respond in writing to open-ended prompts fared better on the end-ofgrade 4 rwtl assessment. we believe that question 2, which called for the child to justify the choice of a pet for a friend, was a particularly sensitive assessment of comprehension because one of the animals was described as harmless and the other as poisonous. table 8. regression analyses of early gmrt scores and reading and writing to learn (rwtl) scores as predictors of growth in rwtl in grades 2, 3, and 4 b se b βeta t sig. spring grade 2 rtwl intercept -.18 1.41 -.127 .90 fall gr. 2 rwtl .33 .07 .33 4.54 <.001 fall gr. 2 comprehension .003 .004 .07 .81 .42 fall gr. 2 decoding .02 .01 .44 3.42 .001 fall gr. 2 word knowledge -.01 .01 -.25 -2.03 .04 spring grade 3 rtwl intercept -1.40 5.28 -.27 .79 fall gr. 2 rwtl .31 .25 .12 1.23 .22 fall gr. 2 comprehension .02 .01 .19 1.70 .09 fall gr. 2 decoding .02 .02 .16 .88 .38 note. r2 = .27 for grade 2, r2 = .17 for grade 3, and r2 = .54 for grade 4. children’s comprehension of informational text / baker, et al. 213 table 8 (continue). regression analyses of early gmrt scores and reading and writing to learn (rwtl) scores as predictors of growth in rwtl in grades 2, 3, and 4 b se b βeta t sig. spring grade 2 rtwl intercept -.18 1.41 -.127 .90 fall gr. 2 rwtl .33 .07 .33 4.54 <.001 fall gr. 2 comprehension .003 .004 .07 .81 .42 fall gr. 2 decoding .02 .01 .44 3.42 .001 fall gr. 2 word knowledge -.01 .01 -.25 -2.03 .04 spring grade 3 rtwl intercept -1.40 5.28 -.27 .79 fall gr. 2 rwtl .31 .25 .12 1.23 .22 fall gr. 2 comprehension .02 .01 .19 1.70 .09 fall gr. 2 decoding .02 .02 .16 .88 .38 fall gr. 2 word knowledge .001 .02 .004 .03 .98 fall gr. 3 rwtl .07 .10 .07 .67 .50 spring grade 4 rtwl intercept 1.61 3.14 .51 .61 fall gr. 2 rwtl .92 .19 .44 4.86 <.001 fall gr. 2 comprehension .01 .01 .09 .82 .42 fall gr. 2 decoding .03 .01 .35 1.86 .07 fall gr. 2 word knowledge -.02 .02 -.25 -1.50 .14 fall gr. 4 rwtl .27 .10 .24 2.64 .01 use of informational text features children’s ability to access information using text features was assessed in the first two years of the project. text access was a central component of the instruction teachers in the text infusion/reading for learning condition were asked to provide. considerable attention was given to text access features in the in-service training, and texts were selected for the classroom libraries based on the quality of these features. samples of the text access items are provided in table 9. the same assessment instrument was used in the fall and spring. in the grade 2 task, children were asked six questions about finding information in a table of contents and an index. in grade 3, a more demanding assessment was used, consisting of 12 questions, most of which required children to decide whether the requested information could be derived from the table of contents, the index, or the glossary. text access was not examined in fourth grade because many students scored at or near ceiling on the third grade measure. the decision to focus only on the primary grades turned out to be consistent with the subsequently-disseminated common core standards, which specifically address use of text access features in grades 2 and 3, but not in grade 4. international electronic journal of elementary education 214 table 9. sample items from the text access tasks grade 2 table of contents what is a bear? 5 where do bears live? 8 what do bears eat? 13 baby bears 20 polar bears 25 brown bears 29 black bears 35 bears in danger 38 bears are wild animals 43 words you should know 46 index 47 1. which page would tell you what bears eat? ______ 1. which page would tell you about brown bears? _____ grade 3 here are the table of contents, glossary, and index from a book about france. (materials are provided to the students.) 1. on what page does the chapter “vacation time” begin? ____ 2. on what page can you find information on castles? ____ 3. what does “immigrant” mean? ____ table 10 provides the mean proportion of items answered correctly on the text access tasks in grades 2 and 3. the grade 2 repeated measures analysis of variance revealed significant improvement from fall to spring, f(1, 173) = 111.99, p = .001, partial η2 = .39. of most importance to the goals of the study, and consistent with predictions, was the presence of a reliable interaction of time of test and treatment condition, f (2, 173) = 3.09, p = .048, partial η2 = .03. children in the text infusion/rfl instruction condition made relatively greater gains over the school year than children in either of the other conditions. in grade 3, repeated measures analysis of variance revealed gains from fall to spring, f (1, 158) = 17.90, p < .001, partial η2 = .10, but no differential improvement across conditions. across all three project years, scores on the gmrt were significantly correlated with text access scores. similarly, scores on the reading and writing to learn tasks were significantly correlated with use of informational text features. motivation and voluntary reading activity because the real project was conceptualized within the engagement model of reading (baker, dreher, & guthrie, 2000), we examined not only reading achievement, but also enablers of that achievement. in this view, students who are motivated to read and who choose to read frequently will be better comprehenders than those who are less motivated and who read less. it was hypothesized at project outset that increasing student access to interesting texts would increase reading motivation and reading activity, which in turn would increase reading comprehension. we briefly report the results of our analyses of motivation and reading activity, but without the level of detail given to comprehension. children’s comprehension of informational text / baker, et al. 215 table 10. mean proportion correct on text access tasks in grades 2 and 3 by treatment condition and time of test grade text infusion/rfl instruction text infusion alone traditional instruction 2 fall .44 .51 .57 spring .82 .84 .78 3 fall .66 .77 .78 spring .75 .80 .81 the motivations for reading questionnaire – primary (mrq-p), a shorter and simplified form of an instrument designed for older elementary school children (baker & wigfield, 1999; wigfield & guthrie, 1997) was administered in the fall and spring of year 1 and in the spring only in years 2 and 3. students rated their motivation with respect to different aspects of reading on a 3-point scale. item ratings were summed for a total motivation score, and subscale scores for four dimensions were also obtained: perceived competence, interest (intrinsic motivation), recognition (extrinsic motivation), and social interaction. analyses of internal consistency reliability were conducted each time the mrq-p was administered, and alphas were at acceptable levels (typically around .85). sample items are shown in table 11. in grade 2, children showed a significant decline in overall motivation from fall to spring, consistent with a number of other studies showing declines in the early grades. however, longitudinal analyses did not reveal further declines in third and fourth grade. similar to the results for the reading assessments, motivation was not impacted differentially by classroom text infusion. analyses of the separate motivation subscales revealed generally similar patterns to the overall scale, with the exception of the perceived competence subscale. although children’s perceived competence in reading declined from fall to spring of second grade, it increased over the next two years. table 11. sample items from the motivation for reading questionnaire (mrq-p) 1. perceived competence how good a reader are you? very good ok not very good 2. interest (intrinsic motivation) how much do you like to read about new things? very much a little not much at all 3. recognition (extrinsic motivation) how much do you like to get praise for your reading? very much a little not much at all 4. social how much do you like to tell your family about what you are reading? very much a little not much at all note. responses at the positive end of the scale were scored as 3, those at the negative as 1. correlational analyses examined whether higher levels of reading motivation were associated with better reading comprehension. motivation scores were associated with performance on the gmrt over the years, with the strength of the correlations greater for particular subscales. for example, in grade 4, perceived competence and reading for international electronic journal of elementary education 216 recognition (extrinsic motivation) were significantly correlated with achievement, but reading for interest (intrinsic motivation) and reading for social interaction were not. motivation scores were more weakly associated with performance on the reading and writing to learn assessment than the gmrt, especially for the assessments administered in the first year of the project. children’s motivation was also weakly associated with performance on the text access tasks, with stronger relations for the perceived competence subscale than for the overall measure or other subscales. table 12 shows the consistent relations between perceived competence and gmrt reading comprehension over the years. it is particularly noteworthy that as early as the beginning of second grade, children’s perceptions of themselves as readers were associated with their actual performance, concurrently and into the future. also noteworthy is the suggestion of bidirectional influences; fall of grade 2 perceived competence was associated with grade 4 comprehension, and fall of grade 2 comprehension was associated with grade 4 perceived competence. table 12. correlations between perceived competence and gmrt comprehension perceived competence reading comprehension fall gr. 2 spring gr. 2 spring gr. 3 spring gr. 4 fall gr. 2 .24 .21 .23 .25 spring gr. 2 .27 .31 .28 .28 spring gr. 3 .25 .27 .22 .21 spring gr. 4 .38 .33 .36 .37 note. listwise n=101. all correlations are significant at p < .05 or better. the reading activity inventory – primary (rai-p), a simplified version of the instrument designed for older children (wigfield & guthrie, 1997) was administered following the mrqp. students responded to two types of questions regarding their reading activity out of school, one asking how often they read particular types of books and the second asking whether they had read that type of book within the past week. the rai-p assesses the amount and breadth of students' reading by examining the frequency with which students read texts of different genres (e.g., mysteries, biographies, nature books). table 13 shows sample questions for fiction and for informational text. in grade 2, analyses revealed significant declines in self-reported reading activity from fall to spring; these declines occurred for children in all treatment conditions. the longitudinal analysis showed that reading activity scores did not decline further in grades 3 or 4. however, the effect of treatment condition was statistically significant, f(2, 109) = 5.85, p =.004, η2 =.10. consistent with one of the real hypotheses, students who received only traditional instruction reported less frequent outside reading over the years than students who experienced text infusions in their classrooms. (for traditional instruction, m = 2.09, sd = .54; for text infusion alone, m = 2.46, sd = .56, and for text infusion/rfl instruction, m = 2.30, sd = .58). note that a mean rating of 2.0 corresponds to reported reading of a particular type of book “about once a month” whereas a mean rating of 3.0 corresponds to “about once a week.” these results suggest that exposing children to interesting informational text in the classroom may stimulate more frequent out-of-school reading. children’s comprehension of informational text / baker, et al. 217 table 13. sample items from the reading activity inventory (rai-p) question regarding fiction did you read a make-believe story or book last week (like a mystery or an adventure for your own interest or for fun? no yes how often do you read make-believe stories or books for your own interest or fun? almost never about once a month about once a week almost every day question regarding informational books did you read a nature book last week for your own interest or for fun? no yes how often do you read nature books for your own interest or for fun? almost never about once a month about once a week almost every day note. responses of “almost every day” were scored as 4; those of “almost never” as 1. self-reported reading activity was consistently related to reading motivation, as is the case when the full versions of the mrq and the rai are used (baker & wigfield, 1999; beall, 2011; wigfield & guthrie, 1997). however, self-reported reading activity was not associated with reading comprehension in this study, across any of the years. this contrasts with other research showing relations between out-of-school reading and reading achievement (beall, 2011; serpell, baker, & sonnenschein, 2005, wigfield & guthrie, 1997). contextual and individual differences opportunities to read informational text in the classroom a central premise of the real project was that reading informational books would increase the likelihood that students would acquire the knowledge and motivation needed to become better comprehenders. an examination of what took place in real classrooms was therefore important as an implementation check and as a means of understanding how contextual factors might impact children’s comprehension of informational text (katenkamp, garrett, & baker, 2005). classroom observations were conducted in all of the real classrooms from october to may during each year of the project, usually three weeks apart. because the use of informational text could occur at any time during the day, rather than being limited to reading instruction, these observations took place for the full school day. observers used an adapted version of the center for the improvement of early reading achievement (ciera) school change classroom observation scheme (taylor & pearson, 2001). this scheme includes seven major categories that reflect who is leading the class, the kind of classroom grouping, the major focus of the lesson, the specific type of activity going on, the type of material being used, the nature of the teacher interaction, and the expected pupil response. in addition, the major focus of instruction (i.e., reading, math, science, social studies), the manner in which students were grouped (i.e., whole group, small group, individual), the number of students engaged in the activity, and the total number of the students in the room were recorded. a five-minute coding interval was followed by a five-minute break during which observers continued to take field notes. we focus here on selected components of the observations. an overall reading composite was created that consisted of how often the coded activity reflected reading or listening to connected text. word or sentence level activities were not included. the amount of classroom time spent in reading increased across grades two through four. more reading took place in text infusion classrooms; effects were significant in text infusion/rfl international electronic journal of elementary education 218 instruction classrooms in grades 2 and 4 and in text infusion alone classrooms in grade 3. however, the amount of time spent reading in the classroom was not related to reading achievement or motivation. increased access to informational books may thus have contributed to increased reading, but increased reading alone was not sufficient to impact comprehension. opportunity to listen to the reading of connected text was also examined. given that teachers in the text infusion/rfl instruction condition were explicitly asked to read aloud the informational books that were provided, it was expected that these teachers would spend proportionally more time in such activity. this expectation was met in second grade, but not in third or fourth. in second grade, teacher read-alouds comprised 20% of classroom reading activity in the text infusion/rfl instruction condition, but only 9% in the text infusion alone and 8% in the traditional instruction condition. proportions decreased to less than 10% in all conditions by 4th grade. the amount of time teachers read aloud to their students was unrelated to reading achievement or motivation. opportunities to read different genres (stories and informational texts) were examined. a composite measure of all informational texts was created; this composite included both text and trade books classified as expository, narrativeinformational and mixed texts. the percentage of materials used that were coded as informational texts increased significantly from 2nd to 4th grade, from 11% to 20% to 26%. students in the text infusion classrooms used informational texts more often than those who were in the traditional instruction classrooms, as would be expected given the nature of the intervention. the difference was most dramatic in second grade, where informational texts were used 23% of the time in the text infusion/rfl instruction classrooms but only 5% and 6% of the time in the other two types of classrooms. contrary to predictions, greater use of informational text was not related to reading comprehension. individual preferences for informational vs. narrative text research has shown that young children respond positively to non-narrative text. for example, when mohr (2003) gave first graders the opportunity to select a book to keep, 84% selected a nonfiction book. relevant information was collected in the real project through reading logs and preference inventories (beall, morse, baker, & dreher, 2005). we focus here on fourth-grade findings, when students were asked to list their two favorite books for the year from any they had read at school or at home, to explain why they liked these books, and to select a book to keep as a thank-you gift for project participation. overall, and in contrast to the results of mohr (2003), the majority of students listed a narrative book as their first and second favorite. informational books were the first choice for 26% of the students and the second choice for 33%. of particular note is that treatment condition was associated with the type of book chosen as a favorite. a greater percentage of students in the text infusion/rfl instruction and text infusion alone conditions listed an informational book as their first favorite than in the traditional instruction condition (41% and 31%, respectively, versus 10%), and the same pattern held for the second favorite (44% and 45% versus 17%), suggesting that increasing informational text availability can influence preferences. the reason most frequently given by students for selecting a particular book as a favorite was emotional appeal (33%; e.g., “it was funny”). personal interest was given as a reason by 19% of the students (e.g., “it has my favorite animals,” and “it had something to do with my hobby”). value to the reader was mentioned by 18% of the students (e.g., “it told me many things about cats,” “it shows the basic steps how to play).” students whose favorite books were informational more frequently gave reasons related to the value of the topic. guthrie et children’s comprehension of informational text / baker, et al. 219 al. (2007) also found that fourth graders’ motivation for reading expository text focused primarily on acquiring new knowledge. whether students identified informational books as their favorite was related to reading achievement in the text infusion/rfl instruction condition. students who selected an informational book as their favorite scored significantly higher on the word knowledge/vocabulary subtest of the gmrt than students who selected a narrative book as favorite. although student preference for a particular genre was not associated with reading comprehension per se, the fact that it was associated with vocabulary is noteworthy, given the importance of academic vocabulary to reading comprehension. that the text infusion/rfl instruction intervention had an impact was also suggested by the fact that students who chose an informational book as their favorite reported significantly higher perceived competence, interest, and overall motivation. in addition, students in this treatment condition who identified an informational book as a favorite reported more frequent out-of-school reading and more diversity of reading genres. no relations among these engagement variables were evident in the other two conditions. gender differences in reading, engaging, and learning in this section we address gender differences on the various outcome measures in the real project. gender is an individual difference variable of particular importance to this topic because it has often been argued that boys find informational text more appealing than do girls. if this is the case, one might expect boys to have differentially better comprehension of informational text than narrative text, or to be more motivated to read informational books than fiction. oakhill and petrides (2007) tested the possibility that the reason why boys in england showed a large reading comprehension increase from one administration of a national reading test to another is because the passage topic was more appealing to them. the topic the previous year had been about leaving home in wartime and that year it was about spiders. using a within-subjects design, they found that boys overwhelmingly preferred the passage about spiders and girls the passage about wartime evacuation. of particular relevance is that boys had better comprehension of the passage about the preferred topic, whereas girls had equally good comprehension of both passages. reading achievement. on major large-scale reading assessments, both national and international in scope, girls consistently outperform boys. for example, on the 2009 naep, fourth-grade girls had significantly higher scale scores than boys, and fewer scored below the basic level (30% vs. 36%). on the 2006 pirls, fourth-grade girls had significantly higher scores than boys in all but two of the 46 participating educational systems around the world (mullis et al., 2007). within the real project, reading achievement on the gmrt was consistently better for girls than for boys, across years and across gmrt subscales. the advantage was not statistically significant in grade two, but it was in grades three and four. across the full longitudinal sample, the main effect of gender on gmrt comprehension was significant, f (1, 104) = 10.55, p = .002, partial η2 = .09. the mean extended scale score for girls at the end of fourth grade was 458.58 whereas for boys it was 441.43. we also tested the possibility that the gender difference in reading comprehension might be attenuated when informational texts are used as opposed to fictional. using our constructed gmrt subscales, we found no evidence of an interaction of gender with text type, either when we compared informational vs. fiction, or when we compared expository and narrative-informational text types separately. in contrast, on the naep, girls had significantly better scores on both literary and informational text than did boys (see table 1). international electronic journal of elementary education 220 the advantage for girls on literary text was 8 scale points, but on informational text it was only 6. on the reading and writing to learn performance assessments, girls and boys performed comparably in second grade. girls scored higher than boys during third grade but the effect of gender was not statistically significant (p = .07), means = 14.60 and 13.75, respectively. the gender difference was statistically significant in grade 4, f (1, 84) = 5.69, p = .012, partial η2 = .06, with mean scores of 12.10 for girls and 10.78 for boys. although considerable evidence shows girls outperforming boys on tests of reading, we are not aware of any studies that have specifically addressed gender differences in information access. performance on the real text access tasks was better for girls than for boys, with statistically significant advantages in both years (second grade, f (1,173) = 5.82, p = .02, partial η2 = .033 and third grade, f (1, 155) = 7.84, p = .006, partial η2 = .048). mean proportions correct in grade 2 were .71 for girls and .61 for boys, and in grade 3, .78 and .69, respectively. reading motivation, reading activity, and book preferences. gender differences in motivation are often reported in the literature. for example, baker and wigfield (1999) found consistently higher scores for fifthand sixth-grade girls on the motivations for reading questionnaire. however, in the real project, boys and girls generally had comparable levels of motivation across all three years. similarly, there were no differences in perceived competence, despite the fact that girls earned objectively higher reading achievement scores than did the boys. at the final assessment point, at the end of grade 4, girls had a mean score of 2.57 and boys 2.47 out of 3 possible. clearly, all students had a positive appraisal of their competencies. girls typically report more frequent out-of school reading than boys (beall, 2011; baker & wigfield, 1999), but gender differences were absent on the reading activity inventory in the real project. gender differences are often reported in the types of books students prefer to read. for example, mohr (2003) found that 96% of first-grade boys selected nonfiction books for themselves, whereas 69% of the girls did so. in addition, oakhill and petrides (2007) reported that an unpublished survey of 10-11-year-olds found that boys preferred factual books and girls preferred storybooks. in contrast, chapman, filipenko, mctavish, and shapiro (2007) found that first graders showed no gender differences in their preferences for stories or informational books when choosing books for themselves. but when asked about what other children would prefer, both boys and girls thought that boys would prefer informational books whereas girls would prefer stories. in the real project as well, no gender differences were found in the types of books that were identified as favorites, indicating that informational books can be equally appealing to both boys and girls. however, when given a choice of which book they would like to receive as a “thank you” gift at the end of the project, more boys than girls chose an informational book, but overall, stories were chosen by 68% of the children. summary and conclusions the real project was designed to investigate whether a classroom intervention that enhances young children's experience with informational books would increase reading achievement and engagement. contrary to expectations, the intervention had minimal impact on student outcomes. students in second grade improved in gmrt reading comprehension from fall to spring to the same extent regardless of whether they had additional informational books in their classrooms and specific instruction in their use, and the same pattern obtained in grades three and four. the rwtl performance assessment, administered in both fall and spring every year, revealed improvement over the year in children’s comprehension of informational text / baker, et al. 221 second and third grades, but no differential progress related to the intervention. the one significant treatment effect was obtained on the text access task, which most closely parallels the instruction teachers were asked to provide. students in the text infusion/rfl instruction condition made greater gains in their ability to use text access features than students in the other two conditions. however, the treatment effect was found only in second grade, not in third. students declined in reading motivation during the course of second grade and into third grade, again regardless of intervention condition. motivation stabilized, showing no further declines in fourth grade. self-reported reading activity also declined over the first year of the project and stabilized subsequently. of importance is that students with increased access to informational books in the classroom reported more out-of-school reading across the years than students without the increased access. by the end of the project in fourth grade, the infusion of informational books into the classroom libraries appears to have influenced student preferences for reading material. students in both text infusion treatments were more likely to select an informational book as one of their two favorites for the year. students who selected informational books tended to do so because they satisfied a desire for information or taught the students something that was important to them. students who expressed a preference for informational books at the text infusion/rfl instruction school may also have experienced an increased interest in reading, as evidenced by more self-reported reading activity out of school. despite the lack of evidence of improved comprehension in the real project that can be attributable to the text infusion intervention, it should not be concluded that students will not benefit from increased access and instruction in using informational books in the early grades. substantial correlational evidence exists that should not be ignored, both within our own study and external to it. moreover, since the real project was conceptualized a decade ago, several empirical studies now provide evidence that interventions designed to increase children’s understanding and use of informational text result in growth in reading comprehension (e.g., guthrie, wigfield, barbosa et al., 2004; guthrie et al., 2007; o’hara, 2007; williams et al., 2005; williams et al., 2009). we intentionally chose not to change classroom practice dramatically, in order to gain greater buy-in on the part of the principals and the teachers. however, it appears that many teachers benefit from having very structured lessons prepared for them to deliver, and it is under these circumstances that intervention effects are most likely to be found (guthrie, wigfield, & perencevich, 2004; williams et al., 2005). the type and amount of professional development that will help teachers successfully enhance children’s comprehension of informational text may depend on the degree of change an intervention requires. taking a broad focus, guthrie, wigfield, barbosa et al., (2004) aimed at changing teachers’ approach to reading instruction entirely. during a 12week intervention, guthrie et al. had teachers devote their daily 90-minute reading block to concept-oriented reading instruction (cori). during cori, they were to teach reading strategies in the context of hands-on science theme units that afforded children access to interesting texts and allowed them to make choices and collaborate. to make such a change possible, guthrie et al. provided teachers with 10 days of professional development including instructional examples, materials, and time to plan lessons using a teacher’s guide and with assistance available. in contrast, some interventions have had a narrower focus. williams et al. (2009) focused on one aspect of informational text comprehension: comparecontrast text structure. they investigated the effect of teaching children to comprehend compare-contrast science texts in lessons that supplemented regular instruction. teachers international electronic journal of elementary education 222 taught 45-minute sessions 3 times a week for a total of 22 sessions. because teachers were supplied with detailed lesson plans and all necessary materials, professional development required only about 30 minutes. the issue of how best to help teachers provide effective instruction relevant to informational text is an important area of investigation. teachers are less comfortable with such text than with stories (donovan & smolkin, 2001; yopp & yopp, 2006), and the task of integrating informational text into literacy instruction is difficult for many. ness (2011) noted that although teachers held a favorable view of informational text, they nevertheless reported under-using it due to factors such as lack of time, lack of resources, and curriculum constraints. dreher and zelinke (2010) reached similar conclusions based on a review of literature, but also identified teacher knowledge as a major challenge. concerns cited in the literature include lack of knowledge about features and types of informational text, the availability of informational text, and the use of such text in instruction. to be successful in implementing instruction, teachers need knowledge about informational text. thus, both initial certification programs and professional development for practicing teachers should include more focus on the why and how of using this genre. however, even if teachers have extensive knowledge about informational text, they likely will need additional support to take full advantage of that knowledge. neuman and cunningham (2009) found that although a professional development course was effective in developing teacher knowledge, it did not result in improving teacher practice unless accompanied by coaching in which an expert helped a teacher learn to apply knowledge to practice. limitations in teacher knowledge and practice with respect to informational text may contribute to null effects on reading comprehension, not only in the real project but also in other recent interventions. in a large-scale study at fifth grade, james-burdumy et al. (2010) found no effect on reading comprehension for supplemental reading curricula, all of which targeted reading to learn. remarkably, these programs, with titles such as read for real and reading for knowledge, also did not lead to increased use of informational text. it seems likely that the lack of improvement in reading comprehension and the failure of teachers to increase the amount of informational text in their instruction are related. although the nature of the instruction itself is of course important, not just an increased use of informational text (purcell-gates, duke, & martineau, 2007), teachers who have difficulty including more informational text may also have difficulty providing optimal instruction on such texts. further research on what will assist teachers to increase their use of and facility with informational text may be particularly important if the common core state standards effort is to be successful. the standards, in various stages of implementation across most states in the us, call for a 50-50 mix of literary and informational text in elementary school, both in grades k-2 and in grades 3-6 (coleman & pimentel, 2011a; 2011b). although there is much still to investigate, the real project provides an informative lesson for those who call for randomized controlled trials as the gold standard of educational research. in her 2005 presidential address to the society for the scientific study of reading, joanna williams referred to the study to illustrate the challenges of conducting classroombased intervention research. although the schools were located within the same district, each school had a different ethos and experienced several unique events each year of the study. one school received additional funding because students failed to show improvement on state assessments, and the principal used this money to implement small class sizes. in addition, the text infusion alone school had a new principal in year 3 who decided to departmentalize instruction in the fourth grade. thus, all children in the study moved from children’s comprehension of informational text / baker, et al. 223 classroom to classroom for language arts, science/social studies, and mathematics, and they were seldom given access to the informational books provided except during homeroom periods. furthermore, the traditional instruction school became a magnet school for gifted and talented students during the course of the project, and many of the non-gifted students were transferred to other schools. and of course the small scale of the study meant that we were unable to control for teacher effects; teachers participating in the study varied dramatically in their experience and their effectiveness. the world of public schooling is such that it cannot be controlled. • • • acknowledgements the original research reported herein was funded by a grant from the spencer foundation to the first two authors. we gratefully acknowledge the foundation’s support. portions of this paper were presented at a symposium at the meeting of the society for the scientific study of reading in july 2005. linda baker is professor and chair of the department of psychology at the university of maryland, baltimore county. her research interests include motivation for reading, metacognition, prevention of reading difficulties, and parental influences on children’s school outcomes. she is a former associate editor of the journal of educational psychology, and she is a fellow of the american psychological association and the american educational research association. mariam jean dreher is professor of reading education at the university of maryland, college park. her research interests include the effects of integrating informational texts into literacy instruction on children’s comprehension, vocabulary, and motivation. she served as a fulbright senior specialist working to help early childhood teachers in finland explore those topics, and is currently a consultant to the national geographic society on a series of trade books for children. angela katenkamp shiplet received her ph.d. in applied developmental psychology and a certificate in american educational policy from the university of maryland, baltimore county in 2008. she served as the lead research assistant on the real project throughout its course. her current interests include the relations between the school and the home. anita n. voelker is an associate professor at messiah college in grantham, pa. she is currently researching how emergent readers respond to picture books on e-readers. lisa carter beall received her doctoral degree in applied developmental psychology from the university of maryland, baltimore county where she studied developmental factors impacting children’s learning. her dissertation examined the role of implementation intentions and choice of reading material on the voluntary reading behavior of fourth-grade students. she is currently involved in assessment of first-year programs that are designed to facilitate the transition of freshman and transfer students to umbc. adia j. garrett, ph.d. received her bs in psychology from morgan state university and a ph.d. in applied developmental psychology from the university of maryland, baltimore county (umbc). she is currently a lecturer at umbc who enjoys teaching courses in human development and research methods. her research interest in the educational contexts of development and her commitment to helping students achieve success has been the foundation for her work in course redesign. she has collaborated with colleagues campus-wide to enhance pedagogy with technology, promote student engagement, and foster peer-assisted learning. her most challenging, yet rewarding, job is being a new mom. international electronic journal of elementary education 224 heather r. schugar is an assistant professor of literacy at west chester university where she explores children's comprehension of informational texts and children's comprehension of interactive books available on e-reader devices. she first worked on the real project as a doctoral student at the university of maryland, college park. maria finger-elam is a doctoral student in the applied developmental psychology program at the university of maryland, baltimore county. her dissertation focuses on parent involvement programs to help boost children’s academic achievement. she is currently an evaluation specialist in the howard county (maryland) public schools office of research and program evaluation. references aarnoutse, c., & schellings, g. (2003). learning reading strategies by triggering reading motivation. educational studies, 29, 387-408. baker, l. & dreher, m. j. (2005, june). balancing learning to read and reading for learning: intervention effects on students’ achievement. in m. j. dreher & l. baker (chairs), does increasing the availability of informational text in urban classrooms improve young children's reading achievement and motivation? symposium presented at the meeting of the society for the scientific study of reading, toronto. baker, l., dreher, m. j., & guthrie, j. t. (eds.). (2000). engaging young readers: promoting achievement and motivation. new york: the guilford press. baker, l., & wigfield, a. (1999). dimensions of children's motivation for reading and their relations to reading activity and reading achievement. reading research quarterly, 34, 452-477. beall, l. c. (2011). implementation intentions as a means of promoting voluntary reading behavior (unpublished doctoral dissertation). university of maryland, baltimore county. beall, l. c., morse, f., baker, l., & dreher, m. j. (2005, june). student book preferences and their links to achievement. in m. j. dreher & l. baker (chairs), does increasing the availability of informational text in urban classrooms improve young children's reading achievement and motivation? symposium presented at the meeting of the society for the scientific study of reading, toronto. beck, i. l., mckeown, m. g., hamilton, r. l., & kucan, l. (1997) questioning the author: an approach for enhancing student engagement with text. newark, de: ira. best, r. m., floyd, r. g., & mcnamara, d. s. (2008). differential competencies contributing to children’s comprehension of narrative and expository texts. reading psychology, 29, 137-164. campbell, j. r., kapinus, b. a., & beatty, a. s. (1995). interviewing children about their literacy experiences: data from naep's integrated reading performance record (irpr) at grade 4. washington, dc: national center for educational statistics. caswell, l. c., & duke, n. (1998). non-narrative as a catalyst for literacy development. language arts, 75, 108-117. cervetti, g. n., bravo, m. a., hiebert, e. h., pearson, p. d., jaynes, c. a. (2009). text genre and science content: ease of reading, comprehension, and reader preference. reading psychology, 30, 487511. chall, j. s., jacobs, v. a., & baldwin, e. (l990). the reading crisis: why poor children fall behind. cambridge, ma: harvard university press. chambliss, m. j., & mckillop, a. m. (2000). creating a printand technology-rich classroom library to entice children to read. in l. baker, m. j. dreher, & j. t. guthrie (eds.), engaging young readers: promoting achievement and motivation (pp. 94-118). guilford press. chapman, j. w., & tunmer, w. e. (1997). a longitudinal study of beginning reading achievement and reading self-concept. british journal of educational psychology, 67, 279-291. chapman, m., filipenko, m., mctavish, m., & shapiro, j. (2007). first graders’ preferences for narrative and/or information books and perceptions of other boys’ and girls’ book preferences. canadian journal of education 30, 531‐553. children’s comprehension of informational text / baker, et al. 225 coleman, d., & pimentel, s. (2011a). publishers’ criteria for the common core state standards in english language arts and literacy, grades k–2. retrieved from http://www.corestandards.org /assets/publishers_criteria_for_k-2.pdf coleman, d., & pimentel, s. (2011b). publishers’ criteria for the common core state standards in english language arts and literacy, grades 3–12. retrieved from http://www.corestandards.org/ assets/publishers_criteria_for_3-12.pdf common core state standards (ccss). (2010). common core state standards for english language arts & literacy in history/social studies, science, and technical subjects. retrieved from http://www.corestandards.org/the-standards culatta, b., hall-kenyon, k. m., & black, s. (2010). teaching expository comprehension skills in early childhood classrooms. topics in language disorders, 30, 323-338. depaola, t. (1984). the popcorn book. new york: holiday house. diakidoy, i. n., stylianou, p., karefillidou, c., & papageorgiou, p. (2005). the relationship between listening and reading comprehension of different types of text at increasing grade levels. reading psychology, 26, 55-80. donovan, c. a., & smolkin, l. b. (2001). genre and other factors influencing teachers' book selections for science instruction. reading research quarterly, 36, 412–440. dreher, m. j. (2000). fostering reading for learning. in l. baker, m. j. dreher, & j. t. guthrie (eds.), engaging young readers: promoting achievement and motivation (pp. 68-93). new york: guilford press. dreher, m. j., & baker, l. (2005, june). balancing learning to read and reading for learning: infusing information books into primary-grade classrooms. in m. j. dreher & l. baker (chairs), does increasing the availability of informational text in urban classrooms improve young children's reading achievement and motivation? symposium presented at the meeting of the society for the scientific study of reading, toronto. dreher, m. j., & voelker, a. (2004). choosing informational books for primary-grade classrooms: the importance of balance and quality. in e. w. saul (ed.), crossing borders in literacy and science instruction: perspectives on theory and practice (pp. 260-276). newark, de: international reading association & arlington, va: the national science teachers association. dreher, m. j., & zelinke, s. b. (2010, december). informational text: why is it still neglected in so many elementary classrooms? paper presented at the literacy research association, fort worth, tx. duke, n. k. (2000). 3.6 minutes per day: the scarcity of informational texts in first grade. reading research quarterly, 35, 202-224. duke, n., k., & carlisle, j. (2011). the development of comprehension. in m. kamil, p. d. pearson, e. b. moje, & p. p. afflerbach (eds.), handbook of reading research (vol. 4, pp. 199-228). new york: routledge. foorman, b. r., & connor, c. m. (2011). primary grade reading. in m. kamil, p. d. pearson, e. b. moje, & p. p. afflerbach (eds.), handbook of reading research (vol. 4, pp. 136-156). new york: routledge. gamse, b. c., jacob, r. t., horst, m., boulay, b., & unlu, f. (2008). reading first impact study: final report (ncee 2009-4038). washington, dc: national center for education evaluation and regional assistance, institute of education sciences, u.s. department of education. guthrie, j. t., hoa, a. l. w., wigfield, a., tonks, s. m., humenick, n. m., & littles, e. (2007). reading motivation and reading comprehension growth in the later elementary years. contemporary educational psychology, 32, 282–313. guthrie, j. t., wigfield, a., barbosa, p., perencevich, k. c., taboada, a., davis, m. h., scafiddi, n. t., & tonks, s. (2004). increasing reading comprehension and engagement through conceptoriented reading instruction. journal of educational psychology, 96, 403-423. guthrie, j. t., wigfield, a., & perencevich, k. c. (2004). motivating reading comprehension: conceptoriented reading instruction. mahwah, nj: erlbaum. hall, k. m., sabey, b. l., & mcclellan, m. (2005). expository text comprehension: helping primary-grade teachers use expository texts to full advantage. reading psychology, 26, 211-234. hoffman, j. v. (1992). critical reading/thinking across the curriculum: using i-charts to support learning. language arts, 69, 121-127. hoffman, j. v., mccarthey, s. j., abbot, j., christian, c., corman, l., curry, c. (1994). so what’s new in the new basals? a focus on first grade. journal of reading behavior, 26, 47-73. international electronic journal of elementary education 226 horowitz, r., & freeman, s. h. (1995). robots versus spaceships: the role of discussion in kindergartners’ and second graders’ preferences for science texts. the reading teacher, 49, 3040. ingham, j. (1982). books and reading development: the bradford book flood experiment (2nd ed.). portsmouth, nh: heinemann educational books inc. international reading association and national council of teachers of english (1996). standards for the english language arts. newark, de: international reading association. james-burdumy, s., deke, j., lugo-gil, j., carey, n., hershey, a., gersten, r., newman-gonchar, r., dimino, j., & haymond, k. (2010). effectiveness of selected supplemental reading comprehension interventions: findings from two student cohorts (ncee 2010-4015). washington, dc: national center for education evaluation and regional assistance, institute of education. jeong, j., gaffney, j. s., & choi, j.-o. (2010). availability and use of informational texts in second-, third-, and fourth-grade classrooms. research in the teaching of english, 44, 435-456. katenkamp, a., garrett, a., & baker, l. (2005, june). opportunities to read in the classroom: observations of reading activities in grades 2-4. in m. j. dreher & l. baker (chairs), does increasing the availability of informational text in urban classrooms improve young children's reading achievement and motivation? symposium presented at the meeting of the society for the scientific study of reading, toronto. kletzien, s. b., & dreher, m. j. (2004). informational text in k-3 classrooms: helping children read and write. newark, de: international reading association. lewis, m., wray, d., & rospigliosi, p. (1994). “ . . . and i want it in your own words.” the reading teacher, 47, 528-536. macginitie, w. h., macginitie, r. k., maria, k., & dreyer, l. g. (2000). gates-macginitie reading test (4th ed.). itasca, il: riverside publishing. macginitie, w. h., macginitie, r. k., maria, k., & dreyer, l. g. (2002). gates-macginitie reading tests (4th ed.): technical report, forms s and t. itasca, il: riverside publishing. mckenna, m. c., ellsworth, r. a., & kear, d. j. (1995). children’s attitudes towards reading: a national survey. reading research quarterly, 30, 934-956. mohr, k. a. j. (2003). children's choices: a comparison of book preferences between hispanic and nonhispanic first-graders. reading psychology, 24, 163-176. moss, b. (2008). the information text gap: the mismatch between non-narrative text types in basal readers and 2009 naep recommended guidelines. journal of literacy research, 40, 201-219. moss, b., & newton, e. (2002). an examination of the informational text genre in recent basal readers. reading psychology, 23, 1-13. mullis, i. v. s., martin, m. o., kennedy, a. m., & foy, p. (2007). pirls 2006 international report: iea’s progress in international reading literacy study in primary schools in 40 countries. chestnut hill, ma: timss & pirls international study center, boston college. national assessment governing board. (2008). reading framework for the 2009 national assessment of educational progress. washington, dc: us government printing office. national center for education statistics (nces). (2009). the nation’s report card: reading 2009 (nces 2010–458). washington, dc: institute of education sciences, u.s. department of education. national council for the social studies (ncss). (1994). expectations of excellence: curriculum standards for social studies. washington, dc: author. national council of teachers of mathematics (nctm). commission on standards for school mathematics. (1989). curriculum and evaluation standards for school mathematics. reston, va: author. national research council. (1994). national science education standards: an enhanced sampler. washington, dc: national academy press. ness, m. (2011). teachers' use of and attitudes toward informational text in k-5 classrooms. reading psychology, 32, 28-53. neuman, s. b. (2001). the role of knowledge in early literacy. reading research quarterly, 36, 468-475. neuman, s. b. (2006). the knowledge gap: implications for early education. in d. k. dickinson & s. b. neuman (eds.), handbook of early literacy research (pp. 29-40). new york: guilford. neuman, s. b. (2010). lessons from my mother: reflections on the national early literacy panel report. educational researcher, 39, 301-304. children’s comprehension of informational text / baker, et al. 227 neuman, s. b., & cunningham, l. (2009). the impact of professional development and coaching on early language and literacy instructional practices. american educational research journal, 46, 532-566. oakhill, j. v., & petrides, a. (2007). sex differences in the effects of interest on boys’ and girls’ reading comprehension. british journal of psychology, 98, 223–235. o’hara, j. d. (2007). the influence of supplemental instructional approaches upon the comprehension, metacognitive awareness, and motivation of struggling thirdand fourth-grade readers (unpublished doctoral dissertation). university of maryland, college park. pappas, c. c. (1993). is narrative “primary”? some insights from kindergartners’ pretend readings of stories and information books. journal of reading behavior, 25, 97-129. purcell-gates, v., duke, n. k., & martineau, j. a. (2007). learning to read and write genre-specific text: roles of authentic experiences and explicit teaching. reading research quarterly, 42, 8-45. ruetschlin, h., finger, m., & dreher, m. j. (2005, june). children’s reading comprehension in grades 2-4 across genre and question type. in m. j. dreher & l. baker (chairs), does increasing the availability of informational text in urban classrooms improve young children's reading achievement and motivation? symposium presented at the meeting of the society for the scientific study of reading, toronto. salinger, t., kamil, m. l., kapinus, b., & afflerbach, p. (2005). development of a new framework for the naep reading assessment. national reading conference yearbook, 54, 334–349. serpell, r. s., baker, l., & sonnenschein, s. (2005). becoming literate in the city: the baltimore early childhood project. new york: cambridge. smolkin, l., & donovan, c. (2001). the contexts of comprehension: the information book read aloud, comprehension acquisition, and comprehension instruction in a first-grade classroom. the elementary school journal, 102, 97-122. snow, c. e., burns, m. s., & griffin, p. (eds.). (1998). preventing reading difficulties in young children. washington, dc: national academy press. taylor, b. m., & pearson, p. d. (2001). the ciera school change classroom observation scheme. university of michigan: center for the improvement of early reading achievement. venezky, r. l. (2000). the origins of the present-day chasm between adult literacy needs and school literacy instruction. scientific studies of reading, 4, 19-39. white, s., chen, j., & forsyth, b. (2010). reading-related literacy activities of american adults: time spent, task types, and cognitive skills used. journal of literacy research, 42, 276-307. wigfield, a., eccles, j. s., yoon, k. s., harold, r. d., arbreton, a., freedman-doan, c., & blumenfeld, p. b. (1997). change in children’s competence beliefs and subjective task values across the elementary school years: a three-year study. journal of educational psychology, 89, 451-469. wigfield, a., & guthrie, j. t. (1997). relations of children’s motivation for reading to the amount and breadth of their reading. journal of educational psychology, 89, 420-432. williams, j. p., hall, k. m., lauer, k. d., stafford, k. b., desisto, l. a., & decani, j. s. (2005). expository text comprehension in the primary grade classroom. journal of educational psychology, 97, 538550. williams, j. p., stafford, k.b., lauer, k. d., hall, k. m., & pollini, s. (2009). embedding reading comprehension training in content-area instruction. journal of educational psychology, 10, 120. yopp, r. h., & yopp, h. k. (2006). informational texts as read-alouds at school and home. journal of literacy research, 38, 37-51. the concurrent development of spelling skills in two languages international electronic journal of elementary education vol. 3, issue 2, march, 2011. issn:1307-9298 copyright © iejee www.iejee.com collaborative teaching of an integrated methods course george zhou* university of windsor, canada jinyoung kim the city university of new york, united states of america judit kerekes the city university of new york, united states of america abstract with an increasing diversity in american schools, teachers need to be able to collaborate in teaching. university courses are widely considered as a stage to demonstrate or model the ways of collaboration. to respond to this call, three authors team taught an integrated methods course at an urban public university in the city of new york. following a qualitative research design, this study explored both instructors‟ and pre-service teachers‟ experiences with this course. study findings indicate that collaborative teaching of an integrated methods course is feasible and beneficial to both instructors and pre-service teachers. for instructors, this collaborative teaching was a reciprocal learning process where they were engaged in thinking about teaching in a broader and innovative way. for pre-service teachers, this collaborative course not only helped them understand how three different subjects could be related to each other, but also provided opportunities for them to actually see how collaboration could take place in teaching. their understanding of collaborative teaching was enhanced after the course. keywords: collaborative teaching; integration; methods course; elementary teacher education. introduction collaborative work is defined as two or more people working together. effective collaboration is mandatory for success in the context of a workplace such as today‟s business environment (beyerlein & harris, 2003). in the area of education, scholars and practitioners have advocated the importance of collaboration as well for a while. as a result, collaboration between university * correspondence: dr. george zhou, faculty of education, university of windsor, 401 sunset ave. windsor on canada n9b 3p4. telephone: (519)253 3000 ext 3813. e-mail: gzhou@uwindsor.ca http://www.iejee.com/ mailto:gzhou@uwindsor.ca international electronic journal of elementary education vol.3, issue 2, march, 2011 124 and industry, college and community, and teacher education institutions and k-12 schools is no longer new to many people. however, collaborative teaching within k-12 schools appears to be an untapped area. the current categorical approach to teacher preparation and lack of attention to collaborative skills are the first barriers to effective collaboration in schools (villa, thousand, nevin, & malgeri, 1996). friend (2000) reminds us that collaborative skills do not come naturally; they need to be honed and cultivated. many scholars therefore suggest that university courses should be employed as the stage where pre-service teachers are exposed to various collaborative practices (kluth & straut, 2003; quinlan, 1998). to respond to this call for collaboration in teacher education courses, three authors collaboratively taught one multidisciplinary methods course (ede 303) for three semesters at an urban public university in the city of new york. this course was designed for an elementary teacher education program. it covered three subjects: science, math, and music. the fundamental purpose of this course was to develop pre-service teachers‟ pedagogical content knowledge in three subjects through an integrated approach. given the innovative nature of this course, three instructors conducted a self study over three semesters to answer the following question: how does this course impact pre-service teachers‟ perspective of collaborative teaching? in addition, this study documented how three instructors worked together during the course and reported their findings about the benefits and challenges of such collaboration. literature review collaborative teaching happens when two or more educators take responsibility for planning, teaching, and monitoring the success of learners in a class. each instructor contributes to the class based on his or her experience and expertise. particularly, since many new programs/courses emerge out of more than one traditional discipline, faculty members find it necessary to combine their expertise in order to address the needs of these courses or programs (kulynch, 1998). collaborative teaching can also happen when faculty work together planning several classes as “cluster courses” (dugan & letterman, 2008). in this case, the clustered courses share the same large issues or one course serves as a base for another course. for example, potterfield and majerus (2008) described the collaboration between a physiology class and a statistical class. real data such as heart rate, blood pressure, and lung volume collected by the physiology students were provided to the statistical class. two classes shared their investigation through multiple formats including a course website, large group discussion, and final presentations. although collaborative teaching can happen within one course or between courses, the literature often focuses on the one-course case. such collaboration can take place in different formats. vogler and long (2003) summarize various types of collaboration including: 1) faculty from diverse collaborative teaching of an integrated methods course / zhou, kim & kerekes 125 departments teaching an interdisciplinary course, 2) faculty from the same department teaching different sections of the same course by individually rotating section to section, repeating lectures in their areas of expertise, 3) team members presenting together in all sections of the course. helms, alvis, and willis (2005) describe three team teaching styles: the interactive model, the participant-observer model, and the rotational model. in the interactive approach of collaboration, collaborators participate in the lecture or activities together with a great deal of interaction and dialogue between them and their students. the participant-observer model requires collaborators to be present simultaneously in the class, but with one independently teaching while the other observes (the collaborators alternate the teacher and observer roles). the observing faculty interacts only when asked questions. under the rotational model, collaborators teach separately and attend class only when teaching their specific areas of the course. this model involves less interaction between collaborators and less integration of course materials. recent studies of collaboration in teaching have suggested that collaborative work is beneficial to both students and instructors. for students, collaborative teaching can foster their interest and enthusiasm (hinton & downing, 1998; letterman & dugan, 2004), improve their achievements (johnson, johnson, & smith, 2000), enhance their team work abilities (kapp, 2009), and promote their interdisciplinary learning (davis, 1995; letterman & dugan, 2004; wilson & martin, 1998). for instructors, collaborative teaching provides them with opportunities to be engaged in more philosophical discussions and to learn from each other‟s experiences and teaching styles (davis, 1995; letterman & dugan, 2004; robinson & schaible, 1995). particularly, collaborative teaching is beneficial for both students and instructors when it promotes diversity by including teaching members from different disciplinary areas in addition to different ethnic and cultural backgrounds (hinton & downing, 1998). in teacher education, collaboration between the education faculty and k12 schools is gaining popularity and is even mandatory in many places. the idea of schools as teaching practice clinics has been adopted by a number of teacher education institutions. school teachers are invited into teacher education classrooms as guest speakers or collaborative teachers. education faculty members go to schools to supervise student teaching, teach courses at the school site (sluss & minner, 1999; surbeck, 1994), and/or provide mentorship to classroom teachers (justiz, 1997). studies of these collaborations have documented improvement in the development of preservice teachers‟ knowledge and skills, the relationship between schools and universities, and the mutual support and respect between faculty and classroom teachers (freeman, 1993). another popular type of collaboration in teacher education is between general education faculty and special education faculty (murawski & swanson, 2001). given the increasing diversity in american schools in terms of learning ability, social-economic status, ethnicity, and culture, education international electronic journal of elementary education vol.3, issue 2, march, 2011 126 faculty members have found that it is hard to be effective when delivering teacher education in isolation. teacher educators who came from different disciplines and differ in cultural backgrounds and research expertise need to teach together in order to prepare pre-service teachers for inclusive instruction. kluth and straut (2003) report a collaborative case of this type including two instructors, one from special education and the other from general education. in two college courses, they co-taught most of the sessions modeling different types of co-teaching such as parallel teaching, station teaching, and one teach/one assist models. in parallel teaching structure, they split the large class into equal sections and chose one of two following options. they either provided each group with the same lesson or activity carried out simultaneously by the two faculty members or they individually taught different topics to a group of students and then switched the student groups and repeated the lesson. in station teaching structure, they divided instructional content into segments and presented the content concurrently at separate locations within the classroom. in the one teach/one assist model of collaboration, one served as the main instructor, and the other acted as an assistant who facilitated group work or provided assistance to individual students in the class. the collaboration reported in this paper represents a different rationale for collaboration, namely integrated curriculum among traditional subjects such as science, math, and music. curriculum integration was proposed in a contrast to the conventional school subjects that were designed to parallel major academic disciplines of mathematics, science, arts, philosophy, and humanities. one of the most cited reasons for curriculum integration is the disconnection between a discipline-based curriculum and the real world. cumming (1994) claimed that this disconnection between a disciplinary curriculum and the real world causes students to think school education is irrelevant to their life experience. another argument for curriculum integration comes from a unified view of knowledge. more than thirty five years ago, hirst (1974) suggested that an integrated curriculum could be justified through a holistic view of knowledge, which looks at knowledge as connected, embodied, ecological, and harmonized. employing this view of knowledge, perkins (1991) criticized individual school disciplines as artificial partitions with historical roots of limited contemporary significance. a third angle that integration supporters take is to look at the way students learn. the disciplinary curriculum is based on the assumption that students will get a holistic picture of knowledge after they learn its parts. this mechanical and analytical point of view has been criticized by scholars who believe that individuals construct knowledge holistically, based upon their life experiences (bredekamp, 1987; mid-continent regional educational laboratory, 1993). in the last two decades of the 20th century, a number of national science and mathematics educational associations such as the american association for the advancement of science (1998), national research council (1996), national council of teachers of mathematics (2000), and national science collaborative teaching of an integrated methods course / zhou, kim & kerekes 127 teachers association (1997), began recommending the use of integrated curriculum as a tool for education reform. integrated curriculum has since become increasingly popular in the field of education (berlin & lee, 2005). as a result of this movement, few of today‟s educators would argue against the need for an integrated curriculum. however, for many teachers the implementation of curriculum integration is still not an easy job. they are simply not prepared for it. most teachers took disciplinary curricula at postsecondary institutions where subjects were taught separately. they had no opportunity to think of the connectedness between disciplines. particularly, the methods courses they took from teacher education programs were often arranged by subjects. they received little training to teach subjects in an integrated way. therefore, although elementary teachers usually teach multiple subjects and have the convenience to integrate them in teaching, they fail to take the opportunity. in order to prepare pre-service teachers to teach an integrated curriculum in elementary schools, the three authors with backgrounds in science, math, and music respectively, collaborated in teaching an integrated methods course. to our knowledge, there is limited research in the literature regarding this type of collaboration, which makes our study unique and significant. course design and description the course was a multidisciplinary methods course designed to equip preservice teachers with knowledge and skills that are essential for integrated instruction of math, science, and music in elementary schools. it involved field teaching experience as well as university classes. for the first five weeks of the course, the whole class met at the university three times a week: tuesday morning and afternoon (two sessions) and thursday morning (one session). each session lasted three hours and focused on one of these three subjects. beginning in the sixth week of the course, the class was randomly divided into three groups. each group met twice a week led by one of the three instructors. on thursdays, the instructor facilitates his or her group at the university to prepare a lesson in his or her specialized subject area. the lesson topics were pre-determined and published in the syllabus. the preservice teachers were required to think of the topic ahead of time so that they came to the thursday class with their own draft lesson plans or ideas for teaching this topic. a final agreed-upon lesson plan was developed through the class discussion. on the following tuesday, the group went to their assigned schools to teach their prepared lesson and spent the rest of day observing classroom teachers. each participant from the same group taught concurrently a group of pupils at a large area, such as the student lunch hall. the assigned instructor of this group observed their teaching practice every time. immediately following the observation, while still on the school site, the instructor debriefed the group about their teaching. this pattern of thursday prep and tuesday execution was continued in three week segments, rotating international electronic journal of elementary education vol.3, issue 2, march, 2011 128 for each of the three subjects so that each pre-service teacher taught three lessons for each subject. as described above, this course involved field teaching experience as well as university classes. while a combination of university courses and field experiences is common in teacher education programs, our integration of field experience with university classes in a single methods course is quite innovative. in the practice of teacher education, university methods course instructors and pre-service teachers‟ faculty advisors for their teaching practicum are often different people. this has the potential to create inconsistencies between what is taught in university courses and what is advised in school teaching practice. this concern, however, did not exist for our collaborative methods course. the three instructors helped the preservice teachers prepare the lessons, and then observed how they implemented these lessons in the school classrooms. this arrangement allowed the instructors to examine whether pre-service teachers understood and appropriately applied what they learned in their coursework. it also provided instructors with the opportunity to modify their university lessons for pre-service teachers‟ needs. research design and data collection participants came from the university‟s science, letter and society program, specially designed for undergraduate students who aimed to become elementary teachers. the program engaged university students, mostly females, in a balanced curriculum between academic disciplines including science, arts, social studies, and humanities before they registered for pedagogy courses. this study was conducted over three sequential semesters in the department of education. all pre-service teachers enrolled in the pedagogical course described above participated in the study. in the first semester, the class size was 25 with one male. in the second semester, the class had 22 pre-service teachers, all females. the third semester had the largest enrollment, 47, with two males. this study had an explorative nature and therefore employed a qualitative research design (creswell, 2008). reflective journals, field notes, and meeting minutes were the data source. in the first semester, student participants were asked to write reflective journals at the beginning of the course, after each lesson, and at the end of semester. to reduce the course workload, participants in the second and third semesters were asked to write only initial and exit reflections at the beginning and end of the course, respectively. in their initial reflective journals, participants were asked to respond to several questions regarding their knowledge competency in each subject, the skills they had to teach each subject, where they had developed the knowledge and skills, their interest in each subject, confidence in teaching it, and initial perspectives of collaborative teaching and curriculum integration. in their after-lesson reflections and particularly exit reflections, participants were asked to write what they had learned from the course in collaborative teaching of an integrated methods course / zhou, kim & kerekes 129 terms of subject knowledge, skills to teach each subject, and confidence to teach it, as well as any changes they experienced regarding their perspectives of collaborative teaching. for the instructors, brief minutes were taken for their meetings at various stages of this course to record the discussed issues, emerging ideas, and agreed-upon decisions of each meeting. each instructor also noted down his or her experience when observing collaborators‟ teaching. in addition, at the end of each semester, the three instructors reflected on their collaboration. a few questions were used to guide the scope of their reflections including what they learned about each other‟s subjects and their collaboration. a content analysis approach was employed to analyze the pre-service teachers‟ reflective journals (berg, 2009). we first used open coding to annotate each participant‟s journals with regard to the topics described above. then, we focused on the segments that report participants‟ experiences and perspectives of collaborative teaching. the following themes were identified: participants‟ learning experiences with this collaborative course and the reported changes in their perspectives of collaborative teaching. instructors‟ meeting minutes, observation notes, and reflective journals were analyzed through a similar approach, with a focus on their learning through the study. data coding was cross-reviewed by two researchers. findings and discussion working together for the benefit of instructors and students at the university where this study took place, elementary teacher education program designs methods courses in an integrated format due to the limited number of program credits and the concern of curriculum integration. it offers two methods courses to cover subject pedagogy: social studies, art, and language arts in elementary education (6 credits) and mathematics, science, and music in elementary education (6 credits). the later course had been offered to pre-service teachers for over ten years before this study took place, however it was primarily taught as three separate methods courses with little connection addressed between the subjects. when the three authors took over the course, they decided to make it more of an interdisciplinary course. they met several times during the university break to prepare and discuss the course before the first semester of this study. during this initial planning, one comprehensive syllabus was developed to replace the three separate syllabi used in the past. the syllabus clearly described the nature of collaboration and integration of the course, and created a common, parallel curriculum sequences and assignments for the three subjects. the assignment guidelines were also included in the syllabus so that the pre-service teachers could follow the directions no matter which group they were in. the schedule for the different groups was listed in international electronic journal of elementary education vol.3, issue 2, march, 2011 130 a table format. the three instructors decided to use the blackboard learning management system as a convenient communication tool. the course outline and assignment requirements were posted on blackboard. pre-service teachers could electronically upload all of their assignments, which included lesson plans, observational papers, and reflective journals. the use of blackboard was also beneficial to the instructors because, through its online grading system, they could easily allocate the grading workload. more importantly, the three instructors made an effort to identify connections or overlaps between the subjects and coordinate their curriculum sequence accordingly. in addition to the collaboration in the course planning, the three instructors met regularly throughout the semester on thursdays after the university class, particularly during the first semester of this study, to discuss the course progress. additionally, when there was a need to discuss emergent course-related issues, conferencing was conducted face-to-face and via email for the purpose of idea sharing and decision making. most meetings took place at the lunch hour in their offices, lunch room, or restaurants in a format of formal/informal dialogues. by having lunch together, they gained the opportunity to get to know each other through informal conversation and shared thoughts that might have not come up during formal meetings. meeting and eating together built a close personal relationship among the instructors and provided them with excellent opportunities to share teaching ideas and get to know each other‟s teaching, subjects, personality, and family and cultural background. for example, during one lunch meeting, the music instructor and math instructor shared their understanding of the connection between musical notes and the concept of fraction in math. the results of the discussion were implemented in the following music session to facilitate students‟ understanding of musical notes such as half, quarter, or eighth note symbols. to better enhance collaboration, the three instructors observed each other‟s sessions at least twice in one semester and recorded brief observation notes and reflections. the observer could join in the class discussion as well or even act as a discussion leader when the topics were relevant to his or her subject. for example, when the music instructor observed the science session on pendulum, she was called on by the science instructor to link the pendulum with the musical instrument “metronome” and demonstrate the integration between science and music. as she held the pendulum at different lengths (resulting in different frequencies of swing), pre-service teachers were asked to sing a common children‟s song along with her in a pace that matched the frequency of the pendulum. peer observations made the three instructors familiar with each other‟s teaching styles and instructional emphases, and more important, they often resulted in new ideas about integration between sessions and subjects. for example, while the math instructor was observing, the science instructor discussed constructivism in one morning session on inquiry-based learning. collaborative teaching of an integrated methods course / zhou, kim & kerekes 131 during the math session in the afternoon of the same tuesday, the math instructor referred to what pre-service teachers had learned from the science session about constructivism and used it to set up the theoretical platform for her math instruction. another example entails the science instructor‟s observation of a music session. in the middle of the class, the music instructor commented how the different thickness of string would generate sounds with various pitches and the length of string will matter as well. at this moment, the science instructor realized the connections between this comment and what he taught in one science session. he politely joined in the class discussion by questioning pre-service teachers: “does the thickness of the string influence the frequency of a pendulum?” scientifically speaking, the pitch was related to vibration and resonation. different types or sizes of materials will vibrate differently and therefore generate differing sounds. therefore, the thickness of string does matter in the generation of the sound. however, the scientific model of a pendulum takes the string as an imaginative line. the thickness of the string is not a concern of the scientific description of a pendulum. this episode helped students understand the connections and differences between music and science and become aware of the limitations of science. to assist the music instructor with pre-service teachers‟ full understanding of the fact that varying lengths and thickness of a string can generate different sounds, the science instructor took it upon himself to relate the science concepts to the music session. he changed his plan for the next science session in order to teach pre-service teachers scientific understanding of vibration and resonation so that they would understand music concepts better. he believed that in-depth knowledge about vibration and sound would help pre-service teachers make sense of what they were playing in the music sessions. the benefit of this observation was clear to the science instructor: questions generated from other subjects created moments or topics for his science session to cover. his modification to the pre-planned curriculum was necessary for the generation of a holistic understanding among pre-service teachers about what they learned from different subjects. constructivists suggest that teachers should let students‟ learning drive what they teach (von glaserfeld, 1995; zhou, 2010). these constructivist notions were clearly reflected in the science instructor‟s reaction. to further the collaboration, the science and music instructors discussed the possibility to develop a joint session on vibration, sound, etc. for the coming semester. instructors’ reflections on the collaboration all three instructors agreed that the collaboration was a process to learn about “working together” as well as “collaborative teaching.” they found that they shared very similar teaching philosophy and possessed a constructivist teaching style. through this collaboration, they were excited to learn that there were many connections between the three subjects. observation, international electronic journal of elementary education vol.3, issue 2, march, 2011 132 reflection, and discussion helped the three instructors gain ideas to connect one subject to another and made it possible for them to teach beyond what was originally planned so that their teaching better met pre-service teachers‟ needs. the three instructors‟ collaboration in this course happened both outside and inside the classroom. outside the classroom, they met for planning and discussion. inside the classroom, they taught through two collaboration models described by kluth and straut (2003): parallel collaboration model, where each of them taught a session in his or her subject area, and one teach/one assist model, where one of them taught the class and the other one facilitated discussion or group work. they also tried some joint sessions as well. for example, three instructors taught a joint session on integrated curriculum at the beginning of the semester. all three instructors felt that they had the desire and interest to develop more joint sessions, such as measurement (math) and matter property (science), sound (science) and pitch (music), notes (music) and fraction (math), etc. so they can model various collaborations to pre-service teachers. the science instructor, who was then a new faculty member at the university, reflected his great appreciation of the benefits the collaboration generated for him. at the end of the first semester of this study, he described the collaboration with two experienced faculty members as a process of being mentored: as a new faculty member, the complexity of this course was initially overwhelming to me. it involves collaboration between three instructors, connections between three subjects, and combination of university learning and school experience. it took me a while to understand how the rotation works between three subjects and three host schools. collaboration with two veteran instructors definitely helped me pass the hurdle. the math and music instructor, who taught this course before, were happy to see the differences this collaboration generated to the course. they appreciated the fresh ideas the science instructor brought into the courses. the music instructor wrote in her reflection: i had been articulating my music sessions only in terms of musical knowledge and skills before the collaboration because i was not teaching mathematics nor science sessions. the collaboration made me see the course in a more integrated way. although i knew that the concepts of musical note symbols could be related to the fraction concept in math, i didn‟t know how i could relate musical concepts to science. in this sense, the science faculty gave me many great insights. pre-service teachers’ reflections on collaborative teaching most participants‟ comments indicated that they had little difficulty getting used to this new format of methods course and applauded the fresh ideas and unique experience this course provided for them. their positive feedbacks confirmed the feasibility of formatting methods courses in a new way through combining: (a) university classes and school teaching experience and (b) collaborative teaching of an integrated methods course / zhou, kim & kerekes 133 multiple subjects. the following are two typical comments from participants‟ final reflective journals: overall, the format of the course was something positively different. combining the class is a good experience but can also be frustrating. it is hard to focus on one subject when you know you have two other teachers trying to show you different material all in the same week. that was something i had to get used to over the first few weeks of lecture class. i have always enjoyed collaborative work when it comes to teaching lessons. the experience for me was great and it did really help me understand what it is going to be like in a classroom environment. ede 303 is a unique course. it is a course taught by three different faculties and three different subjects in one course. although ede 303 separated science, math and music into three different sessions; it integrated them together. while focusing on one subject, another subject was integrated in the lesson. it was amazing how these subjects related. i never realized that all subjects can be related and integrated. i believe integrated curriculum will help students improve their studies. pre-service teachers stated that this course prepared them to better teach children. they greatly appreciated the opportunity to work with an instructor in a smaller group while preparing their lessons at the university and be supervised at the school by the same instructor: i also like the idea that we have thursday's class to prepare us for the future lesson. that helped me to make sure that i was ready to teach, and i had all weekend to gather materials and create an original lesson. i feel that all three professors did a good job in teaching their subjects, and teaching us about integrated curriculum. this was the first collaborative course [our italics] that i have taken at the college and i found it to be useful… it taught me what to teach and how to teach. i thought that taking the three different subjects as well as going to three different schools was good because now at the end of this course i feel better informed and that i know more about math, science, and music. i also feel that it gave me the opportunity to work with different age levels and different populations. through this collaborative course … i do believe that the collaborative course was beneficial... the course modeled how to work together in teaching. as kluth and straut (2003) point out, university teaching, particularly methods courses, has direct influence on pre-service teachers‟ understanding of teaching. faculty collaboration in university teaching impacts future teachers‟ perspectives of collaborative teaching and motivates them to teach collaboratively at schools. many participants applauded the collaboration during the course as they stated in the following comments: ede 303 did an amazing job in integrating the different subjects. it was very useful and interesting. this course was good because it helped me to better understand integrated curriculum. although three different professors taught the course, but they all worked together and integrated their lessons. the professors worked very well together in order to help make the course feel like it is being taught by one professor instead of three. the professors followed the same guideline and they made a good team. international electronic journal of elementary education vol.3, issue 2, march, 2011 134 the collaborative work in this course helped me understand how important it is to be able to work together. when working in a school, i must be able to work together with other teachers and staff, this collaborative course helped me collaboratively work with others. i was able to listen to others and share opinions. not only did this course demonstrate the connections between subjects to pre-service teachers, but it also modeled the way to integrate them in school teaching. students‟ understanding of curriculum integration was enhanced by the end of the course: at the beginning of the semester, when we were asked about integrated curriculum, i really didn‟t understand nor had much information about it. now i have learned how important integrated curriculum is in our schools. it is important for the teachers to connect the subjects. the students will be able to understand the subjects better. this course has helped me better understand integrated curriculum, because each subject ties into one another somehow. there are many mathematical components in music, such as beats and rhythms... throughout each class, i've heard all three professors mention something about integrated curriculum. i feel that all three teachers have helped me to understand, as well as better prepare me, for integrated curriculum. conclusion and implication this study indicates that collaborative teaching of an integrated methods course is feasible and beneficial to both instructors and pre-service teachers. through collaborative teaching, each instructor learned how to teach with partners, gained knowledge beyond the subject he or she normally teaches, and was engaged in thinking about his or her own teaching in a broader and innovative way. more significantly, the collaboration was a reciprocal learning process. the three instructors learned from each other‟s way of teaching and improve their own teaching. for the pre-service teachers, this collaborative course not only helped them understand how three different subjects can be related to each other, but also provided opportunities for them to actually see and experience how collaboration can take place in teaching. pre-service teachers‟ understanding of collaboration was enhanced after the course. despite many benefits, collaborative work has its own obstacles. the lessons we learned from teaching this integration course are informative to other educators. collaborative teaching can be time consuming because it requires more meeting time for planning, sharing, and discussion (davis, 1995). to configure this course, the three instructors took a great amount of time in course preparation, meetings, and observations. their dedication and desire for the course to be successful was a necessary condition for the success of the collaboration. given its heavy load, this course carried 6 credits for pre-service teachers who satisfactorily completed it. however, each instructor only got 3 credits for teaching it, which did not reflect the amount of effort they made into the course. the department chair was made aware of this discrepancy and was suggested to find a solution to properly recognize collaborative teaching of an integrated methods course / zhou, kim & kerekes 135 instructors‟ workload. otherwise, the collaborative nature of this course will not sustain. as bakken, clark, and thompson (1998) stated, collaboration asks for individual member‟s „good‟ personality in working together because there are more possibilities to have adjustment and compromise in decision making. in this study, despite their differences in subject backgrounds and teaching experiences, the three instructors were able to work together. they opened their sessions to collaborators to observe and discuss. their mutual respect and open-mindedness made it possible for them to analyze each other‟s teaching and find solutions for effective curriculum integration. an educator‟s dedication to student learning is essential for good teaching. however, it alone usually is not enough for collaboration to take place and succeed. pleasant and fruitful collaboration starts with friendship. the collaborative experiences in this course convinced the three instructors that friendship and trust were a catalyst for successful collaboration. throughout the course, the three instructors had lunch together once a week and informally discussed their teaching, communities, cultures, and many other topics. this enabled them to build a close relationship and establish trust, thus making them more open to different ideas from their collaborators. another important factor for collaborative teaching is a “sense of parity” among faculty members (bakken, clark, & thompson, 1998). it is not easy to have a sense of parity among instructors who have differences in background, schedule, preferable ways of communication, and so forth. the three instructors built their sense of parity through mutual respect and group decision making. all of the course components such as the course outline, assignments, schedules, and policies were derived from their discussions and negotiations. pritchett (1997) pointed out that communication, involving dialogues, sharing, and negotiation, is crucial for successful team building. each member needs to beware of what is happening, share the information and ideas she or he has, and listen with an open mind to what others offer. during this course, the three instructors frequently used email communication to keep each other updated. weekly meetings provided them with a mechanism for sharing ideas, discussing issues, and making collaborative decisions. as a result, none of them felt being left behind or forgotten in the process. finally, collaborative teaching can be confusing to students who are used to isolated teaching. at the beginning of each semester, instructors occasionally heard complaints from pre-service teachers. more than one subject in one course, group rotation, and going back and forth between the university and schools were too much for some pre-service teachers‟ initial understanding of the course. although a well organized syllabus should be clear enough to address these confusions. however, the instructors found that other solutions needed to be in place to alleviate participants‟ confusions. in international electronic journal of elementary education vol.3, issue 2, march, 2011 136 addition to being available to participating pre-service teachers during office hours, the three instructors used the discussion and announcement tools provided in the blackboard learning management system for timely communication between the instructors and pre-service teachers. although scholars have argued that collaborative teaching promises great benefits for students, dugan and letterman (2008) claimed that little systematic research exists to show how such benefits occur. in their surveybased research, dugan and letterman analyzed and compared student appraisals of team-taught classes to a norm of traditional, solo-instructed courses. results indicated that there were no real differences in student attitudes toward team-taught and traditional classes. this report reminds us of the necessity of future research. our study used the instructors‟ and preservice teachers‟ narratives as evidence to support the type of collaboration we carried in the course. future research may consider to use other methods such as interviews to verify or confirm the value of such a type of collaboration and generate deeper understanding of how this collaboration contribute to pre-service teachers‟ learning. • • • received: 23 july 2010 / revised: 27 december 2010 / accepted: 12 january 2011 george zhou is an associate professor in the faculty of educatoin at the university of windsor, canada. he teaches science methods and graduate courses in science education. his current research interests include discourse in science education, curriculum integration, technology and teacher education, and multicultural and international education. jinyoung kim is an associate professor in the department of education at the college of staten island/the city university of new york. she has taught the multidisciplinary course for many years with specialty in music education. she has published many articles and books about music integration in early childhood and elementary education. judit kerekes is an assistant professor in the department of education at the college of staten island/the city university of new york. she has taught the multidisciplinary course for many years with specialty in mathematics education. references american association for the advancement of science. (1998). blueprints for reform. ny: oxford university press. bakken, l., clark, f. l. & thompson, j. (1998). collaborative teaching: many joys, some surprises, and a few worms. college teaching, 46(4), 154-57. berg, b. l. (2009). qualitative research methods for the social sciences (7th ed.). boston, ma: allyn & bacon. collaborative teaching of an integrated methods course / zhou, kim & kerekes 137 berlin d. f. & lee, h. (2005). integrating science and mathematics education: historical analysis. school science and mathematics, 105(1), 15-24. beyerlein, m. m. & harris, c. (2003). guiding the journey to collaborative work. pfeiffer & company. bredekamp, s. (ed.). (1987). developmentally appropriate practice in early childhood programs serving children from birth through age 8 (expanded ed.). washington, dc: national association for the education of young children. creswell, j. w. (2008). educational research: planning, conducting, and evaluating quantitative and qualitative research (3rd ed.). upper saddle river, nj: merrill/prentice hall. cumming, j. (1994). catering for the needs of all young adolescents: toward an integrated approach, unicorn, 20(2), 12-20. davis, j. r. (1995). interdisciplinary course and teaching: new arrangements for learning. phoenix: oryx press. dugan, k. & letterman, m. (2008). student appraisals of collaborative teaching. college teaching, 56(1), 11-15. freeman, r. e. (1993). collaboration, global perspectives, and teacher education. theory and practice, 32(1), 33-39. friend, m. (2000). myths and misunderstandings about professional collaboration. remedial and special education, 21, 130-132. helms, m. m., alvis, j. m. & willis, m. (2005). planning and implementing shared teaching: an mba team-teaching case study. journal of education for business 81(1), 29–34. hinton, s. & downing, j. e. (1998). team teaching a college core foundation course: instructors’ and students’ assessments. richmond, ky: eastern kentucky university. eric document no. ed 429469. hirst, p. h. (1974). knowledge and the curriculum: a collection of philosophical papers. london: routledge & kegan paul. johnson, d. w., johnson, r. t., & smith, k. a. (2000). constructive controversy. change, 3, 35-39. justiz, m. j. (1997). collaborating for success: case history of a school-college partnership. educational record, 78(2), 31-38. kapp, e. (2009). improving student teamwork in a collaborative project-based course. college teaching, 57(3), 139-143. kluth, p. & straut, d. (2003). do as we say and as we do: teaching and modeling collaborative practice in the university classroom. journal of teacher education, 54(3), 228-240. kulynych, j. j. (1998). crossing disciplines: postmodernism and democratic education. college teaching, 46(4), 144-149. letterman, m. r. & dugan, k. b. (2004). team teaching a cross-disciplinary honors course: preparation and development. college teaching, 52(2), 76-81. mid-continent regional educational laboratory (1993). learner-centered psychological principles: guidelines for school redesign and reform. denver, co: author. murawski, w. w. & swanson, h. l. (2001). a meta-analysis of co-teaching research. remedial and special education, 22(5), 258-267. national council of teachers of mathematics. (2000). principles and standards for school mathematics. reston, va: author. international electronic journal of elementary education vol.3, issue 2, march, 2011 138 national research council. (1996). national science education standards. washington, dc: national academy press. national science teachers association. (1997). nsta pathways to the science standards: guidelines for moving the vision into practice (elementary school edition). arlington, va: author. perkins, d. (1991). educating for insight. educational leadership, 49, 4-8. potterfield, a. & majerus, m. (2008). a collaborative approach to incorporating statistics in the physiology classroom. journal of college science teaching, 37(5), 54-60. pritchett, p. (1997). teamwork: the team member handbook. dallas, texas: pritchett and associates. quinlan, k. (1998). promoting faculty learning about collaborative teaching. college teaching, 46, 43-47. robinson, b. & schaible, r. m. (1995). collaborative teaching: reaping and the benefits. college teaching, 43(2), 8756-7555. sluss, d. & minner, s. (1999). the changing roles of early childhood educators in preparing new teachers: findings from three preparation programs. childhood education, 75(5), 280-84. surbeck, e. (1994). a glimpse through the door: journal writing with preservice teachers. childhood education, 70(4), 232-35. villa, r. thousand, j., nevin, a., & malgeri, c. (1996). instilling collaboration to inclusive schooling as a wayof doing business in public schools. remedial and special education, 17, 169-181. vogler, k. e. & long, e. l. (2003). team teaching two sections of the same undergraduate course: a case study. college teaching, 51,122–26. von glaserfeld, e. v. (1995). a constructivist approach to teaching. in l. p. steffe, and j. gale (eds.), constructivism in education. hillsdale, new jersey: lawrence erlbaum associate. wilson, v.a. & martin, k. m. (1998). practicing what we preach: team teaching at the college level. muskingum, oh: muskingum college. eric document no. ed 417172. zhou, g. (2010). conceptual change in science: a process of argumentation. eurasia journal of mathematics, science and technology education, 6(2), 101-110. international electronic journal of elementary education vol. 3, issue 3, july, 2011. issn:1307-9298 copyright © iejee www.iejee.com a review of (elementary) school selfassessment processes: ontario and beyond thomas g. ryan* nipissing university, canada leslie telfer ontario board of education, canada abstract this review draws attention to issues related to school self-assessment. this process has been similarly implemented in a variety of jurisdictions globally in the past decade hence their inclusion herein as an attempt to understand the school self-assessment process which has also been developed for use in local ontario (canadian) elementary schools. this review includes an examination of the foundational components of effective schools, the purpose of school self-assessment, and the successes and challenges of school selfassessment both locally and internationally. keywords: school, self assessment, self-evaluation introduction school self-assessment (ssa) as a means to improve schools is an important tool and mode to move forward (mcnamara, o'hara, lisi, & davidsdottir, 2011). conceptually school self-assessment is understood as a course of action undertaken by a school, “whereby carefully chosen participants describe and evaluate the functioning of the school in a systematic manner for the purposes of making decisions or undertaking initiatives in the context of … overall school development” (van petegem, 2005, p. 104 ). within an ontario, (canada) context, it has been similarly defined as a process carried out collaboratively by a school, in which chosen staff members systematically gather and analyze evidence to improve the schools * corresponding author : e-mail: thomasr@nipissingu.ca 100 college drive, north bay, on., canada p1b 8l7. 705 voice 474-3450 – fax 705 474-1947. mailto:thomasr@nipissingu.ca international electronic journal of elementary education vol.3, issue 3, july, 2011 172 performance (literacy and numeracy secretariat 2007, p. 11). ssa is an opportunity to advance and validate a school‟s development through systematic and strategic attention. ssa and school self-evaluation (sse) are terms that will be used interchangeably henceforth in order to address six purposeful questions. purpose our purpose was to review and thereby address the following questions: (1) what is school self-assessment (ssa). (2) what are the foundations of ssa? (3) what are the most popular models of ssa? (4) what is the purpose of ssa? (5) what are the components of ssa? (6) what are the successes and challenges of school ssa both locally and internationally? method we performed a search of documented databases such as ebscohost eric, education research complete, scholars portal, educational administration abstracts, and proquest. we further investigated copious numbers of education-focused peer-reviewed journals keying on ssa, including school effectiveness and school improvement, british educational research journal, oxford review of education and learning environment research. pertinent reviews of school self-assessment such as mcnamara, o'hara, lisi, and davidsdottir, 2011; kyriakides, creemers, antoniou and demetriou, 2010; croxford, grek and shaik, 2009; scheerens and bosker, 1997; marzano, 2003; durrant, 2006, were utilized. background the use of a school assessment instrument requires current educational policy-makers to acknowledge the merit and utility of this mode. to do so requires a change in fundamental beliefs and understanding of school evaluation per se. durrant (2006) suggested we need to engage people to hear what they have to say, “… a „pedagogy of voice‟, in which there is recognition of different participants and the distribution of resources such that they may participate equally. … this is the foundation of organisational and systemic improvement” (p. 4). by bringing school community members together we enable discussion and infuse a new community of practice that may avoid past enterprise results which have failed to produce the desired changes and outcomes; in part because school leaders find it difficult to comprehend conventional data/feedback (demie, 2003; mcnamara, et al., 2011, p. 80). researchers have finally heard practitioners who explained they were simply too busy day-to-day to grasp and implement research findings and performance data in schools (ryan & joong, 2005). some of the barriers to implementation included a, “„lack of time to read research publications and implement new ideas; lack of access to research publications; academic languages and statistical analysis that are not fully understandable; [and just a] lack of relevance of the research findings for practice” (demie, 2003, p. 446). as a result researchers such as fullan, hill, and crevola (2006) suggested political leaders “need to take an a review of (elementary) school self-assessment processes / ryan & telfer 173 unprecedented interest in public education in charting a new improvement mode for school systems” (p. 11). consequently, accountability models in the last ten years have gradually changed, shifting focus to measured student achievement as the primary basis for determining a school‟s effectiveness (lezotte, 2005). in addition “research ... has demonstrated that social background factors such as gender, ethnic background, fluency in english, free school meals and mobility rate can influence overall school levels of attainment” (demie, 2003, p. 447). with many variables at play and within arguably diverse contexts locally, regionally and globally, it has been concluded that we need to employ self-assessment to capture the „pedagogy of voice‟ and the contextual nuances unique to each school that be unnoticed using conventional indices such as test results (kyriakides, creemers, antoniou & demetriou, 2010, p. 820). within the past decade, research concerning the effectiveness of schools and publicly funded education has had a significant impact on the current ontario elementary school climate as “political influences, highstakes testing, and budget restraints have placed added pressures on school systems and increased the demands for change” (hulley & dier, 2005 p. 1). with increased accountability locally and globally, an emerging emphasis on school self-assessment has become a primary focus of school improvement initiatives (quist, 2003; van petegem, 2005). the drive for improvement in publicly funded schools has, in general, been initiated by policy makers at various levels of government usually from the top-down, mandate driven requirements that describe the outcomes schools are expected to produce (lezotte, 2005 p. 9). as the accountability agenda has escalated, publicly reported high profile data about schools have become a stalwart of most large scale reform efforts (croxford, grek & shaik, 2009; earl, 2001). foundation: components and models in an attempt to understand the school self-assessment process in ontario‟s elementary schools, consideration must first be given to an examination of the foundational research of educational effectiveness both locally and internationally in order to answer our question: what are the foundations of ssa? it is necessary to acknowledge that the process of ssa has been informed by three decades of effective schools research, some of which has been reviewed via current meta-analyses conducted by kyriakides, creemers, antoniou and demetriou (2010). effective schools research, conducted over a thirty year period has proven repeatedly that schools can, in fact, control enough variables to ensure that all students learn and function well in school (hulley & dier, 2005, p. 2). what is an effective school? lezotte (2002) defined an effective school “as a school that can, in measured student achievement terms, demonstrate the joint presence of quality and equity” (p. 21). several researchers have attempted to categorize the critical components of high achieving, effective schools, for the purpose of organizing research and replicating the success of international electronic journal of elementary education vol.3, issue 3, july, 2011 174 schools in educating all students, regardless of socioeconomic status or family background (kyriakides, creemers, antoniou & demetriou, 2010). historically, the genesis of school assessment can be traced to edmonds, who in 1982 provided the first found identification of „characteristics of effective schools‟. he suggested effective schools had the leadership of the principal who devoted substantial attention to the quality of instructing:  a pervasive and broadly understood instructional focus;  an orderly, safe climate conducive to learning and teaching  teacher behaviours that convey the expectation that all students are expected to obtain at least minimum mastery; and  the use of measures of pupil achievement as the basis for program  evaluation. (p. 10) also essential is an acknowledgement of the scheerens‟ model of school effectiveness displayed in table 1 which was developed based on a review of the instructional and school effectiveness research literature prior to 1997. scheerens‟ integrated model of school effectiveness included the following factors: table 1: the scheerens’ model school level  degree of achievement-oriented policy – educational leader ship  consensus, cooperative planning of teachers  quality of school curricula in terms of content covered, and formal structure  ordery atmosphere – evaluative potential classroom level  time on task (including homework) – structure teaching  opportunity to learn – high expectations of pupils‟ progress  reinforcement (scheerens, & bosker, 1997, p. 45) following the work of scheerens and edmonds, marzano‟s (2003) text entitled, „what works in schools: translating research into action‟, presented an extensive synthesis of effective schools research. marzano (2003) suggested, “thirty-five years of research provides remarkably clear guidance as to the steps schools can take to be highly effective in enhancing student achievement” (p. 11). marzano then developed a categorization a review of (elementary) school self-assessment processes / ryan & telfer 175 scheme which closely resembles those of other contributors. table 2 summarizes marzano‟s model. table 2: factors affecting student achievement factor example school  guaranteed and viable curriculum  challenging goals and effective feedback  parent and community involvement  safe and orderly environment  collegiality and professionalism teacher  instructional strategies  classroom management  classroom curriculum design student  home atmosphere  learned intelligence and background knowledge  motivation (marzano, 2003, p. 10) although an examination of the foundational components of effective schools identifies variations across key models, what may be fundamentally significant is the evidence to support the concept that schools have the ability to affect improvement in student achievement. marzano (2003) explains that “implicit in factors affecting student achievement is the notion that the school (as opposed to the district) is the proper focus for reform” (p. 10). elementary schools have the capacity to impact student learning and achievement when the key factors identified, are present. this summation addresses the importance of school self-assessment as a conduit for the development of schools that are capable of enhancing student achievement. scheerens and bosker (1997) earlier argued that school self-evaluation is an important component of school effectiveness inquiry, as “a second promising applied area is the growing practice of school self-evaluation. particularly when a pupil-monitoring system is part of overall school self-evaluation, there are interesting possibilities for analyzing school-level process-output associations” (p. 319). scheerens and bosker (1997) added, there is a kind of circular dependence of school self-evaluation and answering effectiveness-oriented research questions. first, school selfevaluation instruments may be developed on the basis of school effectiveness knowledge base, by selecting those implicit and process variables that are expected to work. next, the information gathered by means of school selfevaluation instruments based on this developmental rational could be used to further this knowledge base, as a side-product of the practical use that is of primary importance. (p. 321) international electronic journal of elementary education vol.3, issue 3, july, 2011 176 therefore, the viability of school self-assessment can be a legitimate process to foster the development of more effective schools. school self-assessment: purpose school self-assessment is a catalyst for improvement planning and implementation. it is how schools get to know themselves better and identify the strategies that will leverage change and improvement (literacy and numeracy secretariat, 2007). macbeath and swaffield (2005) agreed, pointing out, “school self-evaluation is by definition, something that schools do to themselves, by themselves and for themselves” (p. 239). in theory, self-assessment administered in a reflective, collaborative school setting is most effective in impacting school improvement. the „voice‟ of those who teach with the school grows onto pedagogical conversations which rise from with the school. the ontario school effectiveness framework (sef), which forms the basis for school self-assessment in ontario schools, includes the following basic tenants of the school self-assessment process within the school effectiveness framework:  self-knowledge and self-efficiency are as important for schools as they are for individuals  reflective, self-critical schools are better schools for teachers and students (the literacy and numeracy secretariat, 2008, p. 13). ideally, school self-assessment is a collaborative activity which encourages open, straightforward discussion about a school‟s strengths, areas requiring improvement and next steps (literacy and numeracy secretariat, 2007, p. 11). the literacy and numeracy secretariat (2007) suggests that for school self-assessment to be successful, certain conditions must be in place:  clear communication throughout the process,  personal and professional support, where needed,  shared leadership so that appropriate stakeholders are involved in decision making, and  willingness of teaching staff to share ideas, to explore, to build commitment and to mentor one another. (the literacy and numeracy secretariat, 2008, p. 12) there appears to be a current international understanding of the purpose of elementary school self-assessment which recognizes that to be successful, school self-assessment needs to be a systematic regular process where practices are shared („voice‟) by everyone in the school community (croxford, et al., 2009; quist, 2003). macbeath (2008) clarifies the purpose of self-evaluation, which is to, keep a school “mobile … [via] a continuing a review of (elementary) school self-assessment processes / ryan & telfer 177 process of reflection that becomes implicit in the way people think and talk about their work. it is a process in which teachers construct their own knowledge by surfacing tacitly held data about classroom life and exploring conditions conducive to learning” (p. 396). plowright (2007) added; “self-evaluation should be mainly initiated by the school, to collect systematic information about the schools functioning, to analyze and judge the information regarding the quality of the school‟s education and to make recommendations” (p. 374). in most descriptions, school-self assessment is referred to as a process aimed at school improvement. however when self-assessment is described as a systematic process focused on improvement of measurable outcomes and an assessment of quality and/or product, this implies a dimension of accountability. in an attempt to understand the fundamental purpose of self-assessment, the influence of accountability is examined next. school self-assessment (ssa): accountability a broad analysis of ssa should include a discussion of self-assessment as a measure of accountability and “must also address the tensions between external bureaucratic accountability and internal professional accountability” (macbeath, 2008, p. 396). current research suggests that like ontario, most western-european countries show similar trends emphasizing the responsibility of sse for accountability in relationship with school improvement (croxford, et al., 2009). we must be careful not to use ssa as window dressing for accountability purposes because, “internationally, there is increasing evidence of a shift from confrontational forms of school evaluation primarily concerned with external accountability toward internal systems more focused on capacity building for selfevaluation and professional development” (mcnamara, et al., 2011, p. 64). in a sef developed for ontario schools, the idea of school selfassessment as a measure of accountability is approached as a form of professional accountability. the literacy and numeracy secretariat (2007) states: “the framework will provide ways in which teachers and school and system administrators accept responsibility to hold themselves accountable for ensuring that research-based, effective strategies are consistently implemented across the province” (p. 4). the sef document recognizes the world wide trend that focuses on accountability and suggests that too often; this refers to an accountability that is imposed from external resources. it is our perspective that ontario educators wish to monitor their own effectiveness. indeed, the ultimate form of accountability occurs when “professionals engage in self-assessment and are willing to take steps to bring about improvement” (literacy and numeracy secretariat, 2007, p. 4). reeves (2004) addresses the paradox of accountability acknowledging that, “more real accountability occurs when the teachers actively participate in the development, refinement, and reporting of accountability (p. 3). international electronic journal of elementary education vol.3, issue 3, july, 2011 178 an examination of the literature reveals a dichotomy of views about what accountability means (mcnamara, et al., 2011, p. 63). for instance, earl and katz (2006) explain that “we believe that the essence of accountability is a deep and abiding commitment to making schools as good as they can be for all students” (p. 12). in furthering the discussion of ssa as a measure of accountability, leithwood and earl (2000) suggest, although internal accountability is the responsibility of schools and school leaders, one of the most powerful influences on the nature of the work of a school leader is the context, created in part, by the educational policies under which they operate; in this case, the accountability-driven policy context. this policy context dominates all initiatives to increase accountability. (p. 16) for the most part accountability measures reflect a combination of government imposed self-assessment frameworks that become the responsibility of the individual school community (mcnamara, et al., 2011, p. 64). in the article, quality assurance and evaluation (qae) within scotland: promoting self-evaluation within and beyond the country, croxford, et al. (2009) address an inconsistency within self-evaluation. at first sight, the term self-evaluation might give the impression of a bottomup approach, and to suggest that teachers and school management teams are reflective practitioners thinking about their own practice. however, the reality of the scottish system of self-evaluation is that it is a top-down system using prescribed indicators rather than self-chosen goals. (p. 186) also, a study in the netherlands looked into the effects of initializing school self-evaluation for inspection purposes in europe. the findings presented made a distinction between the two main roles regarding school evaluation: an internal one and an external one and was reported as follows: the external function focuses on the safeguarding of the quality of standards of schools, and in most european countries a national inspectorate of education is responsible for this task. in this respect, the government (through the efforts of the inspectorates) maintains strategic control over the goals of the education system, based upon standards, objectives, and criteria of success regarding the outcomes of a school. at the same time, the daily management practices remain the particular schools responsibility. the internal function is the responsibility of the schools themselves. they are supposed to determine, guarantee, and safeguard their quality and improve the teaching-learning process and their school performance. (the national inspectorate of education, 2006, p. 19) school self-assessment: contradiction in practice, as noted herein, most education systems appear to be moving towards a combination of methods, involving a degree of external monitors of internal self-assessment mechanisms (mcnamara & o‟hare, 2005). although there is ample literature to support the differences between internal and external accountability, research suggests there can be complimentary benefits to both. for instance, hofman, dukstra and adriann hofman (2005) explain: a review of (elementary) school self-assessment processes / ryan & telfer 179 external accountability focuses on the maintenance of quality standards in schools; in the netherlands, this function is executed by the national inspectorate of education. the internal function of accountability in the responsibility of schools themselves and this can be applied to determine, guarantee, and guard the (chosen) school goals, improve the teaching-learning process, and the quality of education provided by the school. in practice, the internal and external components of accountability depend on and influence each other. (p. 254) the conclusion that external accountability seems to strengthen internal monitoring and increase the use of the self-evaluation system within schools (hofman et al., 2005, p. 254), is noteworthy. yet hofman, et al. (2009) further studied the connection between organizational management and internal and external types of school accountability and concluded: “(a) external accountability seems to fortify the internal monitoring and use of evaluation systems within schools and (b) seems to promote the search for successes in failures within the schools‟ educational practices” (p. 50). this contradictory view of self-assessment is also expressed in the work of croxford, et al. (2009) whose research considers the “incongruity of a governance system that promotes self-evaluation, while at the same time requires adherence to external benchmarks and indicators” (p. 179). as well, croxford et al. (2009) when describing the scottish model pointed out, the current system of self-evaluation appears to be indented to change the culture and mindset of teachers. if teachers can be persuaded to internalize [sic] the goals of school improvement, and vision of quality that is defined by the quality indicators, and adopt these as norms for self-review of practice, and then the whole scottish education system will be on a journey to excellence. after years of being pressured to comply with policies and targets imposed from above, reactions to yet another set of quality indicators and policy rhetoric may be characterized as mere compliance with the audit system, and greater emphasis on ‘ticking boxes’ than achieving educational objectives. (p. 186) macbeath (2008) puts forward some clarity within the discussion of internal and external accountability and speaks to the ensuing contradictions that continue to surface, suggesting internal accountability describes the conditions that precede and shape the responses of schools to pressure that originates in policies outside the organization. macbeath concludes that with strong internal accountability schools are likely to be more responsive to external pressure for performance (p. 396). in a similar context, stoll and fink (as cited in mcnamara and o‟hara, 2005), explain: while opening mandated doors will certainly get people’s attention there is little evidence that it engenders commitment on the part of the people who have to implement the change – it is through opening as many internal doors as possible that authentic change occurs. (p. 270) a review of the current research seems to suggest that internal accountability is more likely to affect authentic change in schools; external demands and the force of government policies appear to provide the impetus for initiating the process of self-assessment (mcnamara, et al., 2011, p. 80). international electronic journal of elementary education vol.3, issue 3, july, 2011 180 clarity of purpose when initiating the process of self-assessment seems to ensure an authentic experience. the process for school self-assessment in ontario schools, as described in the literacy and numeracy secretariat (2007) speaks to a process which is initiated by each school. school self-assessment: international lessons although, to date there has been no empirical evidence to support the implied relationship between the sef self-assessment process and improved student achievement in ontario elementary schools, there is, however, an abundance of related research coming out of the united kingdom and several other countries who have implemented large scale educational reform movements. as the literature reveals, school self-evaluation has been on the educational agenda of most european countries for the past ten years. it is reasonable to assume that the development of the sef tools for self-assessment in ontario‟s schools reflects the lessons learned from similar initiatives implemented in a variety of international jurisdictions. common to each educational reform movement is a defined process for school selfevaluation. macbeath (2001) addresses the trend throughout europe and the united kingdom explaining how, the school’s own self evaluation provides the focus and centre piece for external review and in which initiative lies with the school leaders to place self-evaluation at the heart of school and classroom practice. as in europe, in asia, in australia, in new zealand, and in north america there has been a progressive move away from more traditional forms of quality assurance to more school-owned, school-driven form of evaluation and accountability. (p. 387) macbeath‟s (2008) longitudinal research in the implementaton of school self-evaluation and external school review for the hong kong education development bureau provides substantial evidence to support the process. in the final impact study (macbeath, 2008) identified the following achievements associated with the self-evaluation process:  a deepening understanding and heightened confidence of school staff in relation to school self-evaluation (sse) and external self-review (esr),  classroom teaching becoming more engaging, learning-centred, and open and receptive to student voice,  a welcome for the insights of esr teams and setting of clear agendas for improvement following external review,  the enhanced skills of external system review teams in conducting review,  sharing of thinking and practice by teachers beyond the classroom in a whole school dialogue,  a growing willingness to engage with evidence, to move from impressionistic evaluation of quality and performance to a a review of (elementary) school self-assessment processes / ryan & telfer 181 more systematic, rigorous and informed approach to assessing practice (p. 27). macbeath (2008) also identified the importance of effective leadership as part of the school self-assessment process. he states, “in many instances it was the leadership of the principal that distinguished schools in which self-evaluation and external review were seen more as an opportunity than a threat” (p. 390). macbeath‟s study further indicated that the role of the school improvement team was related to successful evaluation and revealed that, “most informed and positive of all were members of school improvement teams, a function of their close involvement in, and ownership of, the self-evaluation process” (p. 388). it is important to recognize that in ontario‟s sef the role of the school improvement team is identified as playing a major role. the school improvement team (sit) is responsible for reviewing priorities, determining the scope of the process (literacy and numeracy secretariat, 2007, p. 13). the responsibility of the school self-assessment process rests with the principal and the school improvement team in collaboration with the entire staff. macbeath (2008) suggested, the impact and quality of leadership was also manifest in the approach of school improvement teams (sits), varying from largely ineffective to exemplary models of shared leadership. the impact study concluded that the efficacy and credibility of the (sits) could be explained by a number of key factors:  team membership includes a cross-section of staff with high credibility among their colleagues.  there was scope to exercise initiative and creativity.  there was a willingness and capability to ask hard questions and instill an ethos of accountability.  there was teamwork which synergises the capacities of all its members.  initiative and ownership were displayed, which create confidence and shared leadership.  vision was evident as to what self-evaluation can achieve and how it can feed into learning and school improvement. (p. 26) the meaningful involvement of school improvement teams in the self-assessment process appears to be a key variable influencing its effectiveness. similar research conducted in the netherlands by schildkamp, visscher and luyten, (2009) also speaks to the positive impact of self-evaluation. zebo (in dutch the acronym stands for self-evaluation in primary schools), is a self-evaluation instrument for dutch primary schools, which was developed on the basis of school effectiveness research findings and input of teachers and principals” (p. 70). the results of schildkamp et al. (2009) longitudinal study of the use and effects of zebo also provides insight into the impact of school selfassessment. teachers and principals involved in the study reported that the international electronic journal of elementary education vol.3, issue 3, july, 2011 182 zebo use had a growing effect on consultation and school functioning and quality, professional development, achievement orientation, and the didactic methods used by teachers in the classroom” (p. 84). schildkamp et al. (2009) presents several key insights that can inform and improve sef in ontario‟s schools for instance: even when schools use the zebo output intensively, this may not lead directly to changes in student learning. although this is the idea underlying selfevaluation instruments, it may not be that easy to accomplish (p. 85). ... for this reason, it may also be necessary to support schools in this respect, to let them gradually develop the skills to diagram, remediate and implement. (p. 86) many lessons can be learned from the experiences of the office for standards in education (ofsted) external expectation of schools in england. the research of plowright (2007) reflects a significant change in ofsted‟s approach to school inspections. he states that, “however, compared with early inspection, there is now an increased focus on school self-evaluation as a contribution to the inspection process” (p. 374). the change in the ofsted process was reflective of the poor experiences reported by many schools where the ofsted inspection was primarily seen as being about accountability rather than development (plowright, 2007, p. 23). as a result of the changes detailed in the document, the future of inspection: a consultation paper (as cited in plowright, 2007), the increased focus on selfevaluation provides the potential for schools to take more responsibility for identifying and addressing their own development needs. similarly in the framework for the inspection of schools in england from september 2005 (as cited in plowright, 2007) makes it clear that inspection from 2005 onwards, will have a strong emphasis on school improvement through the use of the school‟s own self-evaluation, including regular input from pupils, parents and other stakeholders, as the starting point for inspection and for the school‟s internal planning and development (p. 390). mcnamara and o‟harra (2005) illuminate similar research carried out in ireland which reflects the increased emphasis being placed on school self-evaluation and argues that it does constitute the best way forward (p. 267). the department of education and science in ireland (des) has seen a “clear shift in official policy on school evaluation towards a greater emphasis on internally driven self-review as the desired method of achieving the goals both of improvement and accountability” (p. 268). the irish (des) published looking at our schools, and aid to selfevaluation schools in 2003. similar to ontario‟s (sef) framework, schools in ireland are expected to engage in process of collecting and analyzing information and on this evidence recommendation statements will be made. mcnamara and o‟hara (2005) acknowledge that the process of evidence collection provides limited data on which to base empirical judgments suggesting, this sounds impressive until one realized firstly that these bland assertions ignore the fact that very little data (of either quantitative or qualitative nature) a review of (elementary) school self-assessment processes / ryan & telfer 183 are available about any facet of the operation of schools in ireland, and secondly, no attempt is made to suggest who should collect and analyze this information or how they should go about it. (p.276) the local context: ontario, canada the current educational climate in the province of ontario was initiated by the election of dalton mcguinty as the premier of ontario. following his election in 2003, the education platform of the provincial government became and remains a primary focus. the focus established by the minister of education, the honourable gerard kennedy in 2003, was on a more collaborative-relationship among the stakeholders within the education system. this approach continues to anchor the government‟s existing goals. kennedy suggested a relationship that would focus on the new three “r‟s”, respect, responsibility, and results. “there should be common respect, mutual responsibility-taking, and results: in other words, a real partnership” (ontario ministry of education, 2005). kennedy introduced an era of results focused accountability that continues to be at the forefront in 2009. in a paper published to support the government‟s second mandate titled reach every student: energizing ontario education (2008), ontario premier mcguinty presents the government‟s plan to continue working together with their partners to build and energize ontario‟s schools. three core priorities have been identified: (1) high levels of student achievement.  going deeper and wider on literacy and numeracy, including reaching the targets of 75 percent of students achieving at the provincial standard in grade 6. (2) reduced gaps in student achievement.  reducing the gap in achievement for those groups of students who, for whatever reason, need extra help. (3) increased public confidence in publicly funded education.  fostering greater two-way engagement with the public to inform the implementation of the mandate and to foster public confidence.  strengthening the role of schools as the heart of communities.  recognizing the pivotal role of schools in developing the work force and citizens of tomorrow. (p. 14) policy makers in ontario suggested that education leaders have a professional and pedagogical responsibility to focus on what it takes to implement the core priorities as outlined above. educational leaders are accountable to the administrative hierarchy within the ontario ministry of education, to fulfill the government‟s mandate within a culture of shared ownership and responsibility for student outcomes. the current provincial government suggests that it is the shared responsibility of all stakeholders international electronic journal of elementary education vol.3, issue 3, july, 2011 184 to ensure that all children are well served by our publicly funded education system. there is an obvious sense of urgency in ontario schools, to improve achievement for all students. reflective of the momentum initiated by the premier‟s education platform for measurable improvement in ontario schools, the literacy and numeracy secretariat was established in november 2004 to support boards in improving student achievement. in an effort to support this mandate, the literacy and numeracy secretariat developed the sef to assist boards and schools in sustaining a culture of continuous school improvement. the sef: a collegial process for continued growth in the effectiveness of ontario elementary school was developed to guide school and board analysis and improvement planning (literacy and numeracy secretariat, 2007). see appendix a for more information. the sef, as a tool for self-assessment, identifies essential components, factors that have been identified to have an impact on student achievement and supporting indicators to assist schools in identifying areas of strength and areas requiring further development. the sef indicates that at the heart of self-assessment, there are three basic questions:  how effective are we in achieving our student learning and achievement goals?  what is the evidence?  what actions will we take to ensure continuous improvement? the school self-assessment process, (appendix b), outlines the school-based analysis process developed for use in ontario schools. the process outlines five distinct phases where the expectation is that schools review the essential components within the cycle (literacy and numeracy secretariat, 2007, p. 10). this discussion emanates from one author who has acted as the school effectiveness principal lead in an ontario board of education, who has been responsible for the managerial tasks around organization, administration, and implementation of the school effectiveness framework and the school self-assessment process in the board. the other author has examined, reviewed and written extensively about school development, assessment and growth. we believe the implementation of the sef framework and school self-assessment process is intended to provide a structure and process for ensuring that students in ontario are taught by highly skilled educators implementing the best practices supported by research. establishing a common understanding of what makes schools „effective‟ provides the basis for reflection and identifies the primary interest for this literature review. conclusion an attempt to understand the school self-assessment process via ontario schools was undertaken by reporting upon the theory and practice rooted in a review of (elementary) school self-assessment processes / ryan & telfer 185 thirty years of international effective schools research. we endeavored to explain, what school self-assessment (ssa) is, by illumination of the foundations of ssa. we examined the most popular „models‟ of ssa and addressed their purposes briefly. we also detailed the components of ssa and some of the successes and challenges of ssa, both locally and internationally. in doing so, the recognized contributions to effective schools research has identified key factors, components and correlates that provide the foundational structure for schools that are deemed highly effective in enhancing student achievement. significant to the findings was the fundamental belief that all schools can control enough variables to ensure that all students learn and function well in school. the ssa process, when considered in the effective schools context, becomes a companion in the development of increasing the effectiveness of schools. when self-assessment is understood to be an opportunity to explore a school‟s progress and practice related to student achievement, it has the potential to become an effective tool for impacting focused change (mcnamara, et al., 2011, p. 80). the international literature reviewed, appears to support ontario‟s view of self-assessment, a grassroots, schoolbased initiative, as more effective than externally imposed processes of selfevaluation which have been the common practice in the international context. if the purpose of ssa in ontario schools is formative, where the primary focus is on the process of collaboration with students, staff and parents to identify the specific goals related to areas that requiring improvement in an individual school, then research would suggest that we are moving forward in the right direction. schmoker (1996) suggested that, without a common set of goals, schools will not be able to sustain their efforts, hope will dwindle, and low expectations may set in. with it, the entire school community can work as one. collaboration will not happen, however, if goals are too numerous, superficial, or unmeasurable. (p. 105) the continued focus on increased levels of accountability requires that schools and stakeholders become better versed in articulating the individual needs of their learners. the ssa process, as a measure of school improvement, may assist in providing schools with an informed focus for improving student learning. that being said, strong empirical evidence on the effects of school self-assessment is still lacking. trying to demonstrate a causal link between self-assessment and improved student achievement is too simplistic we believe. self-assessment is not a prescription of what schools can do to get better; it is simply a tool for critical analysis that is useful in the right setting. we must be cautious in moving forward to ensure that the purpose of self-assessment is clearly understood. assuming that all schools have developed a collaborative culture where purposeful, supportive peer interaction is the norm may be unrealistic and naïve. next steps a necessary key question needs to be asked: can the process of ssa act as a catalyst for school improvement and enhance student achievement scores? international electronic journal of elementary education vol.3, issue 3, july, 2011 186 or, is the process being used and manipulated by the accountability movement? as we enter the third year of implementation in ontario, it wouldn‟t be out of the ordinary to explore how well schools and school teams are developing their skills and knowledge around the self-assessment process, as a result of the „the school effectiveness framework‟ and „selfassessment process‟ in ontario schools. we are left to wonder, what impact will the implementation of the sef and self-assessment process have on the ontario school improvement planning process? • • • received: 09 march 2011 / revised: 10 june 2011 / accepted: 30 june 2011 thomas g. ryan, associate professor at nipissing university‟s faculty of education in north bay, ontario, canada. he is the author of several books, chapters and many articles. leslie telfer (m.ed.), school effectiveness principal lead at an ontario board of education, who has been responsible for the managerial tasks around organization, administration, and implementation of the school effectiveness framework and the school self-assessment process in the board. references croxford, l., grek, s., & saik, f. j. (2009). quality assurance and evaluation (qae) in scotland: promoting self-evaluation within and beyond the country, journal of education policy, 24 (2), 179-193. demie, f. (2003). using value-added data for school self-evaluation: a case study of practice in inner-city schools. school and leadership management, 23 (4), 445-467. durrant, j. (2006). planning, tracking and evaluating impact: towards an integrated approach. paper presented at the international congress for school effectiveness and school improvement, florida, u.s.a., january 2006 pp.143-161. earl, l. (2001) assessment as learning. in hawley, w. (ed.) the keys to effective schools. national education association, thousand oaks: corwin press. earl, l. m., & katz, s. (2006). leading schools in a data rich world: harnessing data for school improvement. thousand oakes, ca: corwin press. edmunds, r.r. (1982). programs of school improvement: an overview. educational leadership, 40(3), 4-11. fullan, m., hill, p., & crevola, c. (2006). breakthrough. thousand oaks, ca: corwin press. hofman, r. h., dijkstra, n.j. & adriaan, hofman w.h. (2009). school self-evaluation and student achievement, school effectiveness and school improvement, 20 (1), 47-68. hofman, r. h., dukstra, n. j., & adriaan, hofman, w.h. (2005). school self-evaluation instruments: an assessment framework, international journal of leadership in education, 8(3). 253-272. hulley, w., & dier, l. (2005). harbours of hope: the planning for school and student success process. bloomington, ia: national educational service. a review of (elementary) school self-assessment processes / ryan & telfer 187 kyriakides, l., creemers, b., antoniou, p., & demetriou, d. (2010). a synthesis of studies searching for school factors: implications for theory and research. british educational research journal, 36 (5), 807-830. leithwood, k., & earl, l. (2000). educational accountability effects: an international perspective, peabody journal of education, 75(4), 1-18. levine, d.l.u., & ornstein, a.c. (1993). school effectiveness and national reform, journal of teacher education, 44(5). 335-345. lezotte, l.w., & mckee, k.m., (2002). assembly required a continuous school improvement system. okemos, mi: effective schools products, ltd. lezotte, l. w. (2005). more effective schools: professional learning communities in action. in r. dufour, r. eaker, & r. dufour (eds.), on common ground: the power of professional learning communities (pp. 177-191). bloomington, in: national educational service. literacy and numeracy secretariat. (2007). the school effectiveness framework: a collegial process for continued growth in the effectiveness of ontario elementary schools. implementation pilot, september 2007-june 2008. literacy and numeracy secretariat (2008). the school effectiveness framework: a collegial process for continued growth in the effectiveness of ontario elementary schools. toronto, on: the ontario ministry of education and training. macbeath, j. (2004). putting the self back into self-evaluation, improving schools, 7(1). 87-91. macbeath, j. (2008). leading learning in the self-evaluating school, school leadership and management, 28(4). 385-399. macbeath, j., schratz, m., meuret, d., jacobsen, l., & smith, w. j. (2001). schools must speak for themselves: the case for school self-evaluation. journal of educational administration and foundations, 16(1), p. 33-42. marzano, r.j. (2003). what works in schools: translating research into action. alexandria, va: association for supervision and curriculum development. mcnamara, g. & o‟hara, j. (2005). internal review and self-evaluation – the chosen route to school improvement in ireland. studies in educational evaluation, 31, 267-282. mcnamara, g., o'hara, j., lisi, p.l., & davidsdottir, s. (2011). operationalisingselfevaluation in schools: experiences from ireland and iceland. irish educational studies, 30(1), 63-82. doi: 10.1080/03323315.2011.535977. meuret, d., & morlaix, s. (2003). conditions of success of a school‟s self-evaluation: some lessons of a european experience, school effectiveness and school improvement, 14(1). 53-71.national inspectorate of education. (2006).the state of education in the netherlands. minister of education, culture & science. retrieved from http://www.onderwijsinspectie.nl/site/english. ontario ministry of education. (2005). ontario education: excellence for all, volume 2, toronto, on: author. (pp. 1-7). retrieved from http://www.edu.gov.on.ca/eng/document/nr/05.12/leadingeducation.pdf ontario ministry of education (2008). reach every student: energizing ontario education. toronto, on: author. (pp. 1-16). ontario principals council & national association of head teachers (united kingdom) (2008). school self-assessment the road to school effectiveness. toronto, on: nelson education limited. http://www.onderwijsinspectie.nl/site/english http://www.edu.gov.on.ca/eng/document/nr/05.12/leadingeducation.pdf international electronic journal of elementary education vol.3, issue 3, july, 2011 188 plowright, d. (2007). self-evaluation and ofsted inspection, educational management administration and leadership, 35(3), 373-393. reeves, d. b. (2006). the learning leader: how to focus school improvement for better results. alexandria, va: association for supervision and curriculum development. reeves, d. b. (2004). accountability for learning how teachers and school leaders can take charge. alexandria, va: association for supervision and curriculum development. ryan, t. g., & joong, p. (2005). teachers‟ and students‟ perceptions of the nature and impact of a large-scale reforms in ontario secondary schools. canadian journal of educational administration and policy, (38). retrieved march 23, 2005, from http://www.umanitoba.ca/publications/cjeap/articles/ryan_joong.html saunders, l. (1999). who or what is school self-evaluation for? school effectiveness and school improvement, 10(4), 414-429. scheerens, j., & bosker, r. (1997). the foundations of educational effectiveness. oxford, uk: pergamon. schildkamp, k., & teddlie, c. (2008). school performance feedback systems in the usa and in the netherlands: a comparison. educational research and evaluation, 14(3), 255282. schildkamp, k., visscher, a., & luyten, h. (2009). the effects of the use of a school selfevaluation instrument, school effectiveness and school improvement, 20(1), 69-88. schmoker, m. (1999). results: the key to continuous school improvement. alexandria, va: association of supervision and curriculum development. schmoker, m. (2006). results now: how we can achieve unprecedented improvements in teaching and learning. alexandria, va: association for supervision and curriculum development. swaffield, s., & macbeath, j. (2005). school self-evaluation and the role of a critical friend. cambridge journal of education, 35(2), 239-252. van petegem, p. (2005). designing school policy: school effectiveness research as a starting point for school self evaluation, acco, leuven. http://www.umanitoba.ca/publications/cjeap/articles/ryan_joong.html a review of (elementary) school self-assessment processes / ryan & telfer 189 appendix a international electronic journal of elementary education vol.3, issue 3, july, 2011 190 appendix b a review of (elementary) school self-assessment processes / ryan & telfer 191 this page is blank www.iejee.com http://www.iejee.com/ international electronic journal of elementary education, 2013, 5(2), 199-218. issn:1307-9298 copyright © iejee www.iejee.com teaching primary grade students perfectionism through cartoons compared to bibliotherapy miranda l. zousel marion independent school district, marion, iowa, usa audrey c. rule university of northern iowa, cedar falls, iowa, usa stephanie r. logan university of northern iowa, cedar falls, iowa, usa received: 7 january 2013 / revised: 11 march 2013 / accepted: 16 march 2013 abstract this experimental study compared concept acquisition and enjoyment of learning about perfectionism under two conditions: bibliotherapy (control) and analysis and construction of cartoons (experimental) in first, second and third grade students (n=46). posttest results showed students learned significantly more content in the experimental condition with a medium effect size. students were more engaged in the cartoon condition, appreciating the humor and opportunity to be creative. most students reported liking the bibliotherapy but some complained of boredom. the authors recommend that both bibliotherapy and cartoon analysis be used in lessons about perfectionism to maintain student interest and comprehension of concepts. keywords: perfectionism, cartoons, bibliotherapy, elementary students introduction a sobbing first grader crumples her recent test in frustration with her performance. her score was 97 percent, yet, the child judged her work as unacceptably inadequate (personal experience of second author, a former first grade teacher). perfectionism is defined as the need to perform flawlessly in many areas of one’s life (flett & hewitt, 2002). perfectionism particularly affects high-performing students (schuler, 2000) but may be present in all ages and levels of achievers (pacht, 1984). perfectionism is recognized as a multidimensional construct by most investigators (e.g., rice & ashby, 2007). debate over whether perfectionism has adaptive as well as maladaptive aspects is ongoing, with  audrey c. rule, department curriculum and instruction, 618 schindler education center, 1227 west 27th st., university of northern iowa, cedar falls, iowa 50614-0606 phone: 001-319-273-7829; email: audrey.rule@uni.edu http://www.iejee.com/ teaching perfectionism through cartoons / zousel, rule & logan 200 recognition of and research into adaptive aspects of perfectionism continuing to the present (e.g., davis & wosinski, 2012; diprima, ashby, gnilka, & noble, 2011; elion, wang, slaney, & french, 2012; stoeber & otto, 2006). an early researcher in perfectionism, hamachek (1978), identified ‘normal’ perfectionists as those who take pleasure in their efforts and can choose to be less than perfect when expedient; ‘neurotic’ perfectionists, in contrast, seek typically unattainable levels of performance, feel dissatisfaction with their work, and are unable to relax their standards. this difference has been maintained over time through the use of the terms ‘positive’ and ‘negative’ perfectionism (slade & owens, 1998) or ‘adaptive’ and ‘maladaptive’ perfectionism (burns & fedewa, 2005). adaptive perfectionists adhere to high standards, experiencing pleasure and satisfaction even when falling short of goals. positive aspects of perfectionism include organization and orderliness, striving for excellence, and conscientiousness (burns & fedewa, 2005) and are related to positive self-esteem (elion, wang, slaney, & french, 2012) and effective psychological adjustment (rice & mirzadeh, 2000). in contrast, a persistent feeling of failure (depression) along with fear of mistakes and criticism (anxiety) when goals are unmet, leading to procrastination and indecisiveness are characteristics of maladaptive perfectionism (grzegorek, slaney, fanze, & rice, 2004; wang, slaney, & rice, 2007). these traits are related to chronic low self-esteem, negative outlook, and symptoms of psychopathology (bieling, israeli, smith, & antony, 2003; dunkley, zuroff, & blankstein, 2003). however, characteristics of positive and negative forms of perfectionism are not mutually exclusive and have been found to be positively correlated (flett and hewitt, 2006); some differences may lie in the duration and intensity of the feelings (schuler, 2002). examination of perfectionism in young children is in its early stages (gilman & ashby, 2006), but many researchers (dekryger, 2005), parents, teachers, and children themselves have noted its occurrence in early grades. for example, dohnt and tiggemann (2006) found that girls aged five to eight already had developed feelings of body-dissatisfaction and low self-esteem in reaction to appearance-focused television programs and peer attitudes. without intervention, students may experience the devastating effects of perfectionism including depression (stoeber & yang, 2010), anxiety and rigid behaviors (marano, 2008), selfharm behaviors (o’connor, rasmussen, & hawton, 2010), eating disorders (boone, soenens, braet, & goossens, 2010), workaholism (clark, lelchook, & taylor, 2010) setting unreasonable, unreachable goals (stoeber & rambow, 2007; stoeber, hutchfield, & wood, 2008), procrastination (walsh & ugumba-agwunobi, 2002), cheating, lack of motivation, and underachievement (stoeber, hutchfield, & wood, 2008). therefore, it is important for teachers to be aware of the indicators, causes, and mitigations of maladaptive perfectionism so that they may assist students in adopting psychologically healthy mindsets. dimensions of perfectionism perfectionism involves setting high, often unreasonable, expectations for oneself and others (neihart, 2002). there are different perspectives researchers have taken in investigating perfectionism. the frost multi-dimensional perfectionism scale (frost, marten, lahart, & rosenblate, 1990) measured personal standards, concern over mistakes, doubts about action, organization, parental expectations, and parental criticism. hewitt, flett, turnbulldonovan, and mikail (1991) suggested that perfectionism has three dimensions: self-oriented, other-oriented, and socially prescribed. these parameters were measured by the hewitt multidimensional perfectionism scale (hewitt & flett, 1991). persons with self-oriented perfectionism hold unrealistic, self-imposed standards, critically evaluating and scrutinizing their own performance against these values; while other-oriented and socially prescribed perfectionists are motivated by a need to achieve high standards imposed by other people or international electronic journal of elementary education vol.5, issue 2, 199-218, 2013 201 by society in general (hewitt, flett, besser, sherry, & mcgee, 2003). it is when high standards are pursued despite significant adverse consequences that such perfectionist tendencies become clinical disorders (shafran, cooper, & fairburn, 2002). in this article, we focus mostly on the differences between adaptive and maladaptive perfectionism. these ideas were presented to the first through third grade students in this study as ‘healthy’ and ‘unhealthy’ perfectionism, as this terminology was familiar to young elementary school students. healthy or adaptive perfectionists hold high standards, are persistent, and conscientious (bieling, 2003) with the flexibility to choose the areas in which they apply high standards or ignore the need to perform perfectly as the situation dictates. adaptive perfectionist behaviors are associated with positive attitudes and satisfaction in life (stoeber & otto, 2006). youth who strive for excellence show hope of success, motivation in their schoolwork, and school achievement (stoeber & rambow, 2007). in contrast, unhealthy or maladaptive perfectionists set unrealistic goals and compulsively try to meet them in any way possible, even when the path is unhealthy or dangerous. they feel guilt when their performance is imperfect, but they do not allow themselves to feel pride when work meets high standards; instead, they judge the task as too easy, thereby depriving themselves of satisfaction in accomplishment (shafran, cooper, & fairburn, 2002; stoeber, kempe, & keogh, 2008). maladaptive perfectionists do not accept advice well, interpreting it as personal criticism. hanchon (2011) found that both adaptive and maladaptive perfectionist students endorsed mastery goals, but higher self-confidence and academic performance were traits of adaptive perfectionists, while maladaptive perfectionists evidenced more face-saving, a greater desire to outperform others, and more anxiety, depression, and academic problems. brown and beck (2002) recognized the relationship between maladaptive thinking patterns and perfectionism. some of these negative thinking patterns include overgeneralization (flett, hewitt, blankstein, & gray, 1998), catastrophization and self-blame (rudolph, flett, & hewitt, 2007), and all-or-nothing (categorical or dichotomous) thinking (burns & fedewa, 2005). teaching children about perfectionism there are many symptoms of perfectionism in children that teachers may notice (davies, 2005), alerting them to the need for action. these include students choosing exceptionally high standards for themselves and exhibiting selfcriticism, headaches, or other physical complaints when they don’t meet those expectations. perfectionists often have low selfconfidence, are easily embarrassed, express anxiety about making mistakes, and are highly sensitive to criticism. they may procrastinate on challenging or evaluated tasks and have difficulty making decisions. they are frequently emotionally guarded with a tendency to be critical of others. a student who repeatedly starts an assignment over, is reluctant to volunteer answers or share work, views grades as ‘all-or-nothing,’ or has a strong emotional reaction to a relatively minor error, is exhibiting possible signs of perfectionism (cohen, 1996). because many perfectionistic traits have the potential to develop into negative behaviors, it is important to help students recognize and manage their perfectionistic inclinations. students can be taught strategies through social-emotional skill lessons to manage perfectionist tendencies and use any perfectionist traits in a healthy manner. on the positive side, perfectionism can be used as a tool toward successfully reaching one’s goals and may serve as a motivator in tedious or challenging work for many students (stoeber & rambow, 2006). ‘[p]erfectionism must be seen as a potent force capable of bringing either intense frustration and paralysis or intense satisfaction and creative contribution, depending on how it is channeled’ (silverman, 2011, para. 1). perfectionism lessons can be taught in small or teaching perfectionism through cartoons / zousel, rule & logan 202 large group settings at all age and ability levels. nugent (2000) made several suggestions for how classroom teachers might teach students about the topic, including (1) engaging students in art activities that provide free exploration of new media or techniques, allowing an affective outlet; (2) using bibliotherapy; and (3) creating a positive classroom culture. bibliotherapy is a widely used whole class, small group, or individual student strategy in which stories in books are used to create a link between the student and characters having similar emotional experiences. this connection helps the student find solutions and insights into his/her own feelings (halsted, 2009). after the book has been read, the teacher engages students in a discussion of the story related to perfectionism, inserting prompts and questions as needed. a positive classroom culture in which students feel free to respond without criticism and make mistakes is paramount to helping students overcome maladaptive perfectionism. the teacher needs to model his or her own mistakes as pathways to learning. adelson (2007) recommended that the word ‘perfect’ be removed from the classroom vocabulary and replaced with other ideas such as ‘excellence’. process, rather than product, should be emphasized with stories of people who made numerous revisions to their work being read so that students understand that mistakes, set-backs, failures, and improvements are normal. teachers who have overcome maladaptive perfectionism may want to share their struggles, insights, and coping strategies, perhaps leading to a discussion of realistic goal-setting (brophy & rohrkemper, 1989). guidance counselors often play a large role in working with students who evidence maladaptive characteristics of perfectionism. therapy may take the form of role-playing, individual counseling, or small group sessions and activities such as play therapy. a beneficial technique recently applied successfully to children with perfectionism (nobel, manassis, & wilansky-traynor, 2012) is cognitive behavioral therapy. this intervention is based on the theory that students with emotional disorders like anxiety and depression hold cognitive errors or distorted thinking that perpetuate these symptoms (beck, rush, shaw, & emery, 1979). an example of such a thinking error is that one should be perfect and if flawless performance is not accomplished, one is a complete failure. the essence of this therapy is to expose distorted thinking and to examine environmental evidence that challenges these misperceptions. cartoon analysis and creation is a new approach showing potential that can be used by the classroom teacher. this instructional strategy may benefit students through the motivating effects of humor, wordplay, and creative drawing along with the mental manipulations of applying concepts to new (cartoon) situations. analyzing cartoons for humor and academic content activates students’ problem–solving skills (berk, 2002). cartoon panels or strips have long been used as a way to focus student attention at the beginning of a lesson (e.g., trefts & blakeslee, 2000). political cartoons are a popular way to build critical thinking skills, stimulate discussion, and motivate students in history or political science (dougherty, 2002; risinger & heitzmann, 2008). several investigations have been conducted to delineate the teaching strategies, such as making analogical or symbolic substitution lists and concept maps, which lead students to develop more complex, social studies cartoons (e.g., bickford, 2011). unfortunately, few studies have examined how teaching with cartoons can assist elementary students in learning non-political or non-historical content; however, one counterbalanced study (rule & auge, 2005) showed that sixth graders studying rocks and minerals through cartoon analysis and creation excelled in content knowledge and motivation with a large effect size as compared to more traditional exercises. in this study and in another international electronic journal of elementary education vol.5, issue 2, 199-218, 2013 203 report on a motivating science lesson with low-performing eighth graders (sallis, rule, & jennings, 2009), students discussed the academic concepts represented by the cartoons along with their funny aspects. they suggested improvements to the cartoons to add more science content or humor. students then read nonfiction books or listened to factual presentations to gather information to complete partly-finished or original cartoons to show more concepts, incorporating puns, impossible situations, or exaggerated emotional expressions to make them appealing. another use of cartoons in science teaching that employed a different approach is the use of concept cartoons (e.g., ekici, ekici, & aydin, 2007; kabapinar, 2005; keogh & naylor, 1999). in this method, a science concept is explained by different cartoon characters who supply alternative ideas, including the scientifically acceptable one. students then join the cartoon characters in debating the correct explanation. teacher facilitation and probing during the class discussion is important to students arriving at the scientifically correct concept. recently, cartoons were used to teach the affective topic of bullying. preservice teachers used cartoons (shown in rule, logan & kohler, 2012) to teach elementary students about the causes, consequences, and mitigations of bullying (rule, logan, & kohler, in press) with positive results of stimulating interactive discussions, motivating students to participate, and creating a more positive classroom climate. cartoons have been used in conjunction with cognitive behavioral techniques to help students change their thinking patterns related to optimism and other psychological states. ellis (1962) developed an ‘abc’ model based on the idea that people react differently to the same event because of belief systems. the ‘a’ of this model is the ‘activating event’ mediated by ‘b’ – ‘beliefs’ and resulting in ‘c’ ‘consequences’ of emotions and behaviors. a three-panel abc cartoon was used to help pessimistic adolescents identify the link between their thoughts, feelings, and behaviors. the first frame shows an activating event, while the third frame shows the negative emotional result on a person. the middle frame shows blank thought bubbles (‘b’ – beliefs) that the adolescent then completes in a logical manner as a bridge between a and c. identifying how the cartoon person’s beliefs have resulted in the emotional consequences of the third panel gives the student insight into his or her own automatic thoughts and styles of processing information, and is followed by a deeper process of analysis without cartoons (gillham & reivich, 2004). this therapeutic cartoon technique operates differently than the cartoon strategy in the current study in two main ways: (1) humor or clever puns are not used to motivate the student; and (2) the point of the cartoon is to help the student in self-analysis rather than to illustrate an academic concept. however, both cartoon strategies do provide application of concepts in many different scenarios. because of the motivating humor of cartoons and the opportunity for self-expression through completing partly-finished cartoons or creating original ones, the investigators of the current study wondered if cartoons might be successfully applied to teaching elementary students about perfectionism. we chose the widespread, traditional teaching technique of bibliotherapy as a comparison because this teaching strategy is often encountered in classrooms and has been used successfully for decades. the authors of the current study had permission to use a set of cartoons focusing on perfectionism concepts that were made by graduate students in a gifted education course taught by the second author. in that graduate course, teachers preparing to be educators of gifted students researched perfectionist concepts and completed partly-finished cartoons furnished by the instructor that had been created with clip art and drawing functions in powerpoint. the cartoons submitted by graduate students underwent peer review by classmates who each chose the ten most effective in content and humor, describing positive attributes and making suggestions for teaching perfectionism through cartoons / zousel, rule & logan 204 improvements. these reviews were used to choose and improve upon the pool of cartoons used in the current study, which was designed to test their efficacy with primary-age students. the following are research questions addressed by this study: 1. which of the two methods, cartoon analysis and production, or bibliotherapy using books featuring characters experiencing perfectionism, results in greater student learning of perfectionism concepts? 2. what aspects of both methods are particularly effective or engaging for students based on teacher observation? 3. which method of teaching do students perceive as most enjoyable and why? methods participants and research setting forty-six elementary students of mixed abilities (21 male, 25 female; 32 european-americans, 4 hispanic, 8 african-american, 1 native american, 1 asian american) in three grades (first, second, and third; ages 6 to 9 years) at a suburban public school in iowa, united states participated in the study. seventeen percent of the school’s population received free or reduced-price lunches, indicating the generally middle class socio-economic status of the remaining school population. because two mixed-ability, inclusive classes existed at each grade level, one of the two at each grade level was randomly chosen to be part of the control group (11 male, 12 female; 5 first graders, 11 second graders, 7 third graders) while the remaining classes composed the experimental group (10 female, 13 male; 7 first graders, 7 second graders, 9 third graders). permission to conduct the study was obtained from the overseeing university’s human subjects review committee and the school principal. students included in this study and their parents agreed in writing to participate. research design the research design was a pretest-posttest experimental group-control group design. both conditions were taught by the same enthusiastic teacher who was the school’s education of the gifted teacher. her role at the school was to provide enrichment lessons to all students in their classrooms and to deliver high-level instruction to identified gifted students. she had taught several successful lessons about perfectionism previously at another school using several children’s books. however, she was curious as to how cartoon lessons would fare with students. the lessons reported here were enrichment lessons delivered to all students in the primary grades (grades 1, 2, and 3) of the school as enhancement in academic and personal/social development. regular classroom teachers assisted when the lessons were conducted in their classrooms. four thirty-minute lessons were delivered to each of the classrooms involved (according to the randomly assigned condition). the lessons for each condition addressed the same concepts about perfectionism as shown in table 1. table 1. set-up of the study order perfectionism concepts taught pretest lesson 1 30 min. some perfectionism is good, but it can be “bad” or unhealthy also. order, neatness, organization are good aspects of perfectionism. no one can really be or do everything perfectly. pressure from parents or others may cause perfectionism. table 1 (continue). set-up of the study order perfectionism concepts taught international electronic journal of elementary education vol.5, issue 2, 199-218, 2013 205 lesson 2 30 min. some perfectionists are obsessive. perfectionism can result in depression when those who aren’t perfect think they are worthless. perfectionists sometimes procrastinate because they are afraid they may not perform perfectly. need risk-taking in a safe environment. lesson 3 30 min. perfectionists tend to set unreachable goals. some perfectionists cheat to maintain a perfect record. perfectionism kills creativity. accepting advice and constructive criticism is good. unhealthy perfectionists have trouble with criticism because then they feel worthless. it’s okay to be different. lesson 4 30 min. it is important to enjoy process and the journey rather than focusing on the result. even if something is not perfect, it can still be good and fun. accepting some imperfections indicates healthy perfectionism. all or none thinking indicates unhealthy perfectionism. errors are common; it is okay to make some mistakes. posttest after one week same for all participants lesson procedures experimental condition. at the start of each of the four cartoon lessons, each student was given a piece of paper with six cartoons printed on it to use and keep. every cartoon had a number so that the teacher could easily refer to it during the lesson. first, student volunteers or the teacher read the caption for the cartoon and any talking bubbles. the teacher asked questions to lead a discussion of each cartoon: ‘what is happening?’ and ‘how does the character feel?’ ‘why?’ ‘can you think of a time or situation in which you felt this way?’ ‘what is interesting or funny about the cartoon?’ twenty-four different cartoons were used during the four lessons for the experimental group. figures 1 through 3 are example cartoons used in the lessons. at the first lesson, after this initial cartoon discussion (which took most of the half hour because the perfectionism concepts and use of cartoons were new), the teacher showed a large image of a partly-finished cartoon to the class and asked for suggestions on how to complete it to show one or more of the perfectionism ideas that had been discussed during the lesson. the cartoon was then completed with ideas from the students. after the initial discussion of all cartoons in lesson two (as described in the previous paragraph), each student was provided a page of several partly-drawn cartoons and given time to complete one or more with perfectionism ideas just learned. these were shared with the class as time permitted. to introduce more variety on lesson three, after the discussion of new cartoons, students were divided into small groups of four students. each group was assigned one of the cartoons just discussed and given a few minutes to prepare a short skit showing the main perfectionism concept of their cartoon but applied to a new situation. student groups then presented their skits to the rest of the class and class members identified the perfectionism concept being portrayed. more cartoons and perfectionism concepts were introduced during the final cartoon lesson and discussed as described previously. then, students worked together to suggest ideas and to complete a three-panel cartoon strip about perfectionism concepts. teaching perfectionism through cartoons / zousel, rule & logan 206 figure 1. example lesson cartoon showing unhappiness of neurotic perfectionists figure 2. example lesson cartoon showing family pressures figure 3. example lesson cartoon showing lack of balanced perspective of neurotic perfectionists control condition. the control group learned concepts about perfectionism through reading and discussion of children’s books – bibliotherapy. table 2 shows an annotated list of books used in the control group lessons. all books were quality children’s books that had received favorable reviews or recommendations as indicated in this table. each of the four bibliotherapy lessons began with the teacher introducing the perfectionism concept and asking students if they had ever felt that way. then, for the first three lessons, the teacher read the book to the students, stopping to show illustrations and to ask questions about the story. a short discussion of perfectionism concepts related to the story ensued. after the first story and class discussion on alexander’s terrible day, the teacher asked students to write a letter to alexander about his desire for everything to be perfect and how he might consider a new viewpoint. these were shared as time permitted. in lesson two, after reading the story about the carved duck, students first discussed with a partner, then as a whole class, a time when felt they were less than perfect, comparing their feelings to those international electronic journal of elementary education vol.5, issue 2, 199-218, 2013 207 of daniel in the story. in the third lesson, students talked with a partner after the counting rope story to generate ways each person is unique and to think of an example of a situation in which one might enjoy the process even when the result is less than perfect. these ideas were shared with the class as time permitted. the last book was presented as a reading rainbow video of the book rather than a read-aloud by the teacher. after the video, the teacher led a discussion of perfectionism concepts related to the story. table 2. books use to teach perfectionist concepts to the control group. lesson book favorable review or award summary 1 alexander and the terrible, horrible, no good, very bad day (viorst, 1987) included as one of educators’ top 100 children’s books (national education association 2007). listed as recommended literature (k-12) by the california department of education (2002). alexander’s day goes from bad to worse and it seems that nothing is going his way. eventually, alexander realizes that no matter who he is or where he lives, bad days will happen. this book emphasizes that no one can be or do everything perfectly. 2 daniel’s duck (bulla, 1982) favorably reviewed by the new york times (paperbacks: new and noteworthy, 1983) daniel’s duck is a story about a young, aspiring wood carver. at a trade show, daniel’s duck draws attention that he feels is negative. he experiences feelings of worthlessness and depression, but with the help of a fellow wood carver, is able to appreciate imperfections. 3 knots on a counting rope. (martin & archambault, 1997 included as one of educators’ top 100 children’s books (national education association 2007). listed as recommended literature (k-12) (california department of education, 2002). a book focusing on differences, knots on a counting rope follows a traditional native american family. the main character understands and accepts his physical differences with the help of his grandfather. he learns that even though he didn’t meet his goal, he enjoyed the process and tried his personal best. 4 regina's big mistake (moss, 1990) favorably reviewed by sutherland (1990) regina is an elementary student who is attempting to create an original work of art. her classmates criticize her abilities and ideas. by the end of the story, regina is able to make a masterpiece from her mistakes. regina accepted her imperfections and made the most out of them. teaching perfectionism through cartoons / zousel, rule & logan 208 instrumentation the pretest-posttest was constructed by the teacher with the first five items criterionreferenced to the main concepts taught about perfectionism. it contained the following response items (some additional items appeared on the posttest as shown later): 1. tell what the word ‘perfectionism’ means. 2. tell at least four ways that perfectionism can be good – tell the characteristics or actions of a healthy perfectionist. 3. tell at least four ways perfectionism can be bad – tell the characteristics or actions of an unhealthy perfectionist. 4. tell at least two causes of perfectionism. 5. tell four ways to stop or overcome unhealthy perfectionism. additionally, the posttest contained the following items: 6. tell what you liked the most about the way you learned about perfectionism from the teacher. 7. circle a number below (a scale from 1 to 10 was given with ‘didn’t like it at all’ written by ‘1’ and ‘loved learning about this topic’ written by ‘10’) to tell how much you liked learning about this topic of perfectionism compared to other lessons from this teacher. 8. tell three reasons why you rated the perfectionism lessons this way. possible correct responses to these pretest-posttest items were generated by the researchers prior to the lessons being taught. to ensure that students received the same perfectionism content knowledge during both conditions, the teacher reviewed the concepts before teaching the lessons of both conditions, making sure they were all addressed by her teaching of the lessons in both conditions. the pretest was given the day before the lessons started, while the posttest was given a few days following the last lesson. on both the pretest and posttests, students spent some time attempting to answer the questions by writing independently, but then were individually assisted by the teacher (while other students worked on another assignment) by recording their verbal answers in her writing on the questionnaire as needed. this ensured that lack of writing skill did not prevent a student from showing what he or she knew. results and discussion pretest and posttest the pretest and posttest results of the five perfectionism content questions asked on both instruments are shown in table 3. students in both conditions knew little to nothing on the pretest, whereas in both conditions, some students were able to answer most of the components of each question correctly. this confirms that students were presented with and learned concepts of perfectionism in both of the conditions. however, students in the experimental condition supplied significantly more correct responses on the posttest than those in the control condition. a t-test was conducted to determine if the higher posttest score for the experimental group were statistically significant. the p-value was 0.02, indicating that the experimental group scored significantly higher on the posttest questions. the effect size, cohen’s d, was 0.5 indicating a medium effect size (cohen, 1988). international electronic journal of elementary education vol.5, issue 2, 199-218, 2013 209 table 3. student responses to pretest and posttest questions correct response to pretest/ posttest questions number of correct responses control group experimental group pretest posttest pretest post test 1. definition of perfectionism perfectionism means thinking you have to be perfect at everything. 0 10 0 11 2. ways perfectionism can be good or characteristics of a healthy perfectionist being organized and orderly 0 1 0 9 dedicated, hard-working, attention to detail 0 7 0 5 open to advice and criticism 0 2 0 1 people depend on them 0 0 0 3 happy and not depressed 0 1 0 6 total score for question 2 0 11 0 24 3. ways perfectionism can be bad –characteristics or actions of an unhealthy perfectionist sets unreasonable goals. 0 1 0 2 doesn’t enjoy or have fun unless he/she thinks he/she is perfect. 0 1 0 5 may procrastinate or not try at all. 0 2 1 0 may be grumpy, depressed, or feel worthless. 0 12 1 8 they may cheat to get good grades. 0 0 0 2 may be obsessive. 0 0 0 3 be afraid to be different and creative. 0 1 0 2 total score for question 3 0 17 2 22 4. causes of perfectionism living for approval from people who praise too much 0 0 0 3 constant success early in life without experience of struggle or failure 0 2 0 1 parents who are too pushy 0 1 0 0 total score for question 4 0 3 0 4 5. ways to stop or overcome unhealthy perfectionism enjoy the process, rather than focusing on the final product. 0 0 0 1 talk with others about your feelings. 0 1 0 4 set more reasonable goals. 0 2 0 1 learn to accept imperfection. 0 2 1 8 be willing to try new things, take risks, and be different. 0 2 0 3 total score for question 5 0 7 1 17 cumulative score for all questions 0 48 3 78 table 4 shows the aspects students stated they liked most about the lessons. these categories were developed by the constant comparison method in which similar responses to each of the questions were grouped into categories while simultaneously comparing all the responses to the question. the categories were refined as additional responses were read that teaching perfectionism through cartoons / zousel, rule & logan 210 shifted the category labels and defined new relationships (dye, schatz, rosenberg, & coleman, 2000). in general, the main conveyance of the perfectionism ideas for the condition, books or cartoons, was cited the most in its respective condition. although the same teacher delivered all lessons with enthusiasm, students in the experimental condition focused more on the lesson activities as being what they liked most about the lessons, in contrast to the four people of the control condition that said they liked the teacher herself the most. more students in the control condition replied ‘nothing’ or left the answer blank than in the experimental condition, possibly indicating more apathy toward the lessons. table 4. aspects students liked most about the lessons aspect about lessons students liked most number of students indicating this aspect control group n=23 experimental group n=23 the cartoons and cartoon work 0 12 the books the teacher presented 7 0 the teacher 4 1 acting things out in the skit 0 4 learning to like myself the way i am 3 1 nothing/no idea/ no response 8 4 everything 1 1 student ratings of the instructional condition were also analyzed by tabulating the numerical ratings and computing a mean and standard deviation. students in the control condition rated their liking of the way they learned about perfectionism compared to other lessons presented by the same gifted education teacher. students in the control group responded with an average rating of 7.0 (standard deviation = 3.5) out of 10, while students in the experimental group rated the lessons as 8.3 (s. d. = 2.4) out of 10. a t-test showed that the experimental group’s higher rating was statistically significant only at the alpha = 0.10 level (p = 0.08). table 5 shows the reasons students gave for their ratings. the control group gave somewhat fewer positive reasons and provided quite a few more neutral to negative reasons (about a third of the reasons given were neutral or negative) than the experimental group. another cartoon study used lesson reflection data from preservice teachers who taught three short cartoon-based lessons on bullying to special education-inclusive groups of elementary students to evaluate the efficacy of the cartoon teaching strategy (rule, logan & kohler, in press). in that study, preservice teachers reported that the concise nature of the reading task associated with the cartoons, the attention-getting humor (students must pay attention and comprehend the action in the cartoon to understand the joke), and interesting visual nature kept student attention on the topic. these qualities of this cartoon teaching strategy are likely the reasons students rated the cartoon condition higher than the bibliotherapy condition in the current study. international electronic journal of elementary education vol.5, issue 2, 199-218, 2013 211 table 5. reasons given by students for rating the way they learned about perfectionism reasons given number of students giving this reason control group n= 23 experimental group n= 23 positive reasons the lessons were awesome, fun, or interesting. 8 5 the skits were engaging 0 6 the stories or books were interesting. 6 0 the lessons helped me learn about my own perfectionism. 6 4 the cartoons were engaging. 0 5 i like the teacher. 2 4 the lessons were relaxing. 1 0 i like group work. 0 1 total positive reasons 23 25 neutral or negative reasons i liked some parts but didn’t like others. 4 0 i don’t know. 1 0 the lessons were boring. 4 0 the work was hard or difficult. 3 2 i dislike criticizing myself or others. 0 1 total neutral or negative reasons 12 3 teacher observations the control group was initially receptive to bibliotherapy. students were excited to listen to a new book and participate in a whole-class discussion. however, during the second, third, and fourth lessons the excitement diminished markedly. students’ remarks included, ‘this is boring,’ and, ‘we are reading another book?’ when relating perfectionism to the books being read, students’ responses were limited. they made personal connections to the text, but not as many connections to perfectionism as the teacher had hoped. discussion prompts seemed ineffective at times, as student attention and excitement had been lost. students taught about perfectionism through the use of cartoons were engaged in the lessons from beginning to end. there was an aura of excitement in the classrooms when the teacher said, ‘time to pass out the cartoons we will be discussing.’ students smiled, laughed, and said, ‘awesome!’ during elbow partner time students shared their learning and personal examples of ways they had experienced perfectionism related to the cartoons. it was truly an teaching perfectionism through cartoons / zousel, rule & logan 212 interactive and engaging experience. during a second grade lesson, the regular classroom teacher from the control group walked into the experimental class to borrow materials. she later pulled the gifted education teacher aside and said, ‘wow, they really looked like they were having fun. can you do the cartoon lessons with my students next year?’ a factor that may have affected student interest in bibliotherapy is the movement in most elementary schools in the united states to emphasize literacy (dee & jacob, 2010) because of the high stakes standardized tests required by no child left behind legislation (2002). in many schools, including the school in the current study, extra reading lessons are being implemented to improve student skills and test scores. students may feel saturated with book-reading and therefore not as interested in bibliotherapy as students of past decades. cartoon interpretation and production are part of literacy education and therefore could provide variety in the reading curriculum while improving student proficiency, much as graphic novels have motivated many reluctant readers through their engaging illustrations (martin, 2009). student cartoons the teacher provided students with half-finished cartoons (zimmerman, 2012) so that they could complete them to express what they were learning about perfectionism. figure 4 shows the work of two students who completed a cartoon about a person and a thinking cat. both the cartoons show the positive characteristics of healthy perfectionists of being orderly and organized. the top cartoon also contains a pun with the word ‘purrfect.’ two other students, responding to another cartoon template, expressed additional behaviors of perfectionists – being self-critical (the upper cartoon) and having the tendency to avoid situations in which success is not guaranteed (the lower cartoon). the cartoons made by students were evidence that they understood some of the perfectionism concepts being taught in the experimental condition. figure 4. student-completed cartoons using person and cat. international electronic journal of elementary education vol.5, issue 2, 199-218, 2013 213 figure 5. student-completed cartoons featuring man and butterfly. conclusion limitations of the study this investigation examined the effects of teaching perfectionism concepts through cartoons or bibliotherapy to first through third grade students at one suburban school in iowa, united states. although it does provide evidence for the efficacy of teaching with cartoons in this small population, these results may or may not be generalizable to other populations such as urban students of low socio-economic status or students with english as a second language who may have difficulty with puns and subtle humor aspects of the cartoons. additional studies may provide more evidence for this technique with other populations. implications of the findings students demonstrated greater learning of perfectionism concepts along with more excitement and engagement using cartoons than bibliotherapy. humor captivated students’ attention in the cartoon condition in contrast to observed and reported boredom or apathy during bibliotherapy lessons. however, the average ratings of lesson enjoyment for both conditions were quite high (7.0 versus 8.3 out of a possible 10), indicating that students enjoyed both sets of lesson activities for the most part. therefore, we recommend that teachers not abandon bibliotherapy, but add cartoon analysis and creation as another effective strategy for teaching about the important concepts of perfectionism. suggestions for future research teaching with cartoons is a relatively new strategy for subject areas outside of history and political science. cartoons, because of their visual nature, humorous aspects, and concise delivery of content, attract students’ attention, motivating them to consider the message of the cartoon and to understand any puns or humor. completing partly-made cartoons or creating original cartoons allows students self-expression and engages them through allowing teaching perfectionism through cartoons / zousel, rule & logan 214 communication of their humorous or clever ideas. more research into the use of cartoons for teaching other topics besides perfectionism, bullying, and science needs to be conducted as this is a promising approach that engages students. . . . acknowledgements the cartoons shown in this article for teaching about perfectionism were made by melissa wolf, susan boatwright, and miranda zousel. the comic strip panels shown in this article as primary grade students' work were made with cartoon templates from http://www.makebeliefscomix.com. used by permission of author and site creator bill zimmerman. miranda zousel is a gifted education teacher at the marion school district in iowa, usa. she received her master’s degree in education of the gifted from the university of northern iowa in 2012. audrey c. rule is an associate professor and coordinator of the education of the gifted graduate program at the university of northern iowa, usa, teaching graduate courses in creativity, problemsolving, gifted education, and science education. stephanie r. logan is an assistant professor in the department of curriculum and instruction, university of northern iowa, usa. she has several years experience counseling gifted students and teaches a graduate course in this area. references adelson, j. l. (2007). a ‘perfect’ case study: perfectionism in academically talented fourth graders. gifted child today, 30(4), 14-20. berk, r. a. (2002). humor as an instructional defibrillator: evidence-based techniques in teaching and assessment. sterling, va: stylus. beck, a. t., rush, a. j., shaw, b. f., & emery, g. (1979). cognitive therapy of depression. new york: guilford press. bickford, j. (2011c). a comparative analysis of original political cartoon methodology: concept maps and substitution lists. social studies research and practice, 6(3), 65-80. bieling, p. j., israeli, a. l., & antony, m. m. (2003). is perfectionism good, bad or both? examining models of the perfectionism construct. personality and individual differences, 36, 1373-1385. bieling, p. j., israeli, a. l., smith, j., & antony, m. m. (2003). making the grade: the behavioural consequences of perfectionism in the classroom. personality and individual differences, 35, 163–178. boone, l., soenens, b., braet, c., & goosens, l. (2010). an empirical typology of perfectionism in earlyto-mid adolescents and its relation with eating disorder symptoms. behaviour research and therapy, 48(7), 686-691. brown, g. p., & beck, a. t. (2002). dysfunctional attitudes, perfectionism, and models of vulnerability to depression. in g. l. flett & p. l. hewitt (eds.), perfectionism: theory, research, and treatment (pp. 231–251). washington, dc: american psychological association. http://www.makebeliefscomix.com/ international electronic journal of elementary education vol.5, issue 2, 199-218, 2013 215 brown, a. j., parman, k. m., rudat, d. a., & craighead, l. w. (in press, 2012). disordered eating, perfectionism, and food rules. eating behaviors, available online as an in-press article. bulla, c. r. (1982). daniel’s duck. new york: harper trophy. burns, l. r., & fedewa, b. a. (2005). cognitive styles: links with perfectionistic thinking. personality and individual differences, 38, 103–113. brophy, j., & rohrkemper, m. (1989). teachers’ strategies for coping with perfectionist students. research series no. 198. east lansing, mi: institute for research on teaching, michigan state university. eric document reproduction service no. ed 314 401. california department of education. (2002). recommended literature (k-12). retrieved january 7, 2013 from http://www.cde.ca.gov/ci/rl/ll/ clark, m. a., lelchook, a. m., & taylor, m. l. (2010). beyond the big five: how narcissism, perfectionism, and dispositional affect relate to workaholism. personality and individual differences, 48, 786-791. cohen, j. (1988). statistical power analysis for the behavioral sciences (2nd ed.). hillsdale, nj: lawrence erlbaum associates. cohen, l. (1996, fall). dealing with perfectionism: piaget's dynamic equilibrium. understanding our gifted, 9(1), 12-17. davies, l. (2005). perfectionism in children. retrieved january 7, 2013 from: http://www.kellybear.com/teacherarticles/teachertip53.html davis , m. c., & wosinski, n. l. (2012). cognitive errors as predictors of adaptive and maladaptive perfectionism in children. journal rational-emotive and cognitive-behavior therapy, 30, 105-117. dee, t. & jacob, b. (2010). the impact of no child left behind on students, teachers, and schools. brookings papers on economic activity 59(2), 149-207. dekryger, n. (2005). childhood perfectionism: measurement, phenomenology, and development. ph.d. dissertation. university of louisville, louisville, ky. diprima, a. j., ashby, j. s., gnilka, p. b., & noble, c. l. (2011). family relationships and perfectionism in middle school students. psychology in the schools, 48(8), 815-827. dohnt, h., & tiggemann, m. (2006). the contribution of peer and media influences to the development of body satisfaction and self-esteem in young girls: a prospective study. developmental psychology, 42(5), 929-236. dougherty, b. (2002). comic relief: using political cartoons in the classroom. international studies perspectives, 3, 258-270. dunkley, d. m., zuroff, d. c., & blankstein, k. r. (2003). self-critical perfectionism and daily affect: dispositional and situational influences on stress and coping. journal of personality and social psychology, 84, 234–252. dye, j. f., schatz, i. m., rosenberg, b. a., & coleman, s. t. (2000). constant comparison method: a kaleidoscope of data. the qualitative report, 4(1-2) retrieved january 7, 2013 from http://www.nova.edu/ssss/qr/qr4-1/dye.html ekici, f., ekici, e, & aydin, f. (2007). utility of concept cartoons in diagnosing and overcoming misconceptions related to photosynthesis. international journal of environmental and science education, 2(4), 111-124. ellis, a. (1962). reason and emotion in psychotherapy. new york: lyle stuart. flett, g. l., & hewitt, p. l. (eds.). (2002). perfectionism: theory, research, and treatment. washington, dc: american psychological association. teaching perfectionism through cartoons / zousel, rule & logan 216 flett, g. l., & hewitt, p. l. (2006). positive versus negative perfectionism in psychopathology: a comment on slade and owens’ dual process model. behavior modification, 30, 472–495. frost, r. o., marten, p., lahart, c., & rosenblate, r. (1990). the dimensions of perfectionism. cognitive therapy and research, 14, 449–468. gillham, j., & reivich, k. (2004). cultivating optimism in childhood and adolescence. the annals of the american academy of political and social science, 591, 146-163. gilman, r., & ashby, j. s. (2006). perfectionism. in: children's needs iii: development, prevention, and intervention. bear, george g. (ed.); minke, kathleen m. (ed.); washington, dc, us: national association of school psychologists, pp. 303-312. grzegorek, j. l., slaney, r. b., franze, s., & rice, k. g. (2004). selfcriticism, dependency, self-esteem, and grade point average satisfaction among clusters of perfectionists and nonperfectionists. journal of counseling psychology, 51, 192–200. doi:10.1037/0022-0167.51.2.192 hamachek, d. e. (1978). psychodynamics of normal and neurotic perfectionism. psychology: a journal of human behavior, 15, 27–33. hanchon, t. a. (2011). examining perfectionism through the lens of achievement goal theory. north american journal of psychology, 13 (3), 469-490. hewitt, p. l., & flett, g. l. (1991). perfectionism in the self and social contexts: conceptualization, assessment, and association with psychopathology. journal of personality and social psychology, 60, 456–470. hewitt, p. l., flett, g. l., besser, a., sherry, s. b., & mcgee, b. (2003). perfectionism is multidimensional: a reply to shafran, cooper and fairburn. behaviour research and therapy, 41(10), 1221–1236 doi: s0005796703000214 [pii]. hewitt, p. l., flett, g. l., turnbull-donovan, w., & mikail, s. f. (1991). the multidimensional perfectionism scale: reliability, validity, and psychometric properties in psychiatric samples. psychological assessment, 3(3), 464–468. kabapinar, f. (2005). effectiveness of teaching via concept cartoons from the point of view of constructivist approach. colección digital eudoxus. retrieved january 7, 2013 from http://cimm.ucr.ac.cr/ojs/index.php/eudoxus/article/viewarticle/136 keogh, b., & naylor, s. (1999). concept cartoons, teaching and learning in science: an evaluation. international journal of science education, 21, 431-446. marano, h. e. (2008, mar/apr). pitfalls of perfectionism. psychology today, 41(2), 80-87. retrieved january 7, 2013 from: http://www.psychologytoday.com/articles/200802/pitfalls-perfectionism martin, a. (2009). graphic novels in the classroom. library media connection, 28(2), 30-31. martin jr., b., & archambault, j. (1997). knots on a counting rope. new york: henry holt and company. moss, m. (1990). regina’s big mistake. boston: houghton mifflin harcourt. national education association. (2007) educators’ top 100 children’s books. retrieved january 7, 2013 from http://www.nea.org/grants/13154.htm/ neihart, m. (2002). gifted children and depression. in m. neihart, s. reis, n. robinson & s. moon (eds.), the social and emotional development of gifted children. waco, tx: prufrock press. nobel, r., manassis, k., wilansky-traynor, p. (2012). the role of perfectionism in relation to an intervention to reduce anxious and depressive symptoms in children. journal of rational-emotive and cognitive behavior therapy, 30, 77-90. no child left behind (nclb) act of 2001, pub. l. no. 107-110, § 115, stat. 1425 (2002). nugent, s. a. (2000). perfectionism: its manifestations and classroom-based interventions. journal of secondary gifted education, 11(4), 215-221. international electronic journal of elementary education vol.5, issue 2, 199-218, 2013 217 o’connor, r., rasmussen, s., & hawton, k. (2010). predicting depression, anxiety, and self-harm in adolescents: the role of perfectionism and acute life stress. behaviour research and therapy, 48, 5259. pacht, a. r. (1984). reflections on perfectionism. american psychologist. 39, 386-390. paperbacks: new and noteworthy. (1983, august 7). the new york times, p. 31. rice, k. g., & ashby, j. s. (2007). an efficient method for classifying perfectionists. journal of counseling psychology, 54, 72 – 85. rice, k. g., & mirzadeh, s. a. (2000). perfectionism, attachment, and adjustment. journal of counseling psychology, 47, 238–250. risinger, c. & heitzmann, r. (2008). using the internet to teach about political cartoons and their influence on u.s. elections. social education, 72(6), 288-290. rudolph, s. g., flett, g. l., & hewitt, p. l. (2007). perfectionism and deficits in cognitive emotion regulation. journal of rational-emotive and cognitive behavior therapy, 25, 343–357. rule, a. c., and auge, j. (2005). using humorous cartoons to teach mineral and rock concepts in sixth grade science class. journal of geoscience education, 53(5), 575-585. rule, a. c., logan, s. r., & kohler, f. w. (2012). cartoons for teaching elementary students about bullying. eric document reproduction service no. ed532956. rule, a. c., logan, s. r., & kohler, f. w. (in press). preparing preservice teachers to address bullying through cartoon lessons. journal of the american academy of special education professionals. sallis, d. a., rule, a. c., & jennings, e. (2009). cartooning your way to student motivation. science scope, 32 (9), 22-27. schuler, p. a. (2000). perfectionism and gifted adolescents. journal of secondary gifted education, 11(4), 183-196. schuler, p. (2002). perfectionism in gifted children and adolescents. in m. neihart, s. reis, n. robinson & s. moon (eds.), the social and emotional development of gifted children. waco, tx: prufrock press. shafran, r., cooper, z., & fairburn, c. g. (2002). clinical perfectionism: a cognitive-behavioural analysis. behaviour research and therapy, 40, 773-791. silverman, l. k. (2011). the many faces of perfectionism. gifted homeschoolers’ forum. retrieved march 22, 2012 from: http://www.giftedhomeschoolers.org/articles/facesofperfectionism.html slade, p. d., & owens, r. g. (1998). a dual process model of perfectionism based on reinforcement theory. behavior modification, 22, 372–390. stoeber, j., hutchfield, j., & wood, k. v. (2008). perfectionism, self-efficacy, and aspiration level: differential effects of perfectionistic striving and self-criticism after success and failure. personality and individual differences, 45, 323-327. stoeber, j., kempe, t., & keogh, e. j. (2008). facets of self-oriented and socially prescribed perfectionism and feelings of pride, shame, and guilt following success and failure. personality and individual differences, 44, 1506-1516. stoeber, j., & otto, k. (2006). positive conceptions of perfectionism: approaches, evidence, challenges. personality and social psychology review, 10, 295–319. stoeber, j., & rambow, a. (2007). perfectionism in adolescent school students: relations and motivation, achievement, and well-being. personality and individual differences, 42, 1379-1389. stoeber, j., & yang, h. (2010). perfectionism and emotional reactions to perfect and flawed achievements: satisfaction and pride only when perfect. personality and individual differences, 49, 246-251. teaching perfectionism through cartoons / zousel, rule & logan 218 sutherland, z. (1990). regina’s big mistake. the bulletin of the center for children’s books, 44(2), 40. trefts, k. & blakeslee, s. (2000). did you hear the one about the boolean operators? incorporating comedy into library instruction. reference services review, 28(4), 369-378. viorst, j. (1987). alexander and the terrible, horrible, no good, very bad day. new york: atheneum. walsh, j. j., & ugumba-agwunobi, g. (2002). individual differences in statistics anxiety: the roles of perfectionism, procrastination and trait anxiety. personality and individual differences, 33, 239-251. wang, k. t., slaney, r. b., & rice, k. g. (2007). perfectionism in chinese university students from taiwan: a study of psychological well-being and achievement motivation. personality and individual differences, 42, 1279–1290. doi:10.1016/j.paid.2006.10.006 zimmerman, b. (2012). makebeliefs comix for educators. retrieved april 6, 2012 from http://www.makebeliefscomix.com/how-to-play/educators/ international electronic journal of elementary education, 9(3), 523-552, march, 2017 issn:1307-9298 copyright © iejee www.iejee.com u. s. fourth graders’ informational text comprehension: indicators from naep heather r. schugar a  mariam jean dreher b a west chester university, usa b university of maryland, usa received: 12 september 2016 / revised: 31 january 2017 / accepted: 14 february 2017 abstract this study is a secondary analysis of reading data collected from over 165,000 fourth graders as part of the u.s. national assessment of educational progress. using hierarchical linear modelling, the authors investigated factors associated with students’ informational text comprehension, including out-of-school reading engagement, and in-school measures of cross-curricular reading, discussion about reading, and reading-related activities (e.g., book reports, projects). in addition, this study examined the interactions between these factors, informational text comprehension, and students’ eligibility for free and reduced priced meals (farms). there were positive associations between students’ informational text comprehension and their reading engagement, crosscurricular reading, and discussion about reading. however, reading-related experiences were associated with lower than expected scores. in addition, out-of-school reading engagement and inschool reading experiences may not be associated with informational reading comprehension to the same degree for the most at-risk u.s. students, as most results differed in strength of association for farms-eligible students. keywords: informational text, content area literacy, reading engagement, discussion, poverty. introduction closing the achievement gap between students from middleand low-income homes has long been a goal in the united states (us) (coleman, et al., 1966) and remains a major concern as evident in the no child left behind act (2001). to address the achievement gap, reading performance is a top priority (bell, 2009/2010; united states department of education (usde), 2002). promising long-term trend data analyses have shown the gap between low-income and not low-income students in reading achievement narrowing in most states since the enactment of no child left behind (nclb); unfortunately, however, a sizable minority of states have shown a widening gap (e.g., at fourth grade 28% of states had a widening gap between low-income and non-low income students (chudowsky, chudowsky, & kober, 2009). moreover, although the results from these analyses suggest some progress toward the goals of nclb, the gap in reading achievement between  corresponding author: heather r. schugar, 107c recitation hall, west chester pa 19383 usa, phone: 1.610.696.3994, e-mail: hschugar@wcupa.edu http://www.iejee.com/ international electronic journal of elementary education, 9(3), 523-552, march, 2017 524 students from lower and higher socioeconomic backgrounds is still quite large. for example, on the 2011 fourth-grade national assessment of educational progress (naep) reading assessment, there was a 26-point gap (on a 500-point scale) between low-income students and their wealthier peers in public schools for literary reading and a 28-point gap for informational reading (calculated using naep’s data explorer: http://nces.ed.gov/nationsreportcard/naepdata/dataset.aspx). although the achievement gap is a concern for both literary and informational reading, we focus in this study on the reading of informational texts. elementary school students in the us performed statistically significantly lower on measures of informational reading than measures of literary reading on a recent international assessment, although this finding is not necessarily true for students worldwide. [see mullis, martin, kennedy, and foy (2007) for more information on the achievement differences of students in 40 countries.] in the us, informational texts are typically not a focus in reading instruction, particularly in the early grades (e.g., duke, 2000; jeong, gaffney, & choi, 2010; moss, 2008; ness, 2011). [see, however, smith (1986) and venezky (2000) for a view of informational text earlier in the history of us reading instruction.] but there is increasing recognition that without early exposure to this type of text, students may be unprepared to handle the unique demands of informational texts, especially exposition (pressley, wharton-mcdonald, mistrettahampston, & echevarria, 1998; williams, 2005) and may face difficulty as school tasks require them to independently comprehend these texts. as students progress through school and into the workplace, they are expected to comprehend more and more informational writing (common core state standards, www.corestandards.org/thestandards; venezky, 2000; white, chen, & forsyth, 2010), particularly in textbooks (chambliss & calfee, 1998) and on standardized tests (calkins, montgomery, santman, & falk, 1998; flood & lapp, 1986: national assessment governing board, 2008). the expository structures in textbooks and standardized tests can be challenging for many students, as exposition contains structures and features that differ from those found in narrative texts (chambliss & calfee, 1998). thus, without early experiences, students may not have the skills and strategies needed to understand these texts. in this study, we focus on fourth grade, which may be a particularly important juncture for examining informational reading. in her description of the stages of reading development, chall (1983, 1996) noted that it is at about the fourth grade that children must deal with school reading tasks that increasingly move from “learning to read” to “reading to learn.” as this shift in reading demands occurs, children must move beyond stories and relatively familiar vocabulary to deal with unfamiliar vocabulary and expository text structures. for many children, these changing expectations result in what chall termed a fourth-grade slump in reading development. indeed, chall, jacobs, and baldwin (1990) found that lowincome students may be at greater risk of a fourth-grade slump compared to their middleincome peers. among the reasons is that without exposure to a rich curriculum, they may not have the opportunity to develop the academic vocabulary necessary to comprehend informational texts. the purpose of our study was to identify factors that are associated with informational reading achievement at the fourth-grade level. to accomplish this purpose, we used data from the fourth-grade naep reading assessment. considering what factors are related to informational text comprehension may shed light on the achievement gap between students of differing socio-economic status. more specifically, by pinpointing aspects of students’ home and school reading that are associated with students’ informational text comprehension, we sought to provide information that will help inform future research on closing the income-achievement gap. u.s. fourth graders’ informational text comprehension: indicators from naep / schugar & dreher 525 important factors in the consideration of students’ informational text achievement despite the increased focus on the success of low-income students under nclb (usde, 2002), progress towards closing the achievement gap has been slow. for example, the reading achievement gap between low-income students and their peers did not narrow between 2007 and 2009 (national center for educational statistics (nces), 2009). furthermore, as noted, the 2011 naep results for fourth graders -– even after considerable lag time since the implementation of nclb -indicate a performance gap between low-income students and their wealthier peers for both literary reading and informational reading. the 2011 gap differs very little from the gap in 2002 naep performance just as the nclb act was passed (a 25-point gap for literary reading and a 29point gap for informational reading) or from 2005 data used in this study (26-point gap for literary reading and a 28-point gap for informational reading) when fourth graders would have been subject to nclb changes for approximately 2 ½ years before being assessed in the middle of the school year (naep’s data explorer). [a new naep reading framework effective in 2009 altered the definitions of informational and literary reading (nces, 2010b).] some scholars suggest that lack of opportunity to engage with content area materials such as science and social studies textbooks may be an important, but often overlooked, contributor to the achievement gap between students from differing socioeconomic backgrounds. chall and her colleagues (1990) noted that, typically, prior to fourth grade most of the focus of reading instruction is on decoding, fluency, and comprehension of familiar topics and vocabulary. at about fourth grade, students are increasingly expected to uncover the meaning of many technical, content area words, deal with unfamiliar topics and non-narrative text structures, and demonstrate higher order thinking skills. the change in focus between early elementary grades and upper elementary grades may put some low-income students who have not been exposed to a content-rich curriculum at a disadvantage in reading achievement when compared to their middle-income peers. this possibility is supported by longstanding evidence that a broad curriculum is important to reading achievement (singer, mcneil & furse, 1984). by providing early opportunities for students to learn from informational texts both in (dreher, 2000; neuman, 2006; 2010) and out of school (sonnenschein & schmidt, 2000), teachers and parents may expose students to vocabulary and text structures that they would not otherwise encounter. in the following section, we review some of the factors that may be associated with informational reading achievement, organizing them into two groups: (1) factors associated with out-of-school reading engagement and (2) factors associated with inschool reading experiences. because a comprehensive review of all variables related to out-of-school and in-school reading experiences is beyond the scope of this paper, we focused our review on factors that appear relevant to those targeted in the naep questionnaire items used in this study. factors associated with out-of-school reading engagement the types of experiences and interactions that students have with reading outside of school can contribute to their success with reading in school (gottfried, fleming, & gottfried, 1998). one important factor in students’ out-of-school reading engagement is their access to appropriate reading material. academic achievement is associated with the number of books in a child’s home (sheldon & carrillo, 1952). delpit (1988) and purcellgates (1995) have posited that children from low-income homes may not have access to interesting and diverse texts. in fact, feitelson and goldstein (1986) found that 60% of kindergarteners at low-performing schools had no books of their own at home. neuman and celano (2001) also found inequities in the number and quality of books available for international electronic journal of elementary education, 9(3), 523-552, march, 2017 526 purchase and at libraries in low-income areas compared to middle-income neighborhoods. access to appropriate reading material is important for early reading experiences (bus, 2003), and the lack of books in many low-income homes may partially explain why the average middle class child has engaged in over 1,000 hours of read alouds with family members before entering school, while low-income students are predicted to have on average only 25 hours of this type of interaction (adams, 1990). the type of talk that caregivers and children engage in may be another important factor in students’ out-of-school reading engagement. a number of studies have documented an association between parent socioeconomic status and children’s vocabulary growth, with children of higher socioeconomic status parents characterized by larger vocabularies and more rapid vocabulary growth (e.g., hart & risley, 1995; hoff, 2003; rowe, 2008). weizman and snow (2001), for example, noted “profound quantitative and qualitative differences in early vocabulary exposure among low-income preschoolers” (p. 276), with mothers’ use of sophisticated (low-frequency) vocabulary strongly associated with children’s vocabulary later in school. lareau (1989), on the other hand, found that lowincome and middle-income families engaged in similar types of education-related discussions, but the former did so less frequently. peer groups may be another motivating factor for out-of-school reading engagement. studies of library circulation records show that books are often shared among groups of friends and that reading peer-approved books may be important for children’s sense of group membership (moss & mcdonald, 2004). when children then discuss the books they are reading with their friends, research findings as well as expert opinion suggest that they are likely to have better comprehension than if they had not discussed them (goatley, brock & raphael, 1995; guthrie & mccann, 1996; ketch, 2005). experience with informational texts out of school is another factor that can contribute to children’s reading achievement. although experts have argued that children have access to very few expository materials in the home in comparison to narrative texts (moss, 2003), some common resources are more likely to provide access to exposition than others. the presence of newspapers in the home is associated with the science achievement of seventh graders (akyol, sungur, & tekkaya, 2010), which may suggest that students are gaining necessary scientific vocabulary and understanding of concepts from these resources. yet with circulation down nationwide, few students report reading these materials (nippold, duthie, & larsen, 2005; project for excellence in journalism, 2004). magazines are another source of informational text in the home. but magazines written for a male audience are more likely to contain exposition than those written for females, suggesting that children who read magazines geared towards boys would have more exposure to this text type (hall & coles, 1999). however, in one study of sixth graders, spear-swerling, brucker, and alfano (2010) found no correlation between magazine reading and reading achievement. a third source of informational texts in many homes is the internet. having the internet at home is related to overall reading achievement, even when socioeconomic status has been taken into account (atwell & battle, 1999). having access to the internet at home may also be related to informational text achievement, as the vast majority of texts on children’s websites are exposition and many academic tasks require students to strategically read informational internet sites (coiro & dobler, 2007; kamil & lane, 1998). these internet resources may also require students to navigate the multimodal nature of many online texts, requiring an advanced skill set in order to determine purposes for reading, monitor their selected “pathway” for reading, and decide which features are distracting to their purposes (coiro, 2011). in some countries (e.g., the united states, germany, and some canadian provinces), informational text, particularly exposition, is more difficult for children to read (mullis et al., 2007; langer, 1986), but out-of-school engagement with u.s. fourth graders’ informational text comprehension: indicators from naep / schugar & dreher 527 such texts may help facilitate children’s interaction with the challenges they may face when reading exposition especially given that often students have less opportunity to learn to read with this type of text in school (chambliss & calfee, 1998). factors associated with in-school reading experiences because children from low-income homes are less likely to engage in school-like tasks and discussions at home (coleman, et al., 1966), school may be particularly important for them. however, in the us, schools are often both racially and socioeconomically segregated (orfield, 2001), and schools that serve low-income students often have “fewer and lower-quality books, materials, computers […] as well as less-qualified and less experienced teachers…” (darling-hammond, 1995, p. 610). for example, access to interesting, appropriate, and varied texts at school is associated with overall student achievement (chall et al., 1990). yet us classrooms can differ drastically in the types of texts that are available for students. for instance, oakes and saunders (2002) found that one-third of students in california did not have textbooks available to them outside of school to do homework or study for exams – and that access to these resources varied considerably between students in at-risk schools and their counterparts. although access to resources in itself is not enough to jumpstart students’ reading motivation and achievement (morrow & weinstein, 1986), resources such as classroom libraries can certainly promote independent reading (martinez, roser, worthy, strecker, & gough, 1997) and achievement (applebee, langer, & mullis, 1988) when teachers use these resources in an effective manner (mcgill-franzen & allington, 1999). the role of teachers’ classroom practices has been cited as an important factor in closing achievement gaps worldwide. in new zealand, for instance, teachers who created rich literacy environments for their students had practices that were associated with closing the gender achievement gap (wilkinson, 1998). collaboration and student grouping are some of the important instructional techniques that are commonly recognized as ones that can enhance student learning (cohen, 1994; slavin, 1986). collaborative learning experiences are based on social and cognitive theories proposing that when students work together to achieve similar goals, they will attain more than each student could on his or her own (johnson & johnson, 1989/1990). students may particularly benefit from working with a “more knowledgeable other” who can scaffold content for them (vygotsky, 1978). another factor associated with students’ reading achievement is their ability to employ a variety of comprehension strategies (samuelstuen & bråten, 2005; willson & rupley, 1997). unfortunately, strategy instruction is evident in far too few classrooms (e.g., durkin, 1978/1979; pressley, 2002; taylor, pearson, clark, & walpole, 1999), because teachers are often unprepared to teach students how to use strategies (national institute of child health and human development (nichd), 2000). chall and her colleagues (1990) found that students who were afforded opportunities to practice strategies in their classrooms showed gains in their vocabulary and comprehension. yet students frequently struggle when expectations change from “learning to read” to “reading to learn” around fourth grade, oftentimes because they have not developed the skills and strategies needed to effectively do so (chall et al., 1990). research also supports the claim that cross-curricular reading at school is associated with reading achievement (fang & wei, 2010; morrow, pressley, smith, & smith. 1997; romance & vitale, 1992; vitale & romance, 2011). one of the most prominent research agendas regarding cross-curricular reading has been guthrie and colleagues’ conceptoriented reading instruction (cori). when reading and science instruction were integrated using the cori approach, students became intrinsically-motivated readers, and international electronic journal of elementary education, 9(3), 523-552, march, 2017 528 in turn made better use of reading strategies, had better reading comprehension, and read more often and broadly than students who did not participate in this engaging, integrated curriculum (guthrie, anderson, alao, & rinehart, 1999; guthrie et al., 1996, 2004, 2009; guthrie, wigfield, & vonsecker, 2000). cross-curricular reading practices may facilitate reading achievement for several reasons. for example, when students read science and social studies content, they are likely to develop the technical vocabulary and background knowledge necessary to comprehend informational texts. furthermore, it is not uncommon for more than half of all content on standardized reading tests to consist of informational passages (calkins, et al., 1998; national assessment governing board, 2008). thus familiarity with informational reading would likely prepare students for the demands of these tests. along with cross-curricular reading instruction, research indicates that students benefit from opportunities to participate in activities that allow them to extend their understanding of what they have read (beck & mckeown, 2001). similarly, articles in practitioner journals address the need to move away from worksheets, workbooks, and similar assignments toward engaging students in more meaningful, collaborative tasks that involve higher-level thinking about what they read (meyer, 2010; pincus, 2005). classroom discussion is one way to engage students in critically thinking about reading materials. some studies have found that when conversation is peer-led as opposed to teacher-led, it allows for more occasions for students to engage in higher-level thinking to resolve conflicts that arise when reading and responding to the text as well as to opportunities to use more complex language to talk about text (almasi, 1995; murphy, wilkinson, soter, hennessey, & alexander, 2009). however, many discussions that take place in schools follow a pattern of teacher initiation, student response, and teacher evaluation (ire), a participation structure that can limit the possibilities for students to engage in quality, high-level discussions (cazden, 2001). although evidence indicates that certain instructional practices result in better reading achievement than others, decisions made at the school or district level can constrain what teachers are able to do. for example, in an effort to raise student achievement, some schools require that teacher provide scripted instruction or strictly follow a basal reading program. such mandates constrain the range of what is read, the skills that are taught, and the types of interaction students are able to experience with teacher and peers (peasealvarez & samway, 2008). in such schools, instruction is often driven by standardized testing, which may reflect narrowly-focused literacy goals. evidence indicates that in the era of nclb, test-driven curricular constraints are common even in school districts with relatively high performance (valli, croninger, chambliss, graeber, & buese, 2008), but appear even more likely in high poverty, low performing school districts (center on education policy, 2007). however, highly-constrained reading programs rarely provide opportunities for wide reading or promote classroom discussions, activities that have been shown to have a positive impact on student reading comprehension (murphy et al., 2009). the characteristics of a school’s population also may influence students’ reading performance. extensive documentation indicates that a school’s student eligibility for free and reduced-price meals has a strong association with the performance of individual students. students in schools with high percentages of low-income students have lower academic achievement than low-income students who attend other schools (e.g., palardy, 2008; sirin, 2005). to understand students’ reading achievement requires researchers to consider both individual and school factors that may be associated with student performance, as well as u.s. fourth graders’ informational text comprehension: indicators from naep / schugar & dreher 529 both out-of-school and in-school factors that can contribute to their success. the current study investigates factors associated with students’ achievement through a multilevel model aimed at examining out-of-school and in-school factors related to individual and school-wide informational text comprehension. research questions four research questions guided this study: (1) to what degree do farms-eligible students differ from other students in their comprehension of informational text? (2) what individual reading experiences are associated with students’ comprehension of informational text? (3) to what degree are students in some schools better able to comprehend informational text than students in other schools? (4) what characteristics of schools are associated with students’ comprehension of informational text? method data source the data for this study were collected as part of the 2005 fourth-grade naep reading assessment. naep is the only ongoing assessment of us children’s academic progress. federally-mandated in 1969, naep uses nationally-representative samples to examine us students’ achievement in reading, mathematics, civics, science, writing, us history, geography, and the arts (usde, institute of educational sciences, & nces, 2007). data are collected in january through march of the testing year using both stratification and clustering sampling strategies. schools are selected based on their membership in particular explicit (e.g., public, roman catholic, private) and implicit (e.g., percent of minority status, local census division) strata. within selected schools, students are clustered, as student selection is limited by students’ enrollment in a school selected based on school strata membership. the theory behind this sampling technique is that schools selected based on their strata would house the subpopulations targeted by the naep assessment (nces, 2010a). detailed information on the naep assessment, including research design and analysis, can be found at http://nces.ed.gov/nationsreportcard/. the 2005 naep reading assessment gathered data from over 165,000 fourth graders using a reading achievement test, and student, teacher, and administrator questionnaires. the reading achievement test consisted of two-subscales: (1) reading for literary experience and (2) reading for information, each scored on a 0 to 500 scale. about half of the test was multiple choice, while the rest of the test consisted of both brief and extended constructed response questions. naep employs a matrix sampling design, so students answer only 2025 of the 100-170 questions on the achievement test. for questions students are not given, five plausible values are calculated based on their performance on other items and available demographic information. a 32-question student questionnaire enabled naep to collect information about variables such as classroom practices children had experienced, materials in their homes, and reading-related practices they engaged in outside of school, while a 47-question teacher questionnaire and a 19-question administrator questionnaire allowed for the collection of classroom and school information including reading experiences and practices. in the current study, we used achievement data from the reading for information subscale and children’s questionnaire responses, as described in measures. measures measures of students’ background information. in our analysis, we examined data collected from school records regarding students’ (a) race/ethnicity, (b) gender, and (c) free and reduced-price meals (farms) eligibility (usde et al., 2007). since race/ethnicity was not a variable that we were focusing on in this study, we decided to simplify race into international electronic journal of elementary education, 9(3), 523-552, march, 2017 530 a dichotomous variable to minimize the complexity of the analysis model, described below. based on overall naep performance on reading for information, we grouped caucasians, asians, and pacific islanders together in one racial category, and blacks, hispanics, and american indians into another category. the rationale for this grouping was based on past trends in achievement between these racial groups (naep’s data explorer). in fact, on the 2005 fourth grade reading assessment, whites (m = 229, sd = 0.2) and asians/pacific islanders (m = 229, sd = 0.7) had the same mean average score, while black (m = 200, sd = 0.3), hispanic (m = 203, sd = 0.5), and american indian/alaskan native (m = 204, sd = 1.3) performed in a similar range. although we acknowledge concerns with collapsing race/ethnicity variables, we believe that these groupings described above make sense in this context given that race/ethnicity is being used as a control variable. [it should be noted, however, that since the 2005 naep assessment used in this study, the center on educational policy (2010) has reported that a gap between asian americans and caucasian students has emerged, with asian american students performing statistically significantly better than their peers in both reading and math.] farms eligibility was used as a proxy for student poverty and was examined in relationship to students’ out-of-school and in-school reading experiences. in the past, farms has been criticized as an income proxy because this service can be refused by parents and may not be available in some non-public schools. however, in the 2005 database, naep collected information from school records regarding whether students were eligible to receive farms, not whether they were actually receiving these services. in the nation’s report card, nces’s report of general findings from naep, farms is routinely used in isolation to report differences in achievement for children from various socioeconomic backgrounds (nces, 2007). we collapsed a three-category farms measure (eligible for free meals, eligible for reduced-priced meals, and not eligible for free or reduced-priced meals) into two categories (eligible for free or reduced-priced meals and not eligible for free or reducedpriced meals) because we were concerned with comparing students from households with lower incomes to all other students. more specifically, students who come from households where the family income was within 185 percent of the poverty line were classified as farms-eligible in this study (united states department of agriculture, 2011). other researchers have used proxies for students’ family income, including parental education and the presence of educational materials in the home (e.g., guthrie, schafer, & huang, 2001; lee, croninger, & smith, 1997). however, naep no longer collects information on fourth graders’ parental education because many students were unable to provide accurate responses (nces, 2011). although farms is sometimes considered a weaker measure of family income than the aforementioned variables, naep uses school records to determine whether a student’s family reported income would make him or her eligible to receive free or reduced priced meals. since some families may have unreported income, if anything, this approach may have led to a more modest analysis, as students from families with unreported income may have been erroneously included in the farmseligible category. measure of students’ out-of-school reading engagement. as noted, fourth graders completed a questionnaire that provided background information about themselves. with only 32 items, the questionnaire is limited in scope and depth. however, we posit that the information it provides offers a starting point for exploring children’s out-of-school reading engagement and in-school reading experiences that are associated with informational text comprehension. u.s. fourth graders’ informational text comprehension: indicators from naep / schugar & dreher 531 based on literacy research, we identified 11 questionnaire items that were likely to be associated with out-of-school reading engagement with informational texts (see table 1). table 1. items used for initial out-of-school reading engagement analyses item m (sd) in final analysis items with response options: (a) hardly or hardly ever, (b) once or twice a month, (c) once or twice a week, and (d) almost every day. 1. how often do you talk about things you have studied in school with someone in your family? 3.48 (1.52) yes 2. how often do you talk to your friends or family about something you have read? 2.48 (1.14) yes 3. how often do you read to learn about real things (such as facts about dinosaurs or other countries) for fun outside of school? 2.65 (1.16) yes 4. how often do you read stories or articles that you find on the internet for fun outside of school? 2.32 (1.21) yes items with response options: (a) not like me, (b) a little like me, (c) a lot like me. 5. reading is one of my favorite activities. 2.09 (0.78) yes 6. when i read books, i learn a lot. 2.32 (0.63) yes item with response options: (a) few (0-10), (b) enough to fill one shelf (11-25), (c) enough to fill one bookcase (26-100), or (d) enough to fill several bookcases (more than 100). 7. about how many books are there in your home? 2.95 (0.97) no items with response options: (a) yes or (b) no. 8. is there an encyclopedia in your home? it could be a set of books, or it could be on the computer. 1.20 (0.40) no 9. does your family get a newspaper at least four times a week? 1.49 (0.50) no 10. does your family get any magazines regularly? 1.25 (0.43) no 11. is there a computer at home that you use? 1.14 (0.35) no a principal components analysis (pca) was performed to evaluate the relationships among these items (pett, lackey, & sullivan, 2003). based on the pca, items 7 through 11 in table 1 were dropped from the study. the pca of the remaining items (1-6 in table 1) showed loadings of .594, .683, .608, .542, .629, and .668, respectively, with a cronbach alpha of .651. this resulting component, termed out-of-school reading engagement, accounted for 38.75% of the variance in students’ informational text comprehension. an examination of the initial pool of items for this measure suggests that several of the items were likely to be sensitive to socioeconomic status because they queried the presence of particular materials in the home (newspapers, magazines, computers, encyclopedias, and books). however, all these items were dropped based on the pca. in fact, there was little variation in these responses. all the dropped items, except the one international electronic journal of elementary education, 9(3), 523-552, march, 2017 532 that addressed the number of books in the home, were yes/no responses, with most participants answering “yes” for magazines (75%), encyclopedias (80%), and computers (85%). likewise, approximately half of students reported getting newspapers four or more times a week and 70% of students reported having at least 26 books in their home. thus, the remaining items focused on reading-related activity, as opposed to materials in the home. measures of students’ in-school reading experiences. although, as noted, teacher questionnaires are part of naep and also supply some information regarding classroom practices, researchers have argued that student reports are often better indicators of actual classroom practice (mullens & gaylor, 1999). therefore, we only used student reports of classroom practice. drawing on literacy research, we identified ten items from the students’ questionnaire that were likely related to in-school reading experiences with informational texts for the initial analysis (see table 2). table 2. items used for initial in-school reading experiences analyses item m (sd) component items with response options: (a) never or hardly ever, (b) once or twice a month, (c) once or twice a week, and (d) almost every day. 1. how often do you read paperbacks, soft cover books, or magazines for reading? 2.75 (1.18) materials for crosscurricular reading 2. how often do you read paperbacks, soft cover books, puzzle books, or magazines for science? 2.24 (1.15) materials for crosscurricular reading 3. how often do you read paperbacks, soft cover books, or magazines for social studies or history? 2.32 (1.17) materials for crosscurricular reading items with response options: (a) none, (b) 1, (c) 2 or 3, (d) 4 or 5, and (e) 6 or more. 4. so far this year, how many times have you written a book report? 2.76 (0.35) reading-related activities 5. so far this year, how many times have you made a presentation to the class about something you have read? 2.20 (1.22) reading-related activities 6. so far this year, how many times have you done a school project about something you have read (for example, written a play, created a poster)? 2.40 (1.16) reading-related activities 7. when you have reading assignments in school, how often does your teacher give you time to read books you have chosen yourself? 3.09 (1.08) not included in final analysis items with response options: (a) never or hardly ever, (b) a few times a year, (c) once or twice a month, and (d) at least once a week. 8. for school this year, how often do you have a class discussion about something the class has read? 2.95 (1.13) discussions about reading 9. for school this year, how often do you work in pairs or small groups to talk about something you have read?? 2.66 (1.15) discussions about reading 10. for school this year, how often do you write in a journal about something you have read for class? 2.52 (1.50) not included in final analysis u.s. fourth graders’ informational text comprehension: indicators from naep / schugar & dreher 533 based on the results of a pca, items 7 and 10 in table 2 were dropped from the final analysis. the pca of the remaining items resulted in three distinct components after being rotated orthogonally using varimax. the cronbach alpha for the overall pca for in-school reading experiences was .641 and explained a total of 57.03% of the variance in informational text comprehension. the first component, termed materials for crosscurricular reading, explained 28.81% of the total variance and included the three items about the frequency with which students read materials other than textbooks for reading, science, and social studies/history. the loadings for this component were .618, .789, and .776, respectively, with a cronbach alpha of .593. the second component, labeled reading-related activities, accounted for an additional 15.53% of the total variance and consisted of the three items regarding the frequency with which students engaged in book-report writing, projects, and presentations about what they had read. the loadings for this component were .715, .715, and .741, respectively, with a cronbach alpha of .570. the third component, discussion about reading, accounted for an additional 12.69% of the total variance and was comprised of two items related to how often students had a class discussion about reading and how often students talked in small groups about what they had read. the loadings for this component were .786 and .738, respectively, with a cronbach alpha of .396. measure of students’ comprehension of informational texts. student achievement was based on the reading for information subscale questions on the 2005 naep assessment. naep defines the content of the reading for information subscale as follows: “[the reading of texts in which] readers gain information to understand the world by reading materials such as magazines, newspapers, textbooks, essays, and speeches” (nces, 2010b, n.p.). our analysis of retired fourth-grade passages from naep revealed that these texts vary in structure, ranging from mainly expository text to hybrids of narrative and expository texts. measures of school characteristics. to measure school demographic information (e.g., proportion of students’ eligible for farms, proportion of students with minority status, type of school) and school-wide reading practices (average out-of-school reading engagement and in-school reading practices), data were aggregated from information at the student level to the school level in spss, and then analyzed using hierarchical linear modeling, as explained in the next section. analysis because of the clustered nature of the students within schools, we used hierarchical linear modeling (hlm) to analyze the data (bryk, raudenbush, & congdon, 1996). hlm allowed us to build a model that represented the nested nature of the naep data. in our model, we had two levels: (1) level 1, student level information and (2) level 2, school level data. at level 1, the model illustrated the associations between individual variables and individual achievement, while at level 2, the model examined the associations of variables aggregated to the school level and overall achievement within the school. in this sense, hlm allowed us to account for variance attributed to differences at both the individual and school levels. to justify the use of hlm, an unconditional model (a model without within or between factors used to estimate partitioned variance) is first calculated to determine the intraclass correlation (icc) (i.e., the proportion of variance that occurs between schools) (see research question 1). an icc of .10 or higher warrants the use of hlm (bryk & raudenbush, 1992). the icc for this study was .234 meaning that 23.4% of the variance in informational text comprehension in this study occurred between schools. international electronic journal of elementary education, 9(3), 523-552, march, 2017 534 it is important to note that although over 165,000 fourth graders participated in the 2005 naep reading assessment, some data were lost at level 2 (school) in the hlm analysis. no data were missing for race, school type, gender, or any of the plausible values for student informational text comprehension. but, at the school level, farms information was not collected at 680, or 7.8%, of the schools. in addition, for approximately one percent of the schools (190 schools) data were missing for the out-of-school reading and in-school reading factors. however, an analysis of the characteristics of students with missing data for out-of-school reading, in-school reading, and farms eligibility showed no statistically significant differences from the original sample of 8620 schools in terms of achievement, race, school type, and gender. results overall achievement data on the reading for information subscale of the 2005 naep assessment indicate that fourth graders’ informational text comprehension (m=216, sd = 38) is statistically significantly lower than both overall comprehension (m=219, sd = 36) and narrative text comprehension (m=222, sd = 37), according to public-use information available through naep’s data explorer. thus, fourth graders in the united states have more difficulty comprehending informational texts than narrative texts. but do farmseligible students differ from their peers on comprehension of informational texts (research question (rq) 1)? using the naep data explorer, we determined that there is a 28-point gap between the informational text comprehension scores of students eligible and not eligible for farms. students who were eligible for farms (m= 199, sd = 36) scored .80 of a standard deviation below children who were not eligible for farms (m = 227, sd = 34), indicating a sizeable disadvantage for low-income children in terms of their abilities to comprehend informational texts. [as noted earlier, there is a 26-point difference in literary comprehension between farms-eligible (m = 206, sd = 36) and nonfarms-eligible students (m = 232, sd = 33).] in this study, our results identify factors that are associated with informational reading achievement on the naep assessment, with a particular focus on students of differing socio-economic status. we first report the associations between students’ informational reading achievement and out-of-school reading engagement and in-school reading experiences (rq 2). we then report the degree to which students in some schools are able to comprehend informational texts compared to students in other schools (rq 3) and the characteristics of schools that are associated with students’ informational text comprehension, including demographics and school-wide reading practices (rq 4). our original research design called for the examination of whether or not schools’ farms composition is related to students’ informational text comprehension as part of rq 4; however, since naep uses background information such as income, gender, and race to determine the plausible values for unanswered items, we found that these background items became confounded with achievement. in particular, students in similar schools with similar background attributes are more likely to receive similar plausible values, influencing the variability of student responses. therefore, our results for rq 4 address out-of-school and in-school factors related to differences in school-wide informational text comprehension, but they do not address how the farms composition of the school might be associated with school-wide informational text comprehension. students’ reported out-of-school reading engagement as part of addressing rq 2, we analyzed the full within-school model for out-of-school reading engagement. this model compares students within schools (and in this case included the variables of out-of-school reading engagement, farms eligibility, race/ethnicity, and gender), accounted for 10.2% of the group variance (σ2) beyond that u.s. fourth graders’ informational text comprehension: indicators from naep / schugar & dreher 535 explained in the unconditional model. in this study, a standard deviation (sd) increase in the frequency students’ engaged in out-of-school reading activities (e.g., discussions with family and friends about books, reading to learn, reading on the internet) was associated with a 6.29 point coefficient in their overall informational text comprehension score (see table 3). table 3. results for the within-school model (out-of-school reading) ab fixed effect coefficient se t-ratio overall mean 4th grade informational text achievement γ00 214.57 .22 973.37*** mean out-of-school (oos) reading achievement slope γ10 6.29 .18 35.23*** mean minority achievement slope γ20 -13.30 .45 -29.72*** mean farms-eligible achievement slope γ30 -13.10 .45 -28.84*** mean female – achievement slope γ40 2.11 .26 8.24*** mean farms*oos reading – achievement slope γ50 -2.70 .24 11.13*** random effect variance component df chi-square 4th grade informational text achievement level-1 effect rij 109.22 8611 25029.61*** a note: analysis done with adjusted school weight, schoolweight. b*,**, and *** significant at .05, .01, and .001 levels, respectively, using two-tailed t-tests. however, the association between out-of-school reading engagement and informational text comprehension varied with students’ farms status. the coefficient for farmseligible students was 2.11 points lower than the coefficient for students not eligible for farms (p < .001). students’ reported in-school reading experiences to address rq 2, we also examined the contributions of three components of in-school reading: (1) materials for cross-curricular reading, (2) reading-related activities, and (3) discussions about reading. this within-school model – which included the variables of materials for cross-curricular reading, reading-related activities, discussions about reading, farms eligibility, race/ethnicity, and gender -accounted for 11.1% of the group variance (σ2) beyond that explained in the unconditional model (see table 4). table 4. results for the within-school model (in-school reading) fixed effect coefficient se t-ratio overall mean 4th-grade informational text achievement γ00 214.86 .22 984.55*** mean minority achievement slope γ10 -12.83 .45 -28.54*** mean farms-eligible achievement slope γ20 -12.86 .46 -28.02*** mean female – achievement slope γ30 3.37 .26 -12.97*** mean materials for reading achievement slope γ40 1.10 .18 6.05*** mean reading-related activities achievement slope γ50 -3.67 .20 -18.05*** mean discussion of reading achievement slope γ60 3.29 .18 18.43*** mean farms*materials – achievement slope γ70 -.025 .23 -1.13 mean farms*discussion – achievement slope γ80 0.58 .26 2.23* mean farms*reading-related activities – achievement slope γ90 -1.82 .25 -7.29*** international electronic journal of elementary education, 9(3), 523-552, march, 2017 536 table 4 (cont.). results for the within-school model (in-school reading) random effect variance component df chi-square 4th grade informational text achievement level-1 effect rij 108.00 8609 24011.50*** a note: analysis done with adjusted school weight, schoolweight. b*,**, and *** significant at .05, .01, and .001 levels, respectively, using two-tailed t-tests. every sd increase in the frequency with which students reported using materials for cross-curricular reading was associated with a coefficient of 1.10 points on their informational comprehension score (p < .001). for using materials across the curriculum, there was no statistically different association for farms-eligible students compared to non-farms-eligible students. every sd increase in the frequency with which students reported engaging in readingrelated activities (e.g., book reports, presentations, and projects) was associated with a coefficient of -3.67 points (p < .001) on their score. the strength of this negative association between reading-related activities and text comprehension varied with students’ farms status. the coefficient for farms-eligible students was 1.82 points lower than the coefficient for non-farms-eligible students (p < .001). finally, a sd increase in the frequency with which students reported discussions about reading (e.g., talking about what they read as a class and in small groups), was associated with a coefficient of 3.29 points on their text comprehension score (p < .001). the strength of this positive association between student participation in whole-class or small-group discussions and achievement also varied according to students’ farms status. the coefficient for farms-eligible students was .58 points higher than the coefficient for nonfarms-eligible students (p < .05). school characteristics associated with text comprehension we were interested in examining the degree to which students in some schools were able to comprehend informational texts compared to students in other schools (rq 3). we found that school characteristics were often important in predicting students’ text comprehension. because of the way that naep calculates plausible values for students by comparing their performance to students from similar backgrounds, there was not much variance in how farms status was associated with students’ out-of-school reading engagement and in-school reading experiences across schools. however, a sd increase in the proportion of farms eligible students in a school was associated with a coefficient of 10 (see tables 5 and 6). we used the between-school model for out-of-school reading engagement, which compares schools to other schools (and in this case included the variables of out-of-school reading engagement, proportion of farms-eligible students, proportion of minority students, and public/private status) to determine which characteristics of schools were associated with informational text comprehension (rq 4). this model accounted for 66.8% of the group variance (τ00) beyond that explained in the within-school model. in this study, a sd increase in the school average of students’ reported frequency of out-ofschool reading engagement had an 8.59 point positive association with their informational text comprehension (see table 5). u.s. fourth graders’ informational text comprehension: indicators from naep / schugar & dreher 537 table 5. results for the between-school model (out-of-school reading) ab intercept γ00 4th grade informational text achievement γ00 proportion minority γ01 proportion farms eligible γ02 private school γ03 mean out-of-school reading γ04 out-of-school reading γ10 minority γ20 farms eligible γ30 female γ40 farms*out-of-school reading interaction γ50 214.56*** -6.16*** -10.01*** 2.61* 8.59*** 6.29*** -13.30*** -13.10*** 2.11*** -2.70*** a note: analysis done with adjusted school weight, schoolweight. b *,**, and *** significant at .05, .01, and .001 levels, respectively, using two-tailed t-tests. in other words, students in schools with other children who engage in various out-ofschool reading activities have a statistically significant positive association with achievement above and beyond that of just engaging in out-of-school reading on their own. school-wide, in-school reading’s association with average text comprehension to determine how school-wide, in-school reading practices were associated with students’ average text comprehension (rq 4), we examined the full between-school model. the full between-school model for in-school reading experiences included variables for the three aspects of in-school reading experiences: (1) materials for cross-curricular reading, (2) reading-related activities, and (3) discussion about reading, as well as variables for proportion of farms-eligible students, proportion of minority students, and public/private status. this between-school model accounted for 66.6% of the group variance (τ00) beyond that explained in the unconditional model (see table 6). a sd increase in the school average of students' reported frequency using materials for cross-curricular reading was positively associated with informational text comprehension, with a coefficient of 3.46 points on students’ comprehension scores (p < .001).a sd increase in the school average of students' reported frequency of readingrelated activities was negatively associated with comprehension, with a coefficient of -.84 on comprehension scores (p < .01). finally, a sd increase in the school average of students' reported frequency in discussion about reading was positively associated with comprehension, with a coefficient of 5.49 points on students’ comprehension scores (p < .001). there were also statistically significant contextual effects for students who were in schools where students reported using materials for cross-curricular reading (2.36 points, p < .001) and having discussions about reading (2.20 points, p < .001). this means that the coefficient associated with being in a school where students report these activities resulted in a stronger than expected association with achievement scores (based on how these variables were related to student achievement at the individual level). international electronic journal of elementary education, 9(3), 523-552, march, 2017 538 table 6. results for the between-school model (in-school reading) ab intercept γ00 4th grade informational text achievement γ00 proportion minority γ01 proportion farms eligible γ02 mean materials for reading γ03 mean reading-related activities γ04 mean discussion of readings γ05 private school γ06 minority γ10 farms eligible γ20 female γ30 materials for reading γ40 reading-related activities γ50 discussion of readings γ60 farms*materials interaction γ70 farms*discussion interaction γ80 farms*reading-related activities interaction γ90 214.86*** -5.67*** -9.69*** 3.46*** -0.84** 5.49*** 3.20** -12.83*** -12.86*** 3.37*** 1.10*** -3.67*** 3.29*** -0.025 0.58* -1.82*** a note: analysis used adjusted school weight, schoolweight. b *,**, and *** significant at .05, .01, and .001 levels, respectively, using two-tailed t-tests. discussion our results indicate that certain reading experiences are associated with informational text comprehension, and that these associations are not always the same for farmseligible students as for other students. specifically, out-of-school reading engagement was positively associated with the informational reading comprehension for all students, but less so for farms-eligible students. the in-school reading measures of materials for cross-curricular reading and discussion about reading were also positively associated with informational text comprehension, with no difference for farms-eligible and other students on the former but a higher positive association for farms-eligible students on the latter. in contrast, the in-school reading measure reading-related activities was negatively associated with comprehension, with a stronger negative association for farms-eligible students. in addition, being in a school in which other students engaged frequently in certain out-of-school and in-school reading practices (out-of-school reading engagement, material for cross-curricular reading, and discussions about reading) was associated with higher than expected achievement scores. out-of-school reading engagement the difference that we identified in the strength of association between out-of-school reading engagement and informational text comprehension for students from different socioeconomic backgrounds makes sense in relation to the literature. children who enjoy reading and who are motivated to read from multiple genres of texts may be more prepared to handle the cognitive demands of reading informational texts. specifically, expository texts require readers to interact with texts in a way that is different than the way in which most children read narrative texts. in addition to having to navigate the unique text features and structures found in exposition (dreher & kletzien, 2015; pappas, 2006; purcell-gates, duke, & martineau, 2007), readers often read these texts with the purpose of gaining information (rosenblatt, 1978), a stance quite different than that promoted by most teachers as students learn to read. with the abundance of informational text found on the internet, and the multimodal nature of many internet texts, students may be required to both process and create various types of text (e.g., print, video, illustrations, and audio files) in order to make meaning. thus, the evolving nature of informational reading, and its importance in new technological “spaces” with undefined u.s. fourth graders’ informational text comprehension: indicators from naep / schugar & dreher 539 borders, makes preparing students to comprehend informational texts both a pressing and international concern (coiro, knobel, lankshear, & leu, 2008). another reasonable explanation for the association between students’ out-of-school reading and informational text comprehension is that when children talk about (and potentially summarize) what they read with others, they may develop a better understanding of what they have read. such talk may occur at home, but may also take place during extracurricular activities at school, community libraries, churches, and clubs (e.g., green-powell, hilton, & joseph, 2011; kellett, 2009; ly, 2010). participation in such activities has been linked to positive outcomes including achievement (mahoney, larson, eccles, & lord, 2005). however, the poorest children are the least likely to be able to take advantage of such activities (dearing et al., 2009). in their research examining the home and classroom environments of low-income, mexican americans, moll and his colleagues (1992) found that families and communities had “historically accumulated and culturally developed bodies of knowledge and skills essential for household or individual functioning and well-being” that were potentially beneficial to students’ overall academic achievement (p. 134). similar arguments could be made for other low-income groups (e.g., dudley-marling & lucas, 2009). if schools were to bring these “funds of knowledge” into classrooms, it might help bridge the gap between the out-of-school experiences of students from differing backgrounds. in addition, when children talk to their peers about what they read, it can create a sense of community, enhance their understanding of the text, and introduce ideas about the text that children may not have thought of themselves. also, children often read books recommended by peers because doing so can be a way to become socially accepted (fleener, morrison, linek, & rasinski, 1997; moss & mcdonald, 2004; timion, 1992). the sharing of books with peers can lead to more active and frequent reading. being in a school with other children who reported frequently engaging in out-of-school activities was also related to children’s informational text comprehension. students in schools where other children experienced out-of-school reading engagement may have been surrounded by students who were motivated to read, had a lot of practice reading, and were more prepared to talk about what they had read. therefore, even if students are not frequently engaged in out-of-school reading themselves, the motivation and practice of their peers may influence their own school practices. in schools where students do not value reading activities outside of school, students may be less likely to be in school with peers who would find it socially acceptable or important to read in school. although there is a positive association between out-of-school reading engagement and achievement for all students, the association was less strong for farms-eligible students. even though low-income children reported that they engaged in out-of-school readingrelated activities, the naep questionnaire asked only how often they participated in various activities and gathered no information regarding the quality of these experiences. prior research supports the notion that, on average, low-income children have out-ofschool literacy-related experiences, including access to literacy-related resources in the home and community (neuman & celano, 2001, 2012), that are less well matched to school achievement demands than the experiences of their peers. similarly, although lowincome children may have reported frequently discussing books with their families and friends, studies suggest differences between low-income children and their peers, on average, in quality and/or quantity of the kind of parent-child conversation that support the type of vocabulary development expected in school (lareau, 1989;weizman & snow, 2001). such findings may partially explain why out-of-school reading activities were not as highly associated with informational text achievement for low-income students as for wealthier children in our study. international electronic journal of elementary education, 9(3), 523-552, march, 2017 540 as with family discussions, the quality of what children are reading for information is unclear. for example, a child in a middle-class home might have several highly engaging informational texts to read. however, some low-income children may not have quality resources available (neuman, 1999) and may be reading older, out-of-date sources for information, possibly relating to our finding of a weaker association with achievement than for fourth graders’ overall. in-school reading experiences the experiences that children have with reading in schools are important for their understanding of informational texts. schools can provide opportunities for students to read from different genres, respond to texts in multiple ways, and to discuss texts they have read. in this study we explored the relationships between informational text comprehension and reading across the curriculum, reading-related activities, and classroom discussions. reading across the curriculum. reading trade books and magazines in science, social studies, history, and reading is positively related students’ informational text comprehension. according to moss (2003), trade books and magazines are likely to be more accessible informational texts for fourth graders than the typical textbook, because they are often visually engaging, focused on a specific topic, and written at a level that students can understand. textbooks, on the other hand, often give general information about multiple topics, not delving deeply into any particular subject (chambliss & calfee, 1998). moreover, moss (2003) noted that textbooks are often written by experts in various fields (e.g., geology) while the majority of trade books are written by authors who write for children. in addition to potentially substituting for or supplementing textbook instruction, reading trade books and magazines across the curriculum is likely to expose children to informational texts. by reading trade books and magazines, children may gain familiarity with the structures, purposes, and features of informational texts. in using these texts, teachers may be engaging students in activities and strategy instruction that have been shown to enhance student comprehension, such as pulling information from multiple texts, answering questions, making inferences, making connections, and writing about what they have learned (harvey & goudvis, 2000; nichd, 2000; pressley & afflerbach, 1995). in order to do well on the naep assessment, students need to use these strategies with informational texts (nces, 2007). we found a contextual effect associated with being in a school in which students reported reading across the curriculum. this means that being in a school in which students report these activities is associated with higher than expected achievement, based on how reading across the curriculum was related to student achievement at the individual level. there are many plausible explanations for this increase, including having a school culture that values reading to learn, providing exposure and practice with various genres, and integrating the curriculum across subject areas. although some students prefer storybooks and some prefer informational books, many students have an equal affinity for stories and informational books (chapman, filipenko, mctavish, & shapiro, 2007; kletzien, 1999). such evidence suggests that students in schools with positive learning cultures might be more inclined to read informational texts. as noted earlier, some schools mandate scripted instruction and/or adherence to a basal reading program, with this situation more common in schools serving low-income neighborhoods. it is possible that such restrictions limit schools in preparing students to handle the wide variety of genres and content that students will encounter as they are expected to read across the content areas. although basal reading programs in general u.s. fourth graders’ informational text comprehension: indicators from naep / schugar & dreher 541 have become more inclusive of informational texts, a recent study of basal readers found that the distribution still does not match the proportion informational text in the 2009 naep framework (moss, 2008). moreover, strict adherence to a program and the use of a scripted curriculum have been criticized as not serving the needs of children. peasealvarez and samway (2008) noted that these practices result in the prevalence of whole class instruction, lack of time for independent reading (during students might broaden their exposure to diverse texts), and an emphasis on standardized tests. portes and salas (2009) argued that students from low income families typically receive narrow, skillbased instruction that restricts their literacy development, and that a broader view of literacy with rich instructional contexts would better serve the needs of this population. in schools characterized by this type of instruction, students may have fewer opportunities for wide reading or discussions that might promote facility with informational text. reading-related activities. reading-related activities such as writing book reports, making presentations, and doing projects were negatively associated with informational text achievement. practitioner literature reports that in order for these activities to extend thinking, teachers need to scaffold these activities carefully as part of their instruction (atwell, 1998). students likely need clear directions as to how to complete these activities, support when doing them, and clear purposes as to why they are doing them (many, fyfe, lewis, & mitchell, 1996). it is possible that teachers assign these activities as busywork or because they are unfamiliar with other alternatives for how students can respond to texts. thus, they assign book reports, presentations, and projects that may be inauthentic and may not challenge students within their zone of proximal development (vygotsky, 1978). under such conditions, these activities may involve little reading. yet guthrie and mcrae (2011) found that students’ dedication or behavioral engagement, defined as “effort, time, and persistence in reading” (p. 119), is positively correlated with reading achievement even when factors such as socioeconomic status and gender are controlled. as a result, guthrie and mcrae argued that it is important that instructional practices support behavioral engagement in reading as opposed to practices that involve little actual reading. in addition, any reading that may occur as part of book reports and presentations is likely to focus on narrative texts since stories predominate in many elementary classrooms (e.g., jeong et al., 2010; ness, 2011). narrative texts do not provide children with experience with text structures and other features that may facilitate informational text comprehension (pappas, 2006). moreover, if students are working on book reports, presentations, and projects during class time, they may not be sharing what they have read with each other or interacting with the teacher. but, as explained in the next section, the opportunity to discuss what is read with other students can improve comprehension. similarly, research has shown that students benefit from quality interactions with their teachers (coleman et al., 1966; cadima, leal, & burchinal, 2010). likewise, teachers may be “wasting” valuable instructional time if they assign these activities with little direction. instead, students may benefit from response activities that encourage thinking about texts but are less time intensive than reports, presentations, and projects. low-income students might be particularly vulnerable to the negative implications of engaging in these activities, as they are more likely to experience inferior schooling conditions than their middle-class peers (coleman et al., 1966; darling-hammond, 1995). discussing books as a class or small group. research supports the notion that when students have an opportunity to talk about what they read, it may lead to better understanding of these texts (johnson & johnson, 1989/1990). in a meta-analysis of research on discussion about text, murphy and her colleagues (2009) found that classroom discussions which followed an efferent approach were particularly effective at international electronic journal of elementary education, 9(3), 523-552, march, 2017 542 increasing students’ literal and inferential comprehension. these results support the findings here that students’ engagement in discussions with teachers and classmates was positively associated with informational text achievement. students who talked with each other likely benefited from ideas that they might not have developed independently, using other students’ ideas to build, clarify, or enrich their own ideas. when participating in class or small group discussions, students may be forced to juggle several contradicting ideas at the same time. several students may offer alternative explanations or different answers that require discussion participants to evaluate how accurate or relevant they are to the discussion. likewise, when participating in a discussion, students may be encouraged to connect ideas to their own knowledge, a skill that can promote comprehension and retention (anderson & pearson, 1984). at their most basic level, discussions hold students accountable for completing assigned readings because in order to be an active participant in a discussion, students must have read the text being discussed. we also found a contextual effect for attending a school in which other students cited frequent discussion of books. this situation was positively associated with children’s informational text achievement. it is possible that schools in which students reported high frequency of discussion have a more positive and motivating school environment. cori classrooms (see guthrie, mcrae, & klauda, 2007) are a prime example of how discussion may be associated with a motivating social context for students. in cori classrooms, students explore questions of interest to them, have shared learning experiences, talk with one another, and build strategies for answering and communicating the answers to their own questions. instead of a lecture format in which the teacher tells students what is acceptable or important to think, students talk with one another and engage in their own thinking about the concept of interest. although the teacher introduces strategies as the students need them, students have a role in determining their own learning needs and the direction they take in their learning. this, in turn, can be quite motivating for students. similarly, the types of participation structures that are used for classroom discussion may make a difference in how effective classroom discussions are in promoting comprehension of informational texts, as evidence suggests that peer-led discussion can be of higher quality than those that are teacher-led (almasi, 1995). recommendations for future research given the frequency with which students are expected to engage with informational texts in their academic endeavors, it is important that educators understand how students can be prepared for the demands of informational reading. in this study, we identified factors out-of-school reading engagement, cross-curricular reading, discussions about texts, and reading related activities -associated with informational reading and hence deserving of more research to determine how and why they are associated. however, three of the four factors seem particularly worthy of future exploration because their association with informational reading performance differed in degree when farms-eligible students were compared to other students: out-of-school reading engagement, reading-related activities, and discussions about reading. first, although there is a substantial body of literature relating students’ out-of-school reading engagement to reading abilities and socioeconomic status, much of this literature focuses on reading materials and parent/child interactions, while very few studies specifically relate out-of-school reading engagement to informational reading. based on the extant research, we believe that reading literary fiction does not necessarily prepare students for the unique requirements of comprehending informational texts (duke & roberts, 2010; moje, stockdill, kim, & kim, 2011; rand reading study group, 2002); thus, u.s. fourth graders’ informational text comprehension: indicators from naep / schugar & dreher 543 more research regarding attributes of out-of-school reading activities that may affect informational reading is warranted. for instance, future studies might further explore the attributes of community programs such as homework clubs and library services that may be associated with informational reading comprehension. in addition, because the internet provides a major source of out-of-school opportunity to engage with informational text, continued research is needed about how readers navigate and comprehend these texts and how in-school reading instruction might be informed by such informational reading. second, perhaps the most striking finding from our study was the negative association between students’ informational text comprehension and their participation in readingrelated activities such as book reports, presentations, and projects, with the negative association stronger for farms-eligible students than others. although we have discussed possible reasons for the negative association between frequent engagement in these activities and informational text comprehension, more research is needed to understand the quality, content, and purpose of such activities in today’s classrooms. third, the higher than expected association with informational text comprehension for farms-eligible students who reported frequent whole-class and small group discussions warrants further examination about the attributes of discussions that are particularly beneficial for this demographic. in particular, further research might examine how classroom talk introduces academic vocabulary to prepare students from low socioeconomic backgrounds to comprehend informational texts. research on classroom talk seems especially important given that many schools serving low-income children have adopted test-driven, highly-constrained reading instruction that does not provide opportunities for wide reading or promote classroom discussions. in short, although the results from this study cannot explain why the associations that we found exist, they do suggest areas for additional exploration in the quest to close the achievement gap between students of differing socioeconomic backgrounds. limitations a limitation of this study is that naep does not collect prior achievement data for their participants. therefore, it is difficult to isolate whether student achievement is directly related to what students reported happened in their homes and classrooms during fourth grade. however, this lack of data may be less important for this study because it explores fourth graders’ comprehension of informational texts, and research has shown that many students have relatively few experiences with informational texts, particularly exposition, before this juncture in their education. in addition, this study is limited by its use of farms eligibility as a proxy for socioeconomic status. yet, although farms is sometimes considered a weaker measure of family income than other variables (e.g., parental education), naep statisticians consider their farms variable strong enough to use as the sole measure of family income in their nation’s report card results (nces, 2007). furthermore, naep eases some concerns about using farms eligibility by collecting family income information from school records rather than student reports. as with most large-scale assessments, the data are limited in both scope and depth. in our case, we were particularly aware of the fact that in a 32-question survey of background information, there is only room to address a sampling of variables related to out-of-school and in-school reading achievement. we agree with the recent recommendations that call for the reexamination of the scope and content of the naep background questions (smith et al., 2012). in particular, we argue that a socioeconomic status measure that encompasses not only farms but also more specific information about student literacy experiences (e.g., where out-of-school discussions take place; what types of text students read for enjoyment) would make the background data more useful for analyses. in international electronic journal of elementary education, 9(3), 523-552, march, 2017 544 addition, although it is likely that naep examines whether there are differences in how students from various demographics report information on questionnaires as part of their guidelines for developing questions (see http://nces.ed.gov/statprog/2002/stdintro4. asp), it is possible that some subgroups of students may respond differently to items than other groups. for example, the naep questionnaire has a heavy focus on frequency questions, but it is possible that farms-eligible students and non-farms-eligible students report their time use differently for social and/or cultural reasons. finally, the results of this study are not necessarily generalizable to the us population of fourth graders because some data were lost when hlm could not accommodate missing data at level-2. when all data are used, the 2005 naep reading sample represents the population of us fourth graders; however, this study was based on a sample that may have been marginally different than the overall population of us fourth graders. conclusion there has been a longstanding concern for closing achievement gaps in the us (coleman, et al., 1966; usde, 2002), as well as in other countries (e.g., smith, 2011; wilkinson, 1998). in order to make headway in closing these gaps, educators likely need a better understanding of the factors that are associated with differences in achievement. the achievement difference between low-income students and their peers has been a particular concern. recent evidence suggests that in the us the gap between rich and poor students is increasing (reardon, 2011) and that in many states poor children make up the majority in public schools (southern education foundation, 2013). biyearly in the us, the nation’s report card documents the gap in reading achievement between children who are eligible for farms and those who are not eligible (nces, 2007). however, these reports have not addressed potential reasons behind these differences in achievement. this study explored the associations between low-income fourth graders’ out-of-school and in-school reading and their comprehension of informational texts. out-of-school reading engagement, materials for cross-curricular reading, discussions about reading, and reading-related activities were identified as factors associated with fourth graders’ informational text achievement, and some of these factors differed in the strength of the association for farms-eligible and students and fourth graders overall. thus, the results from this study may enhance discussion of and research on factors that may contribute to understanding why the income-achievement gap exists and what to do about it. • • • references adams, m. j. (1990). learning to read: thinking and learning about print. cambridge, ma: mit press. akyol, g., sungur, s., & tekkaya, c. (2010). the contribution of cognitive and metacognitive strategy use to students’ science achievement. educational research and evaluation, 16, 1-21. almasi, j. f. (1995). the nature of fourth graders’ sociocognitive conflicts in peer-led and teacherled discussions of literature. reading research quarterly, 30, 314-351. anderson, r. c., & pearson, p. d. (1984). a schematic-theoretic view of basic processes in reading comprehension. in r. barr, m. l. kamil, & p. b. mosenthal (eds.), handbook of reading research (vol. 1, pp. 255-292). mahwah, nj: lawrence erlbaum associates. u.s. fourth graders’ informational text comprehension: indicators from naep / schugar & dreher 545 applebee, a. n., langer, j. a., & mullis, i. v. s. (1988). who reads best? factors related to reading achievement in grades 3, 7, and 11. princeton, nj: national assessment of educational progress. atwell, n. (1998). in the middle: new understandings about writing, reading, and learning. portsmouth, nh: heinemann. atwell, p., & battle, j. (1999). home computers and school performance. the information society, 15, 1–10. beck, i. l., & mckeown, m. g. (2001). inviting students into the pursuit of meaning. educational psychology review, 13, 225-241. bell, m. (2009/2010). major new literacy initiative proposed in us congress. reading today, 27(3), 1, 4. bryk, a. s., & raudenbush, s. w. (1992). hierarchical linear models: applications and data analysis methods. newbury park, ca: sage publications, inc. bryk, a. s., raudenbush, s. w., & congdon, r. t. (1996). hlm. hierarchical linear and nonlinear modeling with the hlm/2l and hlm/3l programs. chicago: scientific software international. bus, a. g. (2003). socio-emotional requisites for learning to read. in a. van kleeck, s. a. stahl & e. b. bauer (eds.), on reading storybooks to children: parents and teachers (pp. 3-15). mahwah, nj: erlbaum. cadima, j., leal, t., & burchinal, m. (2010). the quality of teacher-student interactions: associations with first graders’ academic and behavioral outcomes. journal of school psychology, 48, 457-482. calkins, l., montgomery, k., santman, d., & falk, b. (1998). a teacher's guide to standardized tests: knowledge is power. portsmouth, nh: heinemann. cazden, c. b. (2001). classroom discourse: the language of teaching and learning (2nd ed.). portsmouth, nh: heinemann. center on education policy. (2007). choices, changes, and challenges: curriculum and instruction in the nclb era. washington, dc: center on education policy. center on education policy (2010). policy implications for trends for asian american students. washington, dc: center on education policy. chall, j. s. (1983). stages of reading development. new york: mcgraw-hill. chall, j. s. (1996). stages of reading development (2nd ed.). fort worth, tx: harcourt brace college publishers. chall, j. s., jacobs, v. a., & baldwin, l. e. (1990). the reading crisis: why poor children fall behind. cambridge, ma: harvard university press. chambliss, m., & calfee, r. c. (1998). textbooks for learning: nurturing children's minds. maiden, ma: blackwell publishers. chapman, m., filipenko, m., mctavish, m., & shapiro, j. (2007). first graders’ preferences for narrative and/or information books and perceptions of other boys’ and girls’ book preferences. canadian journal of education, 30, 531‐553. chudowsky, n., chudowsky, v., & kober, n. (2009). state test score trends through 2007-08, part 3: are all achievement gaps closing and is achievement rising for all? washington, dc: center on education policy. cohen, e. (1994). restructuring the classroom: conditions for productive small groups. review of educational research, 64, 1-35. coiro, j. (2011). talking about reading as thinking: modeling the hidden complexities of online reading comprehension. theory into practice, 50, 107-115. international electronic journal of elementary education, 9(3), 523-552, march, 2017 546 coiro, j., & dobler, e. (2007). exploring the online reading comprehension strategies used by sixthgrade skilled readers to search for and locate information on the internet. reading research quarterly, 42, 214-257. coiro, j., knobel, m., lankshear, c., & leu, d. (2008). handbook of research on new literacies. routledge: new york. coleman, j. s., campbell, e. q., hobson, c. j., mcpartland, j., mood, a. m., weinfeld, f. d., et al. (1966). equality of educational opportunity. washington, d.c.: department of health, education, and welfare, u.s. government printing office. darling-hammond, l. (1995). what happens to a dream deferred?: the continuing quest for equal educational opportunity. in j. a. banks & c. a. m. banks (eds.), handbook of research on multicultural education (pp. 607-630). san francisco, ca: jossey-bass. dearing, e., wimer, c., simpkins, s. d., lund, t., bouffard, s. m., caronongan, p., kreider, h., & weiss, h. (2009). do neighborhood and home contexts help explain why low-income children miss opportunities to participate in activities outside of school? developmental psychology, 45, 1545– 1562. delpit, l. (1998). the silenced dialogue: power and pedagogy in educating other people’s children. harvard educational review, 58, 280-298. dreher, m. j. (2000). fostering reading for learning. in l. baker, m. j. dreher, & j. t. guthrie (eds.), engaging young readers: promoting achievement and motivation (pp. 68-93). new york: guilford press. dreher, m. j. & kletzien, s. b. (2015). teaching informational text in k-3 classrooms: best practices to help children read, write, and learn from nonfiction. new york: guilford press. dudley-marling, c., & lucas, k. (2009). pathologizing the language and culture of poor children. language arts, 86, 362-370. duke, n. k. (2000). 3.6 minutes per day: the scarcity of informational texts in first grade. reading research quarterly, 35, 202-224. duke, n. k., & roberts, k. m. (2010). the genre-specific nature of reading comprehension. in d. wyse, r. andrews, & j. hoffman (eds.), the routledge international handbook of english, language and literacy teaching (pp. 74–86). london: routledge. durkin, d. (1978/1979). what classroom observations reveal about reading comprehension instruction. reading research quarterly, 14, 481-533. fang, z., & wei, y. (2010). improving middle school students’ science literacy through reading infusion. journal of educational research, 103, 262-273. feitelson, d., & goldstein, z. (1986). patterns of book ownership and reading to young children in israeli school-oriented and non-school-oriented families. the reading teacher, 39, 924-930. fleener, c. e., morrison, s. l., linek, w. m., & rasinski, t. v. (1997). recreational reading choices: how do children select books? in w. m. linek & e. g. sturtevant (eds.), exploring literacy (pp. 75-84). commerce, tx: college reading association. flood, j., & lapp, d. (1986). types of texts: the match between what students read in basals and what they encounter in tests. reading research quarterly, 21, 284-297. goatley, v. j., brock, c. h., & raphael, t. e. (1995). diverse learners participating in regular education "book clubs.” reading research quarterly, 30, 352-380. gottfried, a. e., fleming, j. s., & gottfried, a. w. (1998). role of cognitively stimulating home environment in children’s academic intrinsic motivation: a longitudinal study. child development, 69, 1448-1460. u.s. fourth graders’ informational text comprehension: indicators from naep / schugar & dreher 547 green-powell, p. a., hilton, a. a., & joseph, c. l. (2011). creating collaborative partnership with local churches to improve academic performance of k-12 public schools. us-china education review, 8, 64-69. guthrie, j. t., anderson, e., alao, s. & rinehart, j. (1999). influences of concept-oriented reading instruction on strategy use and conceptual learning from text. the elementary school journal, 99, 343-355. guthrie, j. t., & mccann, a. d. (1996) idea circles: peer collaborations for conceptual learning. in l. b. gambrell & j. f. almasi (eds.). lively discussions! fostering engaged reading (pp. 87-105). newark, de: international reading association. guthrie, j. t., & mcrae, a. (2011). reading engagement among african american and european american students. in s. j. samuels & a. e. farstrup (eds.), what research has to say about reading instruction (4th ed., pp. 115-142). newark, de: international reading association. guthrie, j. t., mcrae, a., coddington, c. s., klauda, s. l., wigfield, a., & barbosa, p. (2009). impacts of comprehensive reading instruction on diverse outcomes of lowand high-achieving readers. journal of learning disabilities, 42, 195-214. guthrie, j. t., mcrae, a., & klauda, s. l. (2007). contributions of concept-oriented reading instruction to knowledge about interventions for motivations in reading. educational psychologist, 42, 237-250. guthrie, j. t., schafer, w. d., & huang, c. (2001). benefits of opportunity to read and balanced instruction on the naep. journal of educational research, 94, 145-162. guthrie, j. t., van meter, p., mccann, a. d., wigfield, a., bennett, l., poundstone, c. c., rice, m. e., fabisch, f., hunt, b., & mitchell, a. (1996). growth of literacy engagement: changes in motivations and strategies during concept-oriented reading instruction. reading research quarterly. 31, 306-332. guthrie, j. t., wigfield, a., & vonsecker, c. (2000). effects of integrated instruction on motivation and strategy use. journal of educational psychology, 92, 331-341. guthrie, j. t., wigfield, a., barbosa, p., perencevich, k. c., taboada, a., davis, m. h., scafiddi, n. t., & tonks, s. (2004). increasing reading comprehension and engagement through concept-oriented reading instruction. journal of educational psychology, 96, 403-423. hall, c., & coles, m. (1999). children's reading choices. new york: routledge. hart, b., & risley, t. r. (1995). meaningful differences in the everyday experiences of young american children. baltimore, md: paul h. brookes. hart, b., & risley, t. r. (2003). the early catastrophe. education review, 17, 110-118. harvey, s., & goudvis, a. (2000). strategies that work. portland, me: stenhouse. hoff, e. (2003). the specificity of environmental influence: socioeconomic status affects early vocabulary development via maternal speech. child development 74, 1368–78. jeong, j., gaffney, j. s., & choi, j.-o. (2010). availability and use of informational texts in second-, third-, and fourth-grade classrooms. research in the teaching of english, 44, 435-456. johnson, d. w., & johnson, r. t. (1989/1990). social skills for successful group work. educational leadership, 47, 29-33. kamil, m., & lane, d. (1998). researching the relationship between technology and literacy: an agenda for the 21st century. in d. reinking, l. mckenna, l. labbo & r. kieffer (eds.), handbook of literacy and technology: transformations in a post-typographic world (pp. 323-341). mahwah, nj: lawrence erlbaum associates. kellett, m. (2009). children as researchers: what can we learn from them about the impact of poverty on literacy opportunities? international journal of inclusive education, 13, 395-408. ketch, a. (2005). conversation: the comprehension connection. the reading teacher, 59, 8-15. international electronic journal of elementary education, 9(3), 523-552, march, 2017 548 kletzien, s. b. (1999, december). children’s reading preferences and information books. paper presented at the national reading conference, orlando, fl. langer, j. a. (1986). children reading and writing: structures and strategies. norwood, nj: ablex. lareau, a. (1989). home advantage: social class and parental intervention in elementary education. lanham, md: rowman & littlefield publishers. lee, v. e., croninger, r. g., & smith, j. b. (1997). course-taking, equity, and mathematics learning: testing the constrained curriculum hypothesis in u.s. secondary schools. educational evaluation and policy analysis, 19, 99-121. ly, c. (2010). more than a library?: urban poverty and an exploratory look at the role of a neighborhood institution. perspectives on urban education, 7, 22-33. mahoney, j. l., larson, r. w., eccles, j., & lord, h. (2005). organized activities as developmental contexts for children and adolescents. in j. l. mahoney, r. w. larson & j. eccles (eds.), organized activities as contexts of development (pp. 3-22). mahwah, nj: erlbaum. many, j., fyfe, r., lewis, g., & mitchell, e. (1996). traversing the topical landscape: exploring students' self-directed reading-writing-research processes. reading research quarterly, 31, 122135. martinez, m. g., roser, n. l., worthy, j., strecker, s., & gough, p. (1997). classroom libraries and children's book selections: redefining "access" in self-selected reading. national reading conference yearbook, 46, 265-272. mcgill-franzen, a., & allington, r. l. (1999). putting books in the classroom seems necessary but not sufficient. journal of educational research, 93, 67. meyer, k. e. (2010). a collaborative approach to reading workshop in the middle years. reading teacher, 63, 501-507. moje, e. b., stockdill, d., kim, k., & kim, h. (2011). the role of text in disciplinary learning. in m. kamil, p. d. pearson, e. b. moje, & p. a. afflerbach (eds.), handbook of reading research (vol. iv, pp. 453-486). new york: taylor & francis. moll, l.c., amanti, c., neff, d., & gonzales, n. (1992). funds of knowledge for teaching: using a qualitative approach to connect homes and classrooms. theory into practice, 31, 132-141. morrow, l. m., pressley, m., smith, j. k., & smith, m. (1997). the effect of a literature-based program integrated into literacy and science instruction with children from diverse background. reading research quarterly, 32, 54-76. morrow, l. m., & weinstein, c. s. (1986). increasing children's use of literature through p rogram and physical design changes. elementary school journal, 83, 131-137. moss, b. (2003). exploring the literature of fact: children's nonfiction trade books in the elementary classroom. new york: guilford press. moss, b. (2008). the information text gap: the mismatch between non-narrative text types in basal readers and 2009 naep recommended guidelines. journal of literacy research, 40, 201-209. moss, g., & mcdonald, j. w. (2004). the borrowers: library records as unobtrusive measures of children's reading preferences. journal of research in reading, 27, 401-412. mullens, j. e., & gaylor, k. (1999). measuring classroom instructional processes: using survey and case study field test results to improve item construction (working paper no. 1999-08). washington, dc: department of education, national center for educational statistics. mullis, i. v. s., martin, m. o., kennedy, a. m., & foy, p. (2007). pirls 2006 international report: iea’s progress in international reading literacy study in primary schools in 40 countries. chestnut hill, ma: timss & pirls international study center, boston college. [available at http://timssandpirls.bc.edu/pirls2006/intl_rpt.html] u.s. fourth graders’ informational text comprehension: indicators from naep / schugar & dreher 549 murphy, p., wilkinson, i. g., soter, a. o., hennessey, m. n., & alexander, j. f. (2009). examining the effects of classroom discussion on students’ comprehension of text: a meta-analysis. journal of educational psychology, 101, 740-764. national assessment governing board. (2008). reading framework for the 2009 national assessment of educational progress. washington, dc: us government printing office. national center for educational statistics (nces) (2007). the nation’s report card: reading 2007 (nces 2007-496). institute of education sciences, u.s. department of education, washington, d.c. national center for educational statistics (nces) (2009). the nation’s report card: reading 2009 (nces 2010-458). institute of education sciences, u.s. department of education, washington, d.c. national center for educational statistics (nces) (2010a). how the samples of schools and students are selected for the main assessments: state and national. retrieved from http://nces.ed.gov/nationsreportcard/about/nathow.asp on august 18, 2010. national center for educational statistics (nces) (2010b). reading: what does the naep reading assessment measure? retrieved from http://nces.ed.gov/nationsreportcard/reading/whatmeasure.asp on august 16, 2010. national center for educational statistics (nces) (2011). interpreting naep reading results. retrieved from http://nces.ed.gov/nationsreportcard/reading/interpret-results.asp on december 31, 2011. national institute of child health and human development. (nichd). (2000). report of the national reading panel. teaching children to read: an evidence-based assessment of the scientific research literature on reading and its implications for reading instruction: reports of the subgroup s. (nih publication no. 00-4769). washington, dc: u.s. government printing office. ness, m. (2011). teachers' use of and attitudes toward informational text in k-5 classrooms. reading psychology, 32, 28-53. neuman, s. b. (1999). books make a difference: a study of access to literacy. reading research quarterly, 34, 286-311. neuman, s. b. (2006). the knowledge gap: implications for early education. in d. k. dickinson & s. b. neuman (eds.), handbook of early literacy research (pp. 29-40). new york: guilford. neuman, s. b. (2010). lessons from my mother: reflections on the national early literacy panel report. educational researcher, 39, 301-304. neuman, s. b., & celano, d. (2001). access to print in low-income and middle-income communities: an ecological study of four neighborhoods. reading research quarterly, 36, 8-27. neuman, s. b., & celano, d. c. (2012). giving our children a fighting chance: poverty, literacy, and the development of information capital. new york: teachers college press. nippold, m. a., duthie, j. k., & larsen, j. (2005). literacy as a leisure activity: free-time preferences of older children and young adolescents. language, speech, and hearing services in schools, 36, 93-102. no child left behind act of 2001. p.l. 107–110. washington, dc: u.s. congress. retrieved from http://www2.ed.gov/policy/elsec/leg/esea02/107-110.pdf oakes, j., & saunders, m. (2002). access to textbooks, instructional materials, equipment, and technology: inadequacy and inequality in california's public schools. los angeles, ca: university of california los angeles. orfield, g. (2001). schools more separate: consequences of a decade of resegregation [electronic version]. rethinking schools, 16. retrieved from http://www.rethinkingschools.org/archive/16_01/seg161.shtml international electronic journal of elementary education, 9(3), 523-552, march, 2017 550 palardy, g. j. (2008). differential school effects among low, middle, and high social class composition schools: a multiple group, multilevel latent growth curve analysis. school effectiveness and school improvement, 19, 21-49. pappas, c. c. (2006). the information book genre: its role in integrated science literacy research and practice. reading research quarterly, 41, 226-250. pease-alvarez, l., & samway, k. d. (2008). negotiating a top-down reading program mandate: the experiences of one school. language arts, 86, 32–41. pett, m. a., lackey, n. r., & sullivan, j. j. (2003). making sense of factor analysis: the use of factor analysis for instrument development in health care research. thousand oaks, ca: sage. pincus, a. r. h. (2005). what’s a teacher to do? navigating the worksheet curriculum. reading teacher, 59, 75-79. portes, p., & salas, s. (2009). poverty and its relation to development and literacy. in l. m. morrow, r. rueda & d. lapp (eds.), handbook of research on literacy and diversity (pp. 97-113). new york: guilford. pressley, m. (2002). effective beginning reading instruction. journal of literacy research, 34, 165188. pressley, m., & afflerbach, p. (1995). verbal protocols of reading: the nature of constructively responsive reading. mahwah, nj: erlbaum. pressley, m., wharton-mcdonald, r., mistretta-hampston, j., & echevarria, m. (1998). literacy instruction in 10 fourthand fifth-grade classrooms in upstate new york. scientific studies of reading, 2, 159-194. project for excellence in journalism (2004). the state of the news media 2004: an annual report on american journalism. retrieved from http://stateofthemedia.org/2004/newspapersintro/audience/ purcell-gates, v. (1995). other people’s words: the cycle of low literacy. cambridge, ma: harvard university press. purcell-gates, v., duke, n. k., & martineau, j. a. (2007). learning to read and write genre-specific text: roles of authentic experience and explicit teaching. reading research quarterly, 42, 8-45. rand reading study group. (2002). reading for understanding: toward an r&d program in reading comprehension. santa monica, ca: rand. reardon, s. f. (2011). the widening academic achievement gap between the rich and the poor: new evidence and possible explanations. in r. murnane & g. duncan (eds.), whither opportunity? rising inequality and the uncertain life chances of low-income children (pp. 91-116). new york: russell sage foundation press. romance, n. r., & vitale, m. r. (1992). a curriculum strategy that expands time for in-depth elementary science instruction by using science-based reading strategies: effects of a year-long study in grade four. journal of research in science teaching, 29, 545-554. rosenblatt, l. m. (1978). the reader, the text, the poem: the transactional theory of the literary work. carbondale, il: southern illinois university press. rowe, m. l. (2008). child-directed speech: relation to socioeconomic status, knowledge of child development and child vocabulary skill. journal of child language, 35, 185-205. samuelstuen, m. s., & bråten, i. (2005). decoding, knowledge, and strategies in comprehension of expository text. scandinavian journal of psychology, 46, 107 –117. sheldon, w. d., & carrillo, r. (1952). relation of parent, home and certain developmental characteristics to children's reading ability. elementary school journal, 52, 262-270. singer, h., mcneil, j. d., & furse, l. l. (1984). relationship between curriculum scope and reading achievement in elementary schools. the reading teacher, 37, 608-612. u.s. fourth graders’ informational text comprehension: indicators from naep / schugar & dreher 551 sirin, s. r. (2005). socioeconomic status and academic achievement: a meta-analytic review of research. review of educational research, 75, 417-453. slavin, r. e. (1986). using student team learning (3rd ed.). baltimore: johns hopkins university, center for research on elementary and middle schools. smith, m. c. (2011). which inand out-of-school factors explain variations in learning across different socio economic groups? findings from south africa. comparative education, 47, 79102. smith, m. s., chudowsky, n., ginsburg, a., hauser, r., jennings, j., & lewis, s. (2012). naep background questions: an underused national resource. retrieved from http://www.nagb.org/publications/expert-panel-naep-bq-report.pdf smith, n. b. (1986). american reading instruction. newark, del.: international reading association. sonnenschein, s., & schmidt, d. (2000). fostering home and community connections to support children's reading development. in l. baker, m. j. dreher, & j. t. guthrie (eds.), engaging young readers: promoting achievement and motivation (pp. 264-284). ny: guilford press. southern education foundation. (2013). a new majority: low income students in the south and west. retrieved from http://www.southerneducation.org/cmspages/getfile.aspx?guid=0bc70ce1-d375-4ff6-8340f9b3452ee088 spear-swerling, l., brucker, p. o., & alfano, m. p. (2010). relationships between sixth-graders’ reading comprehension and two different measures of print exposure. reading and writing, 23, 73-96. taylor, b. m., pearson, p. d., clark, k., & walpole, s. (2000). effective schools and accomplished teachers: lessons about primary-grade reading instruction in low-income schools. elementary school journal, 101, 121-165. timion, c. s. (1992). children's book selection strategies. in j. w. irwin & m. a. doyle (eds.), reading and writing connections: learning from research (pp. 204-222). newark, de: international reading association. united states department of agriculture (usda) (2011). national school lunch program. retrieved from http://www.fns.usda.gov/cnd/lunch/ united states department of education (usde) (2002). public law 107-110: the no child left behind act of 2001 (pp. 670). washington, dc: department of education. united states department of education (usde), institute of educational sciences (ies), and national center for educational statistics (nces) (2007). 2005 naep data companion. washington dc: department of education. valli, l., croninger, r., chambliss, m., graeber, a., & buese, d. (2008). test driven: high-stakes accountability in elementary schools. new york: teachers college press. venezky, r. l. (2000). the origins of the present-day chasm between adult literacy needs and school literacy instruction. scientific studies of reading, 4, 19-39. vitale, m. r., & romance, n. r. (2011). adaptation of a knowledge-based instructional intervention to accelerate student learning in science and early literacy in grades 1 and 2. journal of curriculum and instruction, 5, 79-93. vygotsky, l. s. (1978). mind in society: the development of higher psychological processes. cambridge, ma: harvard university press. weizman, z. o., & snow, c. e. (2001). lexical input as related to children's vocabulary acquisition: effects of sophisticated exposure and support for meaning. developmental psychology, 37, 265279. white, s., chen, j., & forsyth, b. (2010). reading-related literacy activities of american adults: time spent, task types, and cognitive skills used. journal of literacy research, 42, 276-307. international electronic journal of elementary education, 9(3), 523-552, march, 2017 552 wilkinson, i. a. g. (1998). dealing with diversity: achievement gaps in reading literacy among new zealand students. reading research quarterly, 33, 144-167. williams, j. p. (2005). instruction in reading comprehension for primary-grade students: a focus on text structure. journal of special education, 39, 6-18. willson, v. l., & rupley, w. h. (1997). a structural equation model for reading comprehension based on background, phonemic and strategy knowledge. scientific studies of reading, 1, 45-63.